Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-08-2025 08:09 AM
Unity Catalog (UC) tracks the metadata and your cloud storage accounts store the your data. This python script will extract the metadata from {catalog}.information_schema into folders in a storage location. Take this and put into a notebook.
Make data backups from the cloud storage console. {catalog} can also include system, which will cover every catalog in UC
%python
storage_location = dbutils.widgets.get("storageLocation")
catalog_name = dbutils.widgets.get("catalogName")
(storage_location, catalog_name)
%python
#
# For every table in information schema, make a backup copy of it to the storage location and all of the other metadata
#
table_list = spark.catalog.listTables(f"`{catalog_name}`.information_schema")
for table in table_list:
print(f'backing up {table.catalog}.information_schema.{table.name} to {storage_location}/{table.name}...')
info_schema_table_df = spark.sql(f"SELECT * FROM {table.catalog}.information_schema.{table.name}")
info_schema_table_df.write.format("delta").mode("overwrite").save(f"{storage_location}/{table.name}")
print('backup complete')
