How to Import an old Unity Catalog Metastore to a new Unity Catalog Metastore?

[Barney]

Hi All,

I have created a new Unity Catalog (UC) metastore (new metastore ID) in west europe and now i want to import alle catalog/schema/tables form the old metastore (west europe) container to the new UC metastore.

Note: coping all tables form the old metastore container to the new metastore container didn't works.

[Anonymous]

@Ron Barneveld​ :

To import all the catalog/schema/tables from the old Unity Catalog (UC) metastore to the new one, you can use the 'Export Metadata' and 'Import Metadata' functionalities provided by Databricks.

Here are the high-level steps to do this:

• Export metadata from the old metastore container to a file. You can do this using the databricks workspace export command in the Databricks CLI

databricks workspace export /FileStore/my-metadata-folder/metadata.json

• Copy the exported file to a location that is accessible from the new Databricks workspace
• Import metadata from the exported file to the new metastore container. You can do this using the databricks workspace import command in the Databricks CLI. This command will import all the objects defined in the metadata file, including notebooks, libraries, and other artifacts. If you only want to import the catalog/schema/tables, you can filter the metadata file to include only the relevant objects before importing it

databricks workspace import /FileStore/my-metadata-folder/metadata.json

• Verify that the catalog/schema/tables have been imported correctly by checking the Unity Catalog UI in the new Databricks workspace

[Barney]

Hi Suteja,

Thanks for your reply. You mention these: databricks workspace export /FileStore/my-metadata-folder/metadata.json but the statement should be like this: databricks workspace export [OPTIONS] SOURCE_PATH TARGET_PATH , right? There must be a SOURCE_PATH of the UC metastore, so where can i find the metadata of the UC metastore that i can export. It doesn't exist in the Databricks workspace filefolder.

By the way i still have my old UC metastore in the ASDL.

[Anonymous]

Hi @Ron Barneveld​ 

Hope everything is going great.

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you. 

Cheers!

[Barney]

Hi Vidula,

I am afraid that our issue is still not resolve. We have created the default UC metastore on prod.

We understand that we wouldn't delete a metastore on daily bases but we still interesting if it's possible to export/import the metadata of an UC metastore.

Thanks any for your interest!

Ron (Barney)

[drii_cavalcanti]

Hi Barney,

Just wondering if you got this issue solved.

Thank you so much,
Adriana Cavalcanti

[Kaniz]

Hi @Barney, To import all catalog/schema/tables from the old metastore to the new Unity Catalog (UC) metastore, you can use the Data Explorer upgrade wizard provided by Databricks.

Here are the steps:

1. Open the Data Explorer by clicking Data in the sidebar.
2. Select hive_metastore as your catalog and select the schema (database) that you want to upgrade.
3. Click **Upgrade** at the top right of the schema detail view.
4. Select all of the tables that you want to upgrade and click **Next**. Only external tables in formats supported by Unity Catalog can be upgraded using the upgrade wizard.
5. Set the destination catalog, schema (database), and owner for each table. Users will be able to access the newly created table in the context of their privileges on the catalog and schema.
6. Review the table configurations. To modify them, click the **Previous** button.
7. Click **Create Query for Upgrade**. A query editor appears with generated SQL statements.
8. Run the query. When the query is done, each table's metadata has been copied from Hive metastore to Unity Catalog. These tables are marked as upgraded in the upgrade wizard.
9. Define fine-grained access control using the **Permissions** tab of each new table.
10. Modify your workloads to use the new table. 

Please note that you need to have CREATE EXTERNAL TABLE permission for the external locations of the tables to be upgraded.

Source: [Docs: migrate](https://docs.databricks.com/data-governance/unity-catalog/migrate.html)

[prasad_vaze]

@Kaniz  @Barney Will this method work to import table and column metadata from one metasrore to another in unity catalg? I thought this only works when upgrading tables from legacy hive_metastore to a catalog in unity-catalog ?   

I have a spreadheet containing table & column descriptions (comments)  Is there a way to upload this against the schema in unity catalog?  Basically instead of running  'Alter table <> alter column <>  comment "description"  '  these commands for every column, is there a way to mass apply column comments in unity catalog schema?