Migration and deletion of metastore

[OwenDB]

Good afternoon,

We are in the process of setting up Unity Cat, we started in our existing workspace and set it up fine. We are particularly interested in federating existing warehouses at this point. We were able to federate one datawarehouse (dwh1) but to federate the other (dwh2) we will need to create a new workspace that will allow a VNET peering to connect to it. This is fine, we have set this up.

So now we have :

Workspace 1 >

• Existing notebooks & code
• Hive metastore with production data and analysis tables
• Empty Unity Catalogue with connection to dwh1

Workspace 2 >

• Empty but set up in correct space in Azure to VNET peer to dwh2

When we go to set up Unity Cat in Workspace 2 we are told that we can only have one metastore per region.

Does this one metastore include our old Hive metastore?

I.E. can we delete the half set-up unity cat in Workspace 1, set up unity cat in Workspace 2, then slowly migrate existing code and notebooks till I can switch the production work to the new workspace?

Thanks,

[Kaniz]

Hi @OwenDB, Yes, you’re correct. Azure Databricks allows only one Unity Catalog metastore per region. This limit includes your existing Hive metastore.

You can migrate your Hive metastore to Unity Catalog. Databricks provides different methodologies for upgrading your Hive metastore tables to Unity Catalo...3. After upgrading to Unity Catalog, you can drop Hive metastore schemas and tables if you no longer ne....

As for migrating your existing code and notebooks, Azure Databricks provides options to export and i.... You can also use Databricks Labs’ migration package to move resources between Databricks Workspaces.

[OwenDB]

Thanks Kaniz,

Can I check those links are ok before I reply? Would like to read everything but the link seems to point to the same article.

Thanks

Owen.

[Kaniz]

Yeah, Sure!

[OwenDB]

is it just these two I should be reading? These are the two linked from above
https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/create-metastore 
https://www.databricks.com/blog/migrating-tables-hive-metastore-unity-catalog-metastore 

Thanks

[Kaniz]

Yes.