Databricks

Arnold_Souza · ‎07-17-2023

Hello,

I would like to change the Metastore location in Databricks Account Console. I have one metastore created that is in an undesired container/storage account. I could see that it's not possible to edit a metastore that is already created. I could also see that it's not possible to create a second metastore in the same Azure region. Well, does anyone know how to move this metastore to another storage account? I was wondering, if I simply move the container to a new Account via Az Copy and then delete and recreate the metatore in the new location, will it work?

Kaniz · ‎07-18-2023

Hi @Arnold_Souza ,

To move a metastore to a new Storage Account in Unity Catalog, you need to follow the below steps:

1. Create a new storage account for Unity Catalog. This must be a Premium performance ADLS Gen2 account in the same region as your Databricks workspaces.
2. Create a container in the new storage account and note the ADLSv2 URI.
3. Create a service principal and generate a client secret for the service principal.
4. Grant the new service principal the Storage blob data contributor role in the storage account.
5. Log in to the account console and click Data.
6. Click the metastore name and then click Edit.
7. Update the ADLS Gen 2 path with the new storage container URI.
8. Click Save to update the metastore with the new storage account.

Sources:
- https://docs.databricks.com/data-governance/unity-catalog/create-metastore.html
- https://docs.databricks.com/data-governance/unity-catalog/get-started.html

Additional Resources

• Docs: uc-metastore-admin-quickstart
• Docs: hive-metastore
• Docs: index

Arnold_Souza · ‎07-18-2023

Hello @Kaniz ,

Thank you for the reply.

As you can see in the image below, there is no step 6. I mean, there is no button to edit the storage path and include a new one.

Because of that, I was wondering: what if I copy the current container to the new Storage account I have, then I delete the current Metastore and recreated it with the new Storage Account address already populated with the contents of the old metastore. Will it work? I would bet the data will be there (tables) but I do not know about the governance side (Table access controls)

mohnishbhagde · ‎07-27-2023

Did you get a solution for this?

Joaquim · ‎08-16-2023

I would like to bump this thread as I am in a similar predicament. New to Databricks here and I have a dev and prod environment in one Databricks account. I created the dev metastore (with "dev" in its name) in the same region where my production metastore would also reside. Dev & prod live in the same region. So my understanding here (because I can't have the metastores in the same region) is that I would have to have dev and prod in separate regions. So I'm assuming that I'm in the same boat where I would need to move my current dev metastore bucket to a different region and am wondering if this would be possible.