Migrate existing Metastore to a new Metastore in same region

[Yulei]

Hi, Databricks Community

I am currently plan to do migrate of existing metastore (not in a desired account and name) to a new one (a different desired account) within the same region. I understand it is not as straightforward it might be and complicated.

Can I ask, what might be the best approach to actually fulfil this?
What could be the issue if Metastore and the S3 bucket is set up on different region?
We have set up external tables and connects with existing Metastore, would we lose tehm once we move to the new one?
Can we set up a sync between the new and existing Metastore and S3 bucket, what impact could that lead to?
is there anything that we need to concern about this idea?

[Kaniz · Accepted solution]

Hi @Yulei, Migrating a metastore to a new account can indeed be a complex process. 

Here are some answers to your questions based on the information I found:

Best Approach to Migrate Metastore: The best approach to migrate a metastore to a new storage accoun.... You would need to create a new storage account, create a container in the new storage account, creat.... However, it’s important to note that you cannot edit a metastore that is already created. Therefore, you might need to delete the current Metastore and recreate it with the new Storage Accou....

Metastore and S3 Bucket in Different Regions: If your Metastore and S3 bucket are set up in differen.... It’s generally recommended to locate the bucket in the same region as the workspaces you want to acc....

Impact on External Tables: If you have external tables set up and connected with the existing Metastore, you will not lose them when you move to the new one. However, you might need to recreate these tables in the new metastore.

Sync Between New and Existing Metastore and S3 Bucket: You can use AWS S3 Replication to automatical...s. This can help in keeping your data synchronized between the new and existing Metastore and S3 bucket. However, it’s important to note that when you make a request to a Multi-Region Access Point, the Mul....

Concerns About Migrating Metastore: Some potential concerns could include differences in the Metasto..., and issues related to the migration of on-premises Hadoop cluster to EMR depending on your existin....

[jose_gonzalez]

Hi @Yulei,

Please follow Kaniz's recommendations. If you still have questions or issues, then lets us know and we will gladly help you to resolve this issue

[Yulei]

hi, @Kaniz . Thank for the recommendations and links, they are helpful and I am going through them one by one.