cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Migrate existing Metastore to a new Metastore in same region

Yulei
New Contributor III

Hi, Databricks Community

I am currently plan to do migrate of existing metastore (not in a desired account and name) to a new one (a different desired account) within the same region. I understand it is not as straightforward it might be and complicated.


Can I ask, what might be the best approach to actually fulfil this?
What could be the issue if Metastore and the S3 bucket is set up on different region?
We have set up external tables and connects with existing Metastore, would we lose tehm once we move to the new one?
Can we set up a sync between the new and existing Metastore and S3 bucket, what impact could that lead to?
is there anything that we need to concern about this idea?

1 ACCEPTED SOLUTION

Accepted Solutions

Kaniz_Fatma
Community Manager
Community Manager

Hi @Yulei, Migrating a metastore to a new account can indeed be a complex process. 

 

Here are some answers to your questions based on the information I found:

 

Best Approach to Migrate Metastore: The best approach to migrate a metastore to a new storage accoun.... You would need to create a new storage account, create a container in the new storage account, creat.... However, it’s important to note that you cannot edit a metastore that is already created. Therefore, you might need to delete the current Metastore and recreate it with the new Storage Accou....

 

Metastore and S3 Bucket in Different Regions: If your Metastore and S3 bucket are set up in differen.... It’s generally recommended to locate the bucket in the same region as the workspaces you want to acc....

 

Impact on External Tables: If you have external tables set up and connected with the existing Metastore, you will not lose them when you move to the new one. However, you might need to recreate these tables in the new metastore.

 

Sync Between New and Existing Metastore and S3 Bucket: You can use AWS S3 Replication to automatical...s. This can help in keeping your data synchronized between the new and existing Metastore and S3 bucket. However, it’s important to note that when you make a request to a Multi-Region Access Point, the Mul....

 

Concerns About Migrating Metastore: Some potential concerns could include differences in the Metasto..., and issues related to the migration of on-premises Hadoop cluster to EMR depending on your existin....

 

View solution in original post

3 REPLIES 3

Kaniz_Fatma
Community Manager
Community Manager

Hi @Yulei, Migrating a metastore to a new account can indeed be a complex process. 

 

Here are some answers to your questions based on the information I found:

 

Best Approach to Migrate Metastore: The best approach to migrate a metastore to a new storage accoun.... You would need to create a new storage account, create a container in the new storage account, creat.... However, it’s important to note that you cannot edit a metastore that is already created. Therefore, you might need to delete the current Metastore and recreate it with the new Storage Accou....

 

Metastore and S3 Bucket in Different Regions: If your Metastore and S3 bucket are set up in differen.... It’s generally recommended to locate the bucket in the same region as the workspaces you want to acc....

 

Impact on External Tables: If you have external tables set up and connected with the existing Metastore, you will not lose them when you move to the new one. However, you might need to recreate these tables in the new metastore.

 

Sync Between New and Existing Metastore and S3 Bucket: You can use AWS S3 Replication to automatical...s. This can help in keeping your data synchronized between the new and existing Metastore and S3 bucket. However, it’s important to note that when you make a request to a Multi-Region Access Point, the Mul....

 

Concerns About Migrating Metastore: Some potential concerns could include differences in the Metasto..., and issues related to the migration of on-premises Hadoop cluster to EMR depending on your existin....

 

jose_gonzalez
Moderator
Moderator

Hi @Yulei,

Please follow Kaniz's recommendations. If you still have questions or issues, then lets us know and we will gladly help you to resolve this issue

Yulei
New Contributor III

hi, @Kaniz_Fatma . Thank for the recommendations and links, they are helpful and I am going through them one by one.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group