Migrating Databricks Metastore Between Accounts

[r_w_]

Is there a way to migrate a metastore to a different Databricks account in Databricks?

If not, what alternative approaches can be considered? I can think of the following two:

• Using Delta Sharing to share objects from the source to the destination, and then creating new tables, etc., in the destination using CTAS.
• Exporting table data from the source as CSV or Parquet files, and then importing those files into the destination.

In both cases, my understanding is that schema and permission settings would need to be reconfigured.

If there are any other better methods, I would appreciate it if you could let me know.

[iyashk-DB]

With Terraform you can automate almost all of the destination setup and much of the migration workflow, but there isn’t a single Terraform resource that “moves a metastore” across accounts. But it is ideal for provisioning the target Unity Catalog, registering data, setting up sharing, and reapplying grants; actual data copy/materialization is done by jobs or SQL that Terraform can orchestrate.

Terraform does not itself “migrate” all tables and data. You still need to materialise data in the destination via CREATE TABLE CLONE (preferred for Delta) or CTAS for other formats, or re-register paths as external tables; Terraform can create the jobs/warehouses/pipelines that run those statements.

[Masood_Joukar]

Hi @r_w_ 

CSV is a an export, what about governance ? what about if the exported data tomorrow has some changes in schema or some updates?

in a similar scenario in a project we picked a Foreign Catalog. if your existing Catalogs are standard in source, then you may need to simply create a Foreign Catalog, push the data, you are going to share with source (I assume the source is also Databricks User), to the Foreign Catalog then share with source  ":

Foreign catalog: a Unity Catalog object that is used only in Lakehouse Federation scenarios. A foreign catalog mirrors a database in an external data system, enabling you to perform read-only queries on that data system in your Databricks workspace.What is Lakehouse Federation?.

[Masood_Joukar]

@r_w_  below with a little more clarity 🙂

in a similar scenario in a project we picked a Foreign Catalog. if your existing Catalogs in source are standard, then you may need to simply create a Foreign Catalog, push the data, you are going to share with target (I assume the target is also Databricks User), to the Foreign Catalog then share with target

[nayan_wylde · Accepted solution]

Databricks does not provide a built‑in way to “move” or migrate a Unity Catalog (UC) metastore from one Databricks account to another.

Here is the list of activities that you can try

Inventory & plan:

1.  Enumerate catalogs, schemas, tables (Delta vs. external), views (incl. dynamic views), volumes, models, notebooks to move.
2. Identify tables on DBFS root and migrate them to cloud storage before the move. [stackoverflow.com]

Stand up destination objects with Terraform:

1. Create metastore, storage credentials, external locations, catalogs/schemas.
2. Define grants with databricks_grants resources.

Move data: 

1. Preferred: Delta Sharing from source → destination; then DEEP CLONE or CTAS to materialize.
2. Alternative: Export files (Delta/Parquet/CSV) to shared cloud storage, then register in destination. (Avoid DBFS root.)

Recreate dependent objects: 

1. Views/materialized views: recreate with destination names/paths.
2. Streaming tables: consider deep clone to carry stream metadata, or rebuild pipelines as needed.

Reapply governance: 

1. Use Terraform (or SQL) to apply catalog/schema/table grants; if you used shares, governance on shared assets follows UC rules.