topic Re: Unity catalog and metadata to backup site in Administration & Architecture

Unity catalog and metadata to backup site

adb-rm — Sat, 08 Jun 2024 16:34:40 GMT

Hi ,

I am looking for Databricks DR(Disaster recovery) site creation. My primary setup is in west us and my primary data bricks created in primary site with unity catalog enable. with same meta data i want bring up the Databricks in my secondary side east us when any event occurred.

For notebooks, general data i have approach but i have blocked with Unity catalog data replication [Meta store/Catalog/Schema/Tables].

can some one share approach if you already build the data bricks DR (secondary ) site.

Thanks

Re: Unity catalog and metadata to backup site

Walter_C — Sat, 08 Jun 2024 20:43:03 GMT

As of now UC does not have DR setting, this is currently in our prioritized roadmap but no ETA is currently available.

Some customers host an external metastore and replicate it across regions to achieve this.

Re: Unity catalog and metadata to backup site

Pranama — Thu, 13 Jun 2024 04:24:54 GMT

We have done it that way where we have a unity catalog created in secondary region with its own single instance of meta store. Given unity catalog replication is not in place, we run a nightly replication to sync up meta data in primary and secondary.

Re: Unity catalog and metadata to backup site

adb-rm — Sat, 15 Jun 2024 17:28:26 GMT

Thank you for your reply. I have replicated the metastore but not able to re-use meta store data in Secondary region. When creating unity catalog in secondary region, I dint got option to re-use metastore data and only have option to create new metastore. please can you elaborate how re using the replicated metastore and catalog data in secondary region.

Thanks

Re: Unity catalog and metadata to backup site

adb-rm — Sat, 15 Jun 2024 17:30:12 GMT

Thans for the information.