topic Re: Unity Catalog is set up for multi region in Data Engineering

Unity Catalog is set up for multi region

Phani1 — Thu, 22 Aug 2024 12:22:50 GMT

Hello Team,

I need some clarification on the below diagram . According to the documentation, the Unity Catalog is set up for each region. If we are using multiple clouds, the diagram shows only one Unity Catalog across regions. Shouldn't there be two Unity Catalogs instead?

Regards,

Janga

Re: Unity Catalog is set up for multi region

-werners- — Fri, 23 Aug 2024 08:40:43 GMT

There are indeed 2 metastores, as displayed in the picture, buth use UC.
The different metastores are not merged into one single metastore.
If one needs access to the other you can use Delta Sharing

https://docs.databricks.com/en/data-governance/unity-catalog/best-practices.html

https://docs.databricks.com/en/data-governance/unity-catalog/best-practices.html#share-data-securely-using-delta-sharing

Re: Unity Catalog is set up for multi region

ShankarM — Fri, 23 Aug 2024 09:09:10 GMT

does this mean each metastore is a UC setup for each region?

Re: Unity Catalog is set up for multi region

-werners- — Fri, 23 Aug 2024 09:13:30 GMT

yes, the metastore is the root, linked to physical storage. for each metastore one has to define catalogs, schemas, tables, volumes, permissions etc.
Perhaps in the future databricks will add a META-metastore, over regions, but afaik that is not yet in the pipeline.

Re: Unity Catalog is set up for multi region

ShankarM — Fri, 23 Aug 2024 09:16:06 GMT

thanks for your reply. So shall i assume that each metastore is a UC in the diagram for each region?

Re: Unity Catalog is set up for multi region

-werners- — Fri, 23 Aug 2024 09:21:06 GMT

Yes, the reason why they are grouped into a single rectangle is probably to show that they both are Unity Catalog enabled. It can indeed be confusing, represented like that.
If you want to let them connect to each other, delta sharing or metastore federation (or even define one as an external location) is the way to go.

Re: Unity Catalog is set up for multi region

LettieTaylor — Tue, 10 Sep 2024 09:34:35 GMT

Thank you, I didn't know it before.