topic UC Enablement in Databricks workspace for metastores in different region - Azure Cloud in Data Governance

UC Enablement in Databricks workspace for metastores in different region - Azure Cloud

Nivethan — Wed, 24 Apr 2024 18:04:48 GMT

We have a UC enabled on Databricks where we have two Metastores,

Metastore in West Europe region
Metastore in North America region

And Databricks workspace.

Workspace in West Europe region

And we want to have a single Databricks workspace for both Metastores in different region.

Existing Solution from Databricks:

Ideally, in Databricks we can’t have Metastore and workspace in different regions, also we can’t have multiple Metastores for one single workspace.

What could be potentially done?

But is there any other way / workaround that could be done inorder to access the different region metastore data in single region workspace?

Re: UC Enablement in Databricks workspace for metastores in different region - Azure Cloud

Yeshwanth — Thu, 25 Apr 2024 06:27:11 GMT

@Nivethan

A metastore is the top-level container for data in Unity Catalog. Unity Catalog metastores register metadata about securable objects (such as tables, volumes, external locations, and shares) and the permissions that govern access to them. Each metastore exposes a three-level namespace (catalog.schema.table) by which data can be organized. You must have one metastore for each region in which your organization operates. To work with Unity Catalog, users must be on a workspace that is attached to a metastore in their region.

Reference Doc: https://docs.databricks.com/en/data-governance/unity-catalog/create-metastore.html#:~:text=A%20metastore%20is,in%20their%20region.

Thank you.

Re: UC Enablement in Databricks workspace for metastores in different region - Azure Cloud

dkushari — Sat, 11 May 2024 19:49:04 GMT

Hi,

Cross region metastore and workspace (WS) is not possible. It is important to understand the use case and why you have certain data in a region where you do not have a compute (WS) or do not need a compute. Ideally have each region has their own WS and metastore and then use delta sharing for sharing data across, if data sharing is the use case here.