topic Re: Azure Databricks Unity Catalogue Limits in Data Governance

Azure Databricks Unity Catalogue Limits

sqlshep — Fri, 29 Sep 2023 20:52:28 GMT

In Azure, what are the limits of Unity Catalogue? One UC per region per subscription, or is it one UC per region per tenant?

Re: Azure Databricks Unity Catalogue Limits

btafur — Sat, 30 Sep 2023 02:37:05 GMT

One metastore per region per tenant is an enforced soft limit that works for nearly all organisations. You have an additional level of isolation that can be achieved by creating Catalogs inside the Metastore. The recommendation is to use those Catalogs to achieve the isolation you need. You can divide your Catalogs per environment (DEV, Staging, PROD), Business Unit, a mix of both or however you find more useful to your needs.

Each catalog can have a separate LOCATION to store their data in different buckets/containers (S3/ADLS) if required to separate the data in the storage. Also, you can isolate which Catalogs can be accessed by which workspaces, giving you full flexibility.

https://docs.databricks.com/en/data-governance/unity-catalog/best-practices.html

Re: Azure Databricks Unity Catalogue Limits

sqlshep — Sat, 30 Sep 2023 16:30:07 GMT

thank you for the response, you mentioned "soft limit" so is this something that can be changed?

Re: Azure Databricks Unity Catalogue Limits

SiddeshSamarth — Fri, 26 Jan 2024 15:00:29 GMT

Does it mean we will have DEV, TEST, PROD catalog in a single Unity catalog?