Azure Databricks Unity Catalogue Limits
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-29-2023 01:52 PM
In Azure, what are the limits of Unity Catalogue? One UC per region per subscription, or is it one UC per region per tenant?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-29-2023 07:37 PM
One metastore per region per tenant is an enforced soft limit that works for nearly all organisations. You have an additional level of isolation that can be achieved by creating Catalogs inside the Metastore. The recommendation is to use those Catalogs to achieve the isolation you need. You can divide your Catalogs per environment (DEV, Staging, PROD), Business Unit, a mix of both or however you find more useful to your needs.
Each catalog can have a separate LOCATION to store their data in different buckets/containers (S3/ADLS) if required to separate the data in the storage. Also, you can isolate which Catalogs can be accessed by which workspaces, giving you full flexibility.
https://docs.databricks.com/en/data-governance/unity-catalog/best-practices.html
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-30-2023 09:30 AM
thank you for the response, you mentioned "soft limit" so is this something that can be changed?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-26-2024 07:00 AM
Does it mean we will have DEV, TEST, PROD catalog in a single Unity catalog?