- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-01-2025 12:31 PM - edited 09-01-2025 12:32 PM
@Coffee77 From the docs, I thought it was 1 metastore per region and 1 Unity Catalog mapped to that:https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/best-practices#meta...
You then have as many workspaces as needed within that region. Each workspace contains the 3-level namespace, catalog->schema->table. You can have as many catalogs (top level-name space) as you require. You can host your bronze silver gold within a single workspace, i.e. workspace called "dev". Perhaps you guys already know about this region constraint?
I think Unity Catalog abstracts the storage part away from us. Essentially, it sits on top of a single ADLS2, as you mention. Databricks nicely handles the storage under-the-hood for us i.e containers.
I think the time you'd need to consider additional ADLS in when we exceed things like this:
https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/best-practices#mana...
That is a really interesting point and one I wouldn't have considered without you raising it. I do fear it's easy to overcomplicate a design, though. Thanks for bringing this type of thing to my attention @Coffee77
I must say, I'm pleasantly surprised to find these finer details in the documentation provided by Databricks.
All the best,
BS