topic Re: Unity catalogues - What would you do in Get Started Discussions

Unity catalogues - What would you do

Rezakorehi — Tue, 07 Oct 2025 00:56:56 GMT

If you were creating Unity Catalogs again, what would you do differently based on your past experience?

Re: Unity catalogues - What would you do

nayan_wylde — Tue, 07 Oct 2025 03:07:24 GMT

That’s a great question. Unity Catalog have changed the entire data governance framework and I would do the following things to make UC projects successful based on my past experience of implementing Unity Catalog.

1. Design the metastore hierarchy early on

Initially, we created catalogs and schemas ad-hoc as teams onboarded.
I’d establish a clear naming convention and ownership model before rollout — for example:

Catalogs aligned with business domains (e.g., sales, finance, marketing)
Schemas mapped to data zones or data products.
This prevents rework and ensures consistent data lineage and access policies.

2. Implement access control as code from day one

Early setups often involved manual grants using the Databricks UI or SQL commands.
I’d now automate permission management using Terraform or Databricks APIs, SDK, DAB etc. treating Unity Catalog permissions as version-controlled code.
This improves auditability and simplifies onboarding of new datasets.

3. Adopt a data ownership and stewardship model upfront

Initially, central data teams managed all permissions, which quickly became a bottleneck.
I’d define data owners and stewards per catalog or schema right from the start and delegate privileges accordingly through groups.Like make Business Owners manage the group permissions

4. Integrate Unity Catalog with external identity and policy systems early

Integration with Azure AD SCIM for group-based access.

5. Plan for cross-workspace and cross-region access

Unity Catalog enables sharing across workspaces, but initial setups often ignored multi-region or DR needs.
I’d now design catalogs with data residency and cross-region replication in mind.

6. Involve stakeholders early

Instead of just IT-led design, I’d include data producers, consumers, and compliance teams during catalog structuring to ensure business relevance and smooth adoption.

Re: Unity catalogues - What would you do

Hubert-Dudek — Tue, 07 Oct 2025 07:34:05 GMT

Hi, just always keep it on seperate storage container 🙂

Re: Unity catalogues - What would you do

szymon_dybczak — Tue, 07 Oct 2025 07:54:07 GMT

From my experiance:

- don't create separate catalogs for every project. Try to think about your desing before implementation

- try to come up with consistent naming convention to avoid cognitive overhead

- principle of least privilege - grant users and groups the minimum permissions necessary for their tasks, leveraging Unity Catalog’s hierarchical privilege inheritance.

Re: Unity catalogues - What would you do

Hubert-Dudek — Wed, 15 Oct 2025 19:16:05 GMT

@szymon_dybczak, I recently heard about a use case where the company is automatically creating a new catalog for every employee... but of course I agree with you.

Re: Unity catalogues - What would you do

nayan_wylde — Wed, 15 Oct 2025 20:03:58 GMT

@Hubert-Dudek Is this a recommended way by databricks? Creating catalogs for each employees will only Complexity explosion: Hard to manage permissions, auditing, and lifecycle.
Performance overhead: Catalog metadata operations scale poorly with excessive catalogs. I would love to see how organisations tackle those challenges.

Re: Unity catalogues - What would you do

szymon_dybczak — Thu, 16 Oct 2025 09:47:52 GMT

Haha, that’s for sure an unconventional approach, but maybe they had a good reason for doing that 😃. In one of the projects I’ve participated in, we did a similar thing, but only for developers. They wanted to have an isolated development environment.

Re: Unity catalogues - What would you do

Hubert-Dudek — Fri, 17 Oct 2025 10:56:17 GMT

@nayan_wylde no don't do that hehe. It was example of extreme approach. Usually use catalog to separate environment + in enterprises to separate divisions like customer tower, marketing tower, finance tower etc