topic Restricting Catalog and External Location Visibility Across Databricks Workspaces in Administration & Architecture

Restricting Catalog and External Location Visibility Across Databricks Workspaces

yvishal519 — Mon, 19 Aug 2024 11:24:53 GMT

Hi Databricks Community,

I need some guidance regarding catalogs and external locations across multiple environments. Here's my situation:

I've set up a resource group (dev-rg) and created a Databricks workspace where I successfully created catalogs (bronze, silver, gold) and external locations using storage credentials. However, I've now created a new resource group (qa-rg) with a new Databricks workspace. Surprisingly, I can see all the external locations and catalogs from the dev environment in the qa environment.

My goal is to create the same catalogs and external locations in the qa environment while keeping the same naming convention across all environments (dev, qa, and prod). However, the catalogs and external locations from the dev environment are also visible in the qa environment, and any new ones I create in qa are also visible in dev. The same issue persists in the prod environment.

I understand that I can restrict access through workspace settings to prevent cross-workspace access, but I need to ensure that catalogs and external locations are not visible across different workspaces. For example, I want to prevent the dev catalog from being visible in the qa environment and vice versa, even though there are some common users across environments.

How can I ensure that catalogs and external locations are isolated to their respective workspaces while maintaining the same naming convention across environments? Any advice or best practices for achieving this would be greatly appreciated.

Thanks in advance for your help!

Re: Restricting Catalog and External Location Visibility Across Databricks Workspaces

menotron — Mon, 19 Aug 2024 13:06:14 GMT

Hi @yvishal519, have you assigned the same metastore to both the workspace?
Ideally, that should be okay as long you have control on who has access to what workspace and use what catalog.
You can't have the same names though. Easier approach would be to name the catalogs with the environment identifier (dev_bronze, qa_silver, etc) and manage permissions in UC.

But as I understand you are trying to stand the two environments in separate resource groups and maintaining the same naming conventions, you can consider creating a separate metastore, the top-level container for metadata in Unity Catalog for each environment.

Re: Restricting Catalog and External Location Visibility Across Databricks Workspaces

Retired_mod — Tue, 20 Aug 2024 12:38:21 GMT

Hi @yvishal519, Thanks for reaching out! Please review the response and let us know if it answers your question. Your feedback is valuable to us and the community.

If the response resolves your issue, kindly mark it as the accepted solution. This will help close the thread and assist others with similar queries.

We appreciate your participation and are here if you need further assistance!

Re: Restricting Catalog and External Location Visibility Across Databricks Workspaces

eshwari — Tue, 28 Oct 2025 17:30:08 GMT

I am facing exact similar issue. I don't want to create separate metastore. and I have added environment name as a prefix to all external locations. All the locations are restricted to their workspaces, so functionality wise everything is fine. my concern is visibility, even though access is restricted, all external locations are still visible across all workspaces.

Is there any way to hide or scope external locations and catalogs so they are only visible within their respective workspaces, without creating separate metastores?