Delta Sharing - Unity Catalog difference

[Anonymous]

Delta Sharing and Unity catalog both have elements of data sharing. Can you please explain when one would use Delta sharing vs Unity Catalog?

[User16826993440 · Accepted solution]

Unity Catalog provides the capability to discover, audit and govern data assets in one place. This capability does not have to utilize data sharing as it can help you manage your self managed Hive, S3, Azure Data Lake, or Google Cloud storages. If you wanted to perform this management of data across organizations or functions, this is where Delta Sharing would be utilized in conjunction with Unity Catalog, allowing for data to be shared across organizations through Delta and managed/governed through the Unity Catalog.

[prasadvaze]

@Semir Redzepagic​  I use azure and have Dev, Test, Prod subscription. I have multiple databricks workspaces in each subscription. Can the unity catalog in each subscription /environment be separate or the same? ( obviously, i do not want the catalog in dev env to see/access tables in prod environment)

[User16826993440]

@prasad vaze​ UC is managed at the account level by the account admins so you have the ability to manage across all workspaces/environments on the account.

[prasadvaze]

The unity catalog is not generally available (or at least it's not enabled in my company's databricks account ) so can not try it out. My question is "can I create separate catalogs between dev/test/prod environments ( multiple catalogs under one account) ? I hope yes but can not try this yet. In each environment, I have multiple databricks workspaces and I understand that one catalog in one environment can administer all workspaces in it. The reason for separate catalog per environment (though all environments are under the same databricks account) is separation of duties and security

[User16826993440]

Yes - you can control the data available in each of the environments/workspaces. If there are security questions, please reach out to your Databricks account team to further discuss your specific scenario.

[DBXC]

Based on the Databricks reply from the post below:

"Unity Catalog does not currently support separating data by workspace or Azure subscription. As you noted, data from all catalogs within a region can be accessed by any workspace within that region, and it is up to user permissions to restrict access appropriately."

https://community.databricks.com/s/question/0D58Y00009kfFo4SAE/how-to-segregate-devstagingprod-data-...