cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Delta Sharing - Unity Catalog difference

Anonymous
Not applicable

Delta Sharing and Unity catalog both have elements of data sharing. Can you please explain when one would use Delta sharing vs Unity Catalog?

1 ACCEPTED SOLUTION

Accepted Solutions

User16826993440
New Contributor III
Unity Catalog provides the capability to discover, audit and govern data assets in one place. This capability does not have to utilize data sharing as it can help you manage your self managed Hive, S3, Azure Data Lake, or Google Cloud storages. If you wanted to perform this management of data across organizations or functions, this is where Delta Sharing would be utilized in conjunction with Unity Catalog, allowing for data to be shared across organizations through Delta and managed/governed through the Unity Catalog.

View solution in original post

6 REPLIES 6

User16826993440
New Contributor III
Unity Catalog provides the capability to discover, audit and govern data assets in one place. This capability does not have to utilize data sharing as it can help you manage your self managed Hive, S3, Azure Data Lake, or Google Cloud storages. If you wanted to perform this management of data across organizations or functions, this is where Delta Sharing would be utilized in conjunction with Unity Catalog, allowing for data to be shared across organizations through Delta and managed/governed through the Unity Catalog.

@Semir Redzepagic​  I use azure and have Dev, Test, Prod subscription. I have multiple databricks workspaces in each subscription. Can the unity catalog in each subscription /environment be separate or the same? ( obviously, i do not want the catalog in dev env to see/access tables in prod environment)

User16826993440
New Contributor III

@prasad vaze​ UC is managed at the account level by the account admins so you have the ability to manage across all workspaces/environments on the account.

The unity catalog is not generally available (or at least it's not enabled in my company's databricks account ) so can not try it out. My question is "can I create separate catalogs between dev/test/prod environments ( multiple catalogs under one account) ? I hope yes but can not try this yet. In each environment, I have multiple databricks workspaces and I understand that one catalog in one environment can administer all workspaces in it. The reason for separate catalog per environment (though all environments are under the same databricks account) is separation of duties and security

User16826993440
New Contributor III

Yes - you can control the data available in each of the environments/workspaces. If there are security questions, please reach out to your Databricks account team to further discuss your specific scenario.

DBXC
Contributor

Based on the Databricks reply from the post below:

"Unity Catalog does not currently support separating data by workspace or Azure subscription. As you noted, data from all catalogs within a region can be accessed by any workspace within that region, and it is up to user permissions to restrict access appropriately."

https://community.databricks.com/s/question/0D58Y00009kfFo4SAE/how-to-segregate-devstagingprod-data-...

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.