cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Delta Sharing - Unity Catalog difference

Anonymous
Not applicable

Delta Sharing and Unity catalog both have elements of data sharing. Can you please explain when one would use Delta sharing vs Unity Catalog?

1 ACCEPTED SOLUTION

Accepted Solutions

User16826993440
New Contributor III
Unity Catalog provides the capability to discover, audit and govern data assets in one place. This capability does not have to utilize data sharing as it can help you manage your self managed Hive, S3, Azure Data Lake, or Google Cloud storages. If you wanted to perform this management of data across organizations or functions, this is where Delta Sharing would be utilized in conjunction with Unity Catalog, allowing for data to be shared across organizations through Delta and managed/governed through the Unity Catalog.

View solution in original post

6 REPLIES 6

User16826993440
New Contributor III
Unity Catalog provides the capability to discover, audit and govern data assets in one place. This capability does not have to utilize data sharing as it can help you manage your self managed Hive, S3, Azure Data Lake, or Google Cloud storages. If you wanted to perform this management of data across organizations or functions, this is where Delta Sharing would be utilized in conjunction with Unity Catalog, allowing for data to be shared across organizations through Delta and managed/governed through the Unity Catalog.

@Semir Redzepagicโ€‹  I use azure and have Dev, Test, Prod subscription. I have multiple databricks workspaces in each subscription. Can the unity catalog in each subscription /environment be separate or the same? ( obviously, i do not want the catalog in dev env to see/access tables in prod environment)

User16826993440
New Contributor III

@prasad vazeโ€‹ UC is managed at the account level by the account admins so you have the ability to manage across all workspaces/environments on the account.

The unity catalog is not generally available (or at least it's not enabled in my company's databricks account ) so can not try it out. My question is "can I create separate catalogs between dev/test/prod environments ( multiple catalogs under one account) ? I hope yes but can not try this yet. In each environment, I have multiple databricks workspaces and I understand that one catalog in one environment can administer all workspaces in it. The reason for separate catalog per environment (though all environments are under the same databricks account) is separation of duties and security

User16826993440
New Contributor III

Yes - you can control the data available in each of the environments/workspaces. If there are security questions, please reach out to your Databricks account team to further discuss your specific scenario.

DBXC
Contributor

Based on the Databricks reply from the post below:

"Unity Catalog does not currently support separating data by workspace or Azure subscription. As you noted, data from all catalogs within a region can be accessed by any workspace within that region, and it is up to user permissions to restrict access appropriately."

https://community.databricks.com/s/question/0D58Y00009kfFo4SAE/how-to-segregate-devstagingprod-data-...

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group