Unity Catalog and Data Accessibility

SenthilJ
New Contributor III

Hi,

I got a few question about the internals of #Unity Catalog in #Databricks

1. Understand that we can customize the UC metastore at different levels (catalog/schema). Wondering where is the information about UC permission model stored for every data object (tables/views) in Databricks?

2. Assume the following scenario while using #Azure

  • Databricks Workspaces A and B are under the same region in the US and the same Databricks account registered to a Unity Catalog metastore called "uc-metastore-1". These two workspaces are separated out using their own VNets in Azure.
  • Workspace A connects to Azure ADLS ADL1 and workspace B connects to Azure ADLS ADL2 using their respective access connectors.
  • User X is part of the workspace A and user Y is part of the workspace B. 
  • User X created a data object "X-DB-Table1" and User Y created a data object "Y-DB-Table1" in their respective workspaces. Both are external delta tables from custom storage location
  • Metastore Admin grants User Y access to User X's data object "X-DB-Table1". After the assignment, the User Y is now able to query the table "X-DB-Table1" directly from his Workspace B

What happens under the hood when such querying happens?

  1. How does Workspace B query the table "X-DB-Table1" that's linked to Workspace A using it's own Access Connector. Because the data for "X-DB-Table1" is under the Workspace A network.
  2. Does Unity automatically elevate the privileges of Workspace B to allow access to Workspace A's access connector?

 

 

 

SenthilJ
New Contributor III

thank you @Retired_mod ,your response really helps. A quick follow up - when Unity Catalog uses its permissions to access objects across workspaces, what kind of connection method does it use to access the data object i.e. in this case, when User Y queries the table “X-DB-Table1” from Workspace B ? Also, where is the Unity Catalog's permission metadata (in metastore) physically stored - in Control Plane?