Unity Catalog and Data Accessibility

SenthilJ — Wed, 17 Jan 2024 13:49:19 GMT

Hi,

I got a few question about the internals of #Unity Catalog in #Databricks

1. Understand that we can customize the UC metastore at different levels (catalog/schema). Wondering where is the information about UC permission model stored for every data object (tables/views) in Databricks?

2. Assume the following scenario while using #Azure

Databricks Workspaces A and B are under the same region in the US and the same Databricks account registered to a Unity Catalog metastore called "uc-metastore-1". These two workspaces are separated out using their own VNets in Azure.
Workspace A connects to Azure ADLS ADL1 and workspace B connects to Azure ADLS ADL2 using their respective access connectors.
User X is part of the workspace A and user Y is part of the workspace B.
User X created a data object "X-DB-Table1" and User Y created a data object "Y-DB-Table1" in their respective workspaces. Both are external delta tables from custom storage location
Metastore Admin grants User Y access to User X's data object "X-DB-Table1". After the assignment, the User Y is now able to query the table "X-DB-Table1" directly from his Workspace B

What happens under the hood when such querying happens?

How does Workspace B query the table "X-DB-Table1" that's linked to Workspace A using it's own Access Connector. Because the data for "X-DB-Table1" is under the Workspace A network.
Does Unity automatically elevate the privileges of Workspace B to allow access to Workspace A's access connector?

Re: Unity Catalog and Data Accessibility

SenthilJ — Fri, 19 Jan 2024 04:55:22 GMT

thank you @Retired_mod ,your response really helps. A quick follow up - when Unity Catalog uses its permissions to access objects across workspaces, what kind of connection method does it use to access the data object i.e. in this case, when User Y queries the table “X-DB-Table1” from Workspace B ? Also, where is the Unity Catalog's permission metadata (in metastore) physically stored - in Control Plane?

topic Re: Unity Catalog and Data Accessibility in Data Engineering

Unity Catalog and Data Accessibility

Re: Unity Catalog and Data Accessibility