cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Can Unity catalog grant the access to a file inside azure datalake storage?

maaaxx
New Contributor III

Hi databricks community,

I have searched quite a while through the internet but did not find an answer. If I have configured the azure datalake connection in Unity data catalog, is it possible to grant the access to users for a specific file or a folder to them? Have seen quite a lot of examples so far for the structured data only.

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

-werners-
Esteemed Contributor III

No.

Unity catalog enforces permissions on the table level (and catalog and schema etc), but not on the storage level.

Unity itself uses a managed identity or service principal for storage access btw. This id should have access to the data lake.

What you can do is create dynamic views to make a row-level security setup.

https://learn.microsoft.com/en-us/azure/databricks/data-governance/table-acls/object-privileges#row-...

View solution in original post

5 REPLIES 5

-werners-
Esteemed Contributor III

No.

Unity catalog enforces permissions on the table level (and catalog and schema etc), but not on the storage level.

Unity itself uses a managed identity or service principal for storage access btw. This id should have access to the data lake.

What you can do is create dynamic views to make a row-level security setup.

https://learn.microsoft.com/en-us/azure/databricks/data-governance/table-acls/object-privileges#row-...

Hubert-Dudek
Esteemed Contributor III

As @werners said service principal needs to have access to the file level.

In the unity catalog, you can use "READ FILES"/"WRITE FILES" permission to give someone the possibility of reading files from the storage level (but through databricks).

maaaxx
New Contributor III

Hi @Hubert Dudekโ€‹ @Werner Stinckensโ€‹ , thank you for the idea. In our scenario, we would need to share the files inside the azure datalake in the same folder.

Imagine that we have a folder ORDER001 and file1, file2 and file3. Can we use databricks to share the access to user A the access of file1 and file2 but for user B the access to file3?

Some people have suggested to copy the files outside and create separate container. However, this will unavoidably create duplication and we would like to avoid.

Have you an idea how the acsess control in this scenario could be achieve through databricks?

Many thanks

Hubert-Dudek
Esteemed Contributor III

It is messy as:

  • Files are in the same folder (so it complicates using an external location and read-write permission)
  • Unity Catalog is designed to have tables, and you grant access to tables

I don't know what the files are. Unstructured data can be included in the delta file / metastore table (array or binary).

You could also put these files outside of databricks and manage access separately.

@Werner Stinckens,โ€‹ is it possible to have Unity Catalog and mount another storage container under the dbfs path using credentials passthrough?

-werners-
Esteemed Contributor III

I am not sure. Someone at Databricks once told me that mounts and Unity are not friends.

The easiest way to achieve this on file level is either:

  • not using Unity and use AAD credential passtrough. then define the file access with ACLs on the data lake.
  • forget about the file access and use dynamic views f.e. to create row level security.

Frankly using ACLs always gets on my nerves. Hard to maintain.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group