Fine grained control of volumes

ossinova — Thu, 22 Feb 2024 15:33:26 GMT

Is it possible to provide fine grained control (folder level/file level) for a given volume?

I have two SCIM integrated groups who have read volume access at the catalog level, but those two groups need different permissions on a lower level. Preferably at a folder or file level within the volume.

Volume:

(Top level)

landing/

(Inner level)

- landing/PDF (group 1 needs access only)
- landing/CSV (group 2 needs access only)

Is it possible to achieve this without having to mount the inner levels as a top level or creating X schemas? Any recommendations are highly appreciated.

Re: Fine grained control of volumes

Sidhant07 — Thu, 21 Mar 2024 15:53:47 GMT

Yes, it is possible to provide fine-grained control at the folder or file level within a volume in Databricks Unity Catalog. You can achieve this by creating managed or external volumes in the Unity Catalog and granting specific groups or users access to the desired directories or files within the volume. With managed volumes, you can create governed storage for working with files without the need for configuring access to cloud storage, while external volumes allow you to add governance to existing cloud object storage directories.To create a managed volume, you can use the CREATE VOLUME command in SQL or the Catalog Explorer UI. For example:

CREATE VOLUME <catalog>.<schema>.<volume-name>;

To create an external volume, you can specify the location within an external location using the CREATE EXTERNAL VOLUME command in SQL or the Catalog Explorer UI. For example:

CREATE EXTERNAL VOLUME <catalog>.<schema>.<external-volume-name> LOCATION 's3://<external-location-bucket-path>/<directory>';

Once the volumes are created, you can grant permissions to specific groups or users using the GRANT command in SQL. For example:

GRANT READ VOLUME, WRITE VOLUME ON VOLUME <volume-name> TO <group-name>;

You can then access and work with the files in the the volume using SQL, %fs magic command, Databricks utilities, or other libraries. The path to access files in volumes follows the format: /Volumes/<catalog>/<schema>/<volume>/<path>/<file-name> or dbfs:/Volumes/<catalog>/<schema>/<volume>/<path>/<file-name>.

https://docs.databricks.com/data-governance/unity-catalog/best-practices.html, https://docs.databricks.com/connect/unity-catalog/volumes.html,

https://docs.databricks.com/discover/files.html, https://databricks.com/blog/announcing-public-preview-volumes-databricks-unity-catalog)

Re: Fine grained control of volumes

rkalluri-apex — Wed, 27 Mar 2024 22:31:56 GMT

Can you define the external location at the Landing level and create two Volumes one for PDF and other for CSV and provide access to the respective groups 1 and 2.

topic Re: Fine grained control of volumes in Data Governance

Fine grained control of volumes

Re: Fine grained control of volumes

Re: Fine grained control of volumes