Access Control in hive_metastore Based on Cluster Type

DeltaTrain — Wed, 09 Aug 2023 22:05:39 GMT

Hello Databricks Community, I asked the same question on the Get Started Discussion page but feels like here is the right place for this question.

I'm reaching out with a query regarding access control in the hive_metastore. I've encountered behavior that I'd like to understand better and potentially address.

To illustrate the situation:

I've set up three users for testing purposes: admin, dataengineer1, and dataanalyst1.
The admin user granted permissions to dataengineer1 for three specific tables: circuits, country_regions, and results.

Case 1: When using SQL Warehouse (as seen in the screenshot, labeled as serverless-sql-wh) or a Cluster with shared Access mode, dataengineer1 can only view the tables they have permissions for. This is the expected behavior.

Case 2: However, when a Single User Access mode cluster is activated (in the screenshot, labeled as dataengineer1@d...), dataengineer1 can view all schemas and tables. This is not the desired behavior.

I'm hoping to find a solution that ensures even in Single User Access Mode, users can only access Schemas and Tables for which they have permission.

Any insights or suggestions would be greatly appreciated. I value the expertise of this community and look forward to your responses.

Thank you,

DeltaTrain

Re: Access Control in hive_metastore Based on Cluster Type

User16752239289 — Tue, 12 Sep 2023 18:44:43 GMT

That is expected. The single user mode is the legacy standard + UC ACL enabled. https://docs.databricks.com/en/archive/compute/cluster-ui-preview.html#how-does-backward-compatibility-work-with-these-changes

For your case, you need the hive table acl enabled to restrict the list schemas and list table actions.

You can add below two spark conf to enabled the hive metastore ACL:

spark.databricks.acl.dfAclsEnabled true spark.databricks.repl.allowedLanguages python,sql

topic Access Control in hive_metastore Based on Cluster Type in Data Engineering

Access Control in hive_metastore Based on Cluster Type

Re: Access Control in hive_metastore Based on Cluster Type