SQLSTATE: 42501 - Missing Privileges for User Groups

[ez]

Dear All

I'm investigating missing privileges for some of our users.

When connecting to an Oracle database via JDBC and attempting to display a DataFrame, we encounter the following error:

User does not have permission SELECT on any file. SQLSTATE: 42501

This issue only occurs for regular users running on Shared Cluster Compute. However:

• It works fine when the same users switch to Personal Compute.

• Additionally, when an admin runs the same code on Shared Cluster Compute, there is no issue

• It also works when calling some different API and displaying data using Shared Cluster Compute.

As a workaround, I executed the following command based on the guidance from the Databricks documentation: User does not have permission SELECT on ANY File - Databricks:

GRANT SELECT ON ANY FILE TO `<user@domain-name>`

This resolved the issue and the user can now view the DataFrame. However, I'm unsure if granting access to ANY FILE is the appropriate or most secure solution. Ideally, I'd like to identify and assign the specific, minimal permissions required instead of using such a broad grant.

Could you please advise on the correct or recommended privilege to use in this case?

Code:

df = spark.read.format("jdbc") \
                .option("url", jdbc_url) \
                .option("user", user) \
                .option("password", password) \
                .option("dbtable", table_name) \
                .option("driver", "oracle.jdbc.driver.OracleDriver") \
                .load()

display(df)

Thanks

[nayan_wylde]

@ez Avoid granting ANY FILE unless absolutely necessary because it gives broad access to all files, bypassing Unity Catalog governance.Prefer to use governed access use like Lakehouse Federation for Oracle or other external sources. This integrates with Unity Catalog, supports fine-grained privileges, and removes the need for ANY FILE.

If Federation isn’t possible short-term. Grant SELECT ON ANY FILE (read-only) to a restricted group, not individuals.

Run JDBC workloads on Personal Compute or Dedicated Job clusters, where ANY FILE isn’t enforced.

[ez]

@NayanThank you, your answer is clear, and I agree with everything.

However, I would also like to understand why this is happening and what the Shared Cluster is trying to access in this example that it doesn’t have privileges for.

[nayan_wylde]

Shared clusters run in Standard access mode, which enforces Unity Catalog’s secure access model.
When your code uses a custom JDBC driver and tries to read data, Databricks treats this as direct file access outside Unity Catalog governance. It may also access storage paths (like /tmp or DBFS) that aren’t tied to a UC table or volume.
In Standard mode, these operations require the ANY FILE privilege, because UC cannot guarantee governance over arbitrary file paths.

Personal Compute uses Single User (Dedicated) access mode, which does not enforce ANY FILE restrictions. The user is trusted to access their own files.

https://docs.databricks.com/aws/en/data-governance/table-acls/any-file