topic Insufficient Permissions Issue on Databricks in Get Started Discussions

Insufficient Permissions Issue on Databricks

MOUNIKASIMHADRI — Fri, 26 Apr 2024 15:51:39 GMT

I have encountered a technical issue on Databricks.

While executing commands both in Spark and SQL within the Databricks environment,

I’ve run into permission-related errors from selecting files from DBFS.

"org.apache.spark.SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges"

and

"SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SELECT on any file."

Tried many times and I have all the privileges owner has but still unable to resolve the issue.

Re: Insufficient Permissions Issue on Databricks

Walter_C — Sat, 27 Apr 2024 16:00:11 GMT

Hello Mounika, many thanks for your question, are you using a shared access cluster? If yes, shared clusters requires you to grant Select permission on Any file to be able to access DBFS as mentioned on this doc https://docs.databricks.com/en/data-governance/table-acls/any-file.html#how-does-any-file-interact-with-unity-catalog
You can grant this permission by running https://kb.databricks.com/en_US/data/user-does-not-have-permission-select-on-any-file

Another solution will be to use single user cluster which does not requires it.

Re: Insufficient Permissions Issue on Databricks

NandiniN — Wed, 01 May 2024 06:55:11 GMT

Hi @MOUNIKASIMHADRI ,

Workspace admins get ANY FILE granted by default. They can explicitly grant it to non-admin users.

Hence as suggested in the kb,

GRANT SELECT ON ANY FILE TO `<user@domain-name>`

Re: Insufficient Permissions Issue on Databricks

mpalacio — Tue, 03 Dec 2024 14:52:55 GMT

Hi, I am having the same issue. The Databricks extension is well installed and configured, and my user has enough permissions as I have been working without issues the whole time, but now when I run my notebooks to read tables in the same databricks catalog this error comes up the whole time, along with 'no module named dbruntime'.

I have tried reinstalling the extension and the environment, and changing the profile and some settings but still doesn't work.

Re: Insufficient Permissions Issue on Databricks

NandiniN — Wed, 04 Dec 2024 09:31:08 GMT

Hi @mpalacio ,

For no module named 'dbruntime'

Are you using the dbutils in any code that runs on a Spark worker node.

If yes then it will throw the above error because this is an expected behaviour that we cannot use dbutils in any code that runs on a Spark worker node since there are no user credentials on it.
All the UDF and Spark transforms (filter, map, etc) runs on the worker node.

Re: Insufficient Permissions Issue on Databricks

NandiniN — Wed, 04 Dec 2024 09:31:53 GMT

Please refer to some of the other community articles with the no module error https://community.databricks.com/t5/data-engineering/udf-importing-from-other-modules/td-p/58988

Re: Insufficient Permissions Issue on Databricks

MB09 — Wed, 28 May 2025 08:14:21 GMT

Hi Walter, I am facing a similar issue but while accessing an Azure Blob File from Databricks Shared cluster.
1. As an admin I tried and am able to access the file
2. While another user with just developer access is facing issues while accessing the Blob file - Insufficient Privileges

3. What I have made sure is that the user is granted with the read &write access on the storage container and file

Could you please help?