Insufficient Permissions Issue on Databricks

[MOUNIKASIMHADRI]

I have encountered a technical issue on Databricks.

While executing commands both in Spark and SQL within the Databricks environment, 

I’ve run into permission-related errors from selecting files from DBFS. 

"org.apache.spark.SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges" 

and

"SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SELECT on any file."

Tried many times and I have all the privileges owner has but still unable to resolve the issue.

[Walter_C]

Hello Mounika, many thanks for your question, are you using a shared access cluster? If yes, shared clusters requires you to grant Select permission on Any file to be able to access DBFS as mentioned on this doc https://docs.databricks.com/en/data-governance/table-acls/any-file.html#how-does-any-file-interact-w...
You can grant this permission by running https://kb.databricks.com/en_US/data/user-does-not-have-permission-select-on-any-file 

Another solution will be to use single user cluster which does not requires it.

[NandiniN]

Hi @MOUNIKASIMHADRI ,

Workspace admins get ANY FILE granted by default. They can explicitly grant it to non-admin users.

Hence as suggested in the kb, 

GRANT SELECT ON ANY FILE TO `<user@domain-name>`

[mpalacio]

Hi, I am having the same issue. The Databricks extension is well installed and configured, and my user has enough permissions as I have been working without issues the whole time, but now when I run my notebooks to read tables in the same databricks catalog this error comes up the whole time, along with 'no module named dbruntime'.

I have tried reinstalling the extension and the environment, and changing the profile and some settings but still doesn't work.

[NandiniN]

Hi @mpalacio ,

For no module named 'dbruntime'

Are you using the dbutils in any code that runs on a Spark worker node. 

If yes then it will throw the above error because this is an expected behaviour that we cannot use dbutils in any code that runs on a Spark worker node since there are no user credentials on it.
All the UDF and Spark transforms (filter, map, etc) runs on the worker node.

[NandiniN]

Please refer to some of the other community articles with the no module error https://community.databricks.com/t5/data-engineering/udf-importing-from-other-modules/td-p/58988