[INSUFFICIENT_PERMISSIONS] Insufficient privileges:

Tamizh035
Databricks Partner

‎10-23-2024 04:58 AM

While reading csv file using spark and listing the files under a folder using data bricks utils, I am getting below error:

[INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SELECT on any file. SQLSTATE: 42501
File <command-1200317151894398>, line 2 1 spark_df = spark.read.csv("sample_csv.csv") ----> 2 display(spark_df)
File /databricks/python_shell/dbruntime/display.py:131, in Display.display(self, input, *args, **kwargs) 129 # This version is for Serverless + Spark Connect dogfooding. 130 elif self.spark_connect_enabled and isinstance(input, ConnectDataFrame): --> 131 self.display_connect_table(input, **kwargs) 132 elif isinstance(input, ConnectDataFrame):
------
But able to read and write csv files using pandas in the same environment. 
Below is the code which I used:

import pandas as pd
df = pd.read_csv("sample_csv.csv")
 
sample_csv.csv is the file I created and I have "can manage" privilege on it.
 
Kindly help me with resolving the issue.
display(df)
 
spark_df = spark.read.csv("sample_csv.csv")
display(spark_df)
 
files = dbutils.fs.ls("path")
 





 
0 Kudos

Panda
Valued Contributor

‎10-23-2024 05:07 AM

@Tamizh035 , If your file in dbfs or external location or local folder ?
Use dbutils.fs.ls to verify that the path exists and you have access:

files = dbutils.fs.ls("dbfs:/path_to_your_file/")
display(files)

 

Tamizh035
Databricks Partner
In response to Panda

‎10-23-2024 10:00 PM

Hi @Panda 

Listing the files using dbutils too is giving same error.  I have created the folder and I have "can manager" privilege on it.

ExecutionError: An error occurred while calling o415.ls.
: org.apache.spark.SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges:
User does not have permission SELECT on any file. SQLSTATE: 42501

0 Kudos

mpalacio
New Contributor II

‎12-03-2024 06:48 AM

I have the same issue, did you manage to solve it?

I have the Databricks extension well configured and my role has enough permissions. Everything used to work propertly but now when I run my notebooks is giving me this issue and 'no module named dbruntime' all the time.

0 Kudos