cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

[INSUFFICIENT_PERMISSIONS] Insufficient privileges:

Tamizh035
New Contributor II

While reading csv file using spark and listing the files under a folder using data bricks utils, I am getting below error:

[INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SELECT on any file. SQLSTATE: 42501
File <command-1200317151894398>, line 2 1 spark_df = spark.read.csv("sample_csv.csv") ----> 2 display(spark_df)

File /databricks/python_shell/dbruntime/display.py:131, in Display.display(self, input, *args, **kwargs) 129 # This version is for Serverless + Spark Connect dogfooding. 130 elif self.spark_connect_enabled and isinstance(input, ConnectDataFrame): --> 131 self.display_connect_table(input, **kwargs) 132 elif isinstance(input, ConnectDataFrame):
------
But able to read and write csv files using pandas in the same environment. 
Below is the code which I used:

import pandas as pd
df = pd.read_csv("sample_csv.csv")
 
sample_csv.csv is the file I created and I have "can manage" privilege on it.
 
Kindly help me with resolving the issue.
display(df)
 
spark_df = spark.read.csv("sample_csv.csv")
display(spark_df)
 
files = dbutils.fs.ls("path")
 





 
3 REPLIES 3

Panda
Valued Contributor

@Tamizh035 , If your file in dbfs or external location or local folder ?
Use dbutils.fs.ls to verify that the path exists and you have access:

files = dbutils.fs.ls("dbfs:/path_to_your_file/")
display(files)

 

Tamizh035
New Contributor II

Hi @Panda 

Listing the files using dbutils too is giving same error.  I have created the folder and I have "can manager" privilege on it.

ExecutionError: An error occurred while calling o415.ls.
: org.apache.spark.SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges:
User does not have permission SELECT on any file. SQLSTATE: 42501

mpalacio
New Contributor

I have the same issue, did you manage to solve it?

I have the Databricks extension well configured and my role has enough permissions. Everything used to work propertly but now when I run my notebooks is giving me this issue and 'no module named dbruntime' all the time.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group