[INSUFFICIENT_PERMISSIONS] Insufficient privileges:

[Tamizh035]

While reading csv file using spark and listing the files under a folder using data bricks utils, I am getting below error:

[INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SELECT on any file. SQLSTATE: 42501

File <command-1200317151894398>, line 2 1 spark_df = spark.read.csv("sample_csv.csv") ----> 2 display(spark_df)

File /databricks/python_shell/dbruntime/display.py:131, in Display.display(self, input, *args, **kwargs) 129 # This version is for Serverless + Spark Connect dogfooding. 130 elif self.spark_connect_enabled and isinstance(input, ConnectDataFrame): --> 131 self.display_connect_table(input, **kwargs) 132 elif isinstance(input, ConnectDataFrame):
------
But able to read and write csv files using pandas in the same environment. 
Below is the code which I used:

import pandas as pd

df = pd.read_csv("sample_csv.csv")

sample_csv.csv is the file I created and I have "can manage" privilege on it.

Kindly help me with resolving the issue.

display(df)

spark_df = spark.read.csv("sample_csv.csv")

display(spark_df)

files = dbutils.fs.ls("path")

[Panda]

@Tamizh035 , If your file in dbfs or external location or local folder ?
Use dbutils.fs.ls to verify that the path exists and you have access:

files = dbutils.fs.ls("dbfs:/path_to_your_file/")
display(files)

[Tamizh035]

Hi @Panda 

Listing the files using dbutils too is giving same error.  I have created the folder and I have "can manager" privilege on it.

ExecutionError: An error occurred while calling o415.ls.
: org.apache.spark.SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges:
User does not have permission SELECT on any file. SQLSTATE: 42501