How to read a file in pandas in a databricks environment?

[tinendra]

Hi,

When I was trying to read the CSV files using pandas I am getting an error which I have mentioned below.

df=pd.read_csv("/dbfs/FileStore/tables/badrecord-1.csv")

Error: FileNotFoundError: [Errno 2] No such file or directory: '/dbfs/FileStore/tables/badrecord-1.csv'

df=pd.read_csv("/FileStore/tables/badrecord-1.csv")

Error: FileNotFoundError: [Errno 2] No such file or directory: '/FileStore/tables/badrecord-1.csv'

Anyone, please tell me how to resolve the above error. thanks

[Anonymous]

So pandas runs on the driver and will read from the driver's filesystem. DBFS is the distributed file system. You can use %sh fs ls to explore the files on the driver.

[tinendra]

dbutils.fs.ls("/FileStore/tables/badrecord-1.csv")

the above file is there in that particular location but still getting the same error