Error as no such file when reading CSV file using pandas
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-24-2022 09:22 AM
I'm trying to read a CSV file saved in data using pandas read_csv function. But it gives No such file error.
%fs
ls /FileStore/tables/
df= pd.read_csv('/dbfs/FileStore/tables/CREDIT_1.CSV')
df= pd.read_csv('/dbfs:/FileStore/tables/CREDIT_1.CSV')
df= pd.read_csv('/FileStore/tables/CREDIT_1.CSV')
Can someone pls help with this?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-25-2022 12:05 AM
I see you use pandas to read from dbfs.
But pandas will only read from local files,
see this topic also. It is about databricks-connect but the same principles apply.
So what you should do is first read the file using spark.read.csv and then converting the spark df to a pandas df.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-01-2022 12:01 AM
Hi @Kelum Perera following - up did you get a chance to check @Werner Stinckens previous comments or do you need any further help on this?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-03-2022 09:33 AM
Thanks to @Werner Stinckens for the answer.
I understood that I have to use spark to read data from clusters.

