- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-12-2022 05:52 AM
Hi Can you help me why Pandas code not working..but Pyspark is working..
import pandas as pd
pdf = pd.read_csv('/FileStore/tables/new.csv',sep=',')
Error : No such file exists...
below is worked..
df = spark.read.csv("/FileStore/tables/new.csv", sep=",", header='True')
- Labels:
-
Pandas Code
-
Pyspark
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-12-2022 05:55 AM
Try to add /dbfs/ or dbfs: prefix
My blog: https://databrickster.medium.com/
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-12-2022 06:14 AM
Yeah..I tried all options...still no file exists..
So I am converting Py spark DF to Pandas DF...
I am interested to know why below is not working..
pdf = pd.read_csv('/FileStore/tables/new.csv',sep=',')
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-12-2022 08:41 AM
It might has to do with the path as @Hubert Dudek already mentioned:
df = spark.read.csv("dbfs:/FileStore/tables/new.csv", sep=",", header='True')
