pd.read_csv failed
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-22-2022 12:52 AM
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-23-2022 01:12 PM
Pandas will not see dbfs storage this way. You need to change to pandas on spark or use the classic path.
import pyspark.pandas as pd
pd.read_csv("dbfs:/your_path/your_file.csv")
import pandas as pd
pd.read_csv("/dbfs/your_path/your_file.csv")My blog: https://databrickster.medium.com/
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-24-2022 04:43 AM
Thanks for your kind reply:
Below works for me:
But why, as you mentioned, using the classic path, below does not work?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-07-2022 05:05 PM
Hi @THIAM HUAT TAN ,
Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.
