How do you read files from the DBFS with OS and Pandas Python libraries?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2023 05:20 AM
I created translations for decoded values and want to save the dictionary object the DBFS for mapping. However, I am unable to access the DBFS without using dbutils or PySpark library.
Is there a way to access the DBFS with OS and Pandas Python libraries? At work, we can directly use the same path as the PySpark functions to write/ read from the DBFS without issue.
- Confirm files exist within DBFS
- Confirm ability to read file with PySpark
- Error 1 - recommends using "/dbfs" instead of "dbfs:"
- Updated "/dbfs" - error persists
- Removed DBFS entirely:
- and one last shot....
Thank you!
Labels:
- Labels:
-
DBFS
-
Pandas Python
-
PySpark Error