writing spark dataframe as CSV to a repo
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-12-2024 11:25 PM
Hi,
I wrote a spark dataframe as csv to a repo (synced with github). But when I checked the folder, the file wasn't there.
Here is my code:
spark_df.write.format('csv').option('header','true').mode('overwrite').save('/Repos/abcd/mno/data')
No error message ! Nevertheless no file inside the folder 'data'. In addition 'abcd' is my repo name, inside it is a folder 'mno' which contains 'data'.
-Thx
