Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-01-2022 05:58 AM
Struggling with how to export a Spark dataframe as a *.csv file to a local computer. I'm successfully using the spark_write_csv funciton (sparklyr R library R) to write the csv file out to my databricks dbfs:FileStore location. Becase (I'm assuming) databricks is creating 4 *.csv partitions. So I gather from the databricks documentation I need to coalesce the partitions files . So I'm am using the following command....
df.coalesce(1).write.option("header","true").csv("dbfs:FileStore/temp/df.csv")
And then I receive a NameError: name 'df' is not defined.
So am I missing a step, syntax wrong? I'm working in a R Notebook
Jeff
Labels:
- Labels:
-
Local computer