How can I write a single file to a blob storage using a Python notebook, to a folder with other data?

Danielsg94
Databricks Partner

‎08-24-2022 12:59 AM

When I use the following code: 

df
   .coalesce(1)
   .write.format("com.databricks.spark.csv")
   .option("header", "true")
   .save("/path/mydata.csv")

it writes several files, and when used with .mode("overwrite"), it will overwrite everything in the folder.

I want to add a single .csv file to the folder in the blob, without overwriting the content of the path.

Also I am not sure how to name a file. When you specify mydata.csv it creates a folder with that name, and several files inside it.

0 Kudos