vida
Databricks Employee
Databricks Employee

I was able to turn that off by setting the quote option to be a single white space. The problem with this is I am not sure how you can espace strings can contain your delimiter - "," - or whatever you set that too. If you are sure none of your strings have the delimiting character, you should be fine.

(df
  .repartition(1)
  .write
  .format("com.databricks.spark.csv")
  .option("header", "true")
  .option("quote", " ")
  .save("/FileStore/test"))