If the data isn't more than a few GB then you can coalesce your dataset prior to writing it out.

Something like:

df.coalesce(1).write.format("com.databricks.spark.cvs").save("...path...")

then copy to a single file using a dbutils.fs command:

dbutils.fs.cp("...path...", "..path.. ..csv")