@Philospher1425,

The problem is, in order to generate a single .csv file you have to coalesce your dataset to one partition and lose all parallelism that spark provides. While this might work for small datasets, such pattern will certainly lead to memory issues on larger datasets. 

If you think that the pattern you described is a good and valid idea, please submit your idea to https://github.com/apache/spark or Databricks Ideas Portal.