artsheiko
Databricks Employee
Databricks Employee

Hi Sai,

I assume that the problem comes not from the PySpark, but from Excel.

I tried to reproduce the error and didn't find the way - that a good thing, right ? Please try the following : 

 

df.write.format("csv").save("/Volumes/<my_catalog_name>/<my_schema_name>/<my_volume_name>/<file_name>")

 

(how to create a Volume ; Volumes have a lot of value, but if you prefer to use a path in your object storage it's also ok)

Once the file is downloaded locally, open it in Excel. It will not recognize the columns. Click on the Data tab in the ribbon and find the "Text to Columns" button. In the opened dialog : Delimited -> Comma -> Finish.
 
Hope it helps,
 
Best,
Artem