Special character data preservation

[Priya_Data_Eng]

1. This data frame has two columns name and info. Name has value as John and info has vale as 1® VOC.

After writing this data, I can read correct values in databricks but when I download the csv file and load it in notepad ( utf-8 ) , it is showing no values.

even if I click on edit in container, it is showing empty values.

What can be the reason? And what is the solution ?

df. write. format (‘csv').option ('header', True).option( 'encoding', 'UTF-8').option(' sep', '|^|’) .mode ('overwrite'). save('/mnt/Filestore/data/job1/test.dta')

[amr]

Try to read the file back into databricks using spark.read, do you the see the charchaters showing? if yes, then it is an editor problem, use another editor such as Notepad++, if not, then the data is not on the write encoder, try different encoder other than UTF-8, maybe UTF-16