Databricks not able to preserve special character while writing data
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-04-2024 09:03 AM
I am trying to write data in csv format
(df. coalesce(1) write
- format("csv")
- option ("header","true")
- Option (" sep","|시")
- option("encoding", "UTF-8”)
- mode ("overwrite")
- save("/mnt/Filestore/data/job1))Now here the problem is - there is one column in this data frame df with some value starting with a ? Enclosed in square box.we can read this data from source query in this df but after writing using above code , that special character is missing in csv file. Even if I read data in databricks, this special character is missing.could you please suggest what can be the solution here to preserve this character