Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-12-2018 02:04 PM
hi @Dominic Robinson I'm unable to create a simple reproduction of this issue. I was able to write out a file with the Euro symbol as the column using dataframe.write.csv(path), and the symbol was fine when I read the file back in using spark.read.csv(path). I think you are correct that the problem is the interaction between the csv source and whatever is producing your files.
Did you try this out with the built-in csv source yet?
If you are continuing to have problems, please raise a support ticket with Databricks. It could be a bug, or it could be your particular use case is unsupported and could be added to the csv source by Databricks.