Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-12-2024 12:22 AM
I checked the library you are using to write to Excel and it seems there is a new version available that has improved data type handling.
https://github.com/crealytics/spark-excel
- Spark-Excel V2 with data source API V2.0+, which supports loading from multiple files, corrupted record handling and some improvement on handling data types. See below for further details
To use V2 implementation, just change your .format from .format("com.crealytics.spark.excel") to .format("excel").
Check the github readme for details. If your dataframe has the same datatypes as the Excel table, I'm hoping this gives you some more luck.
A Spark plugin for reading and writing Excel files - nightscape/spark-excel