Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-14-2023 04:02 AM
- You can read the input file using Pandas or Koalas (https://koalas.readthedocs.io/en/latest/index.html)
- then rename the columns
- then convert the Pandas/Koalas dataframe to Spark dataframe. You can write it back with the correct column name, so the next time you use it, the error will not happen.