Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-11-2022 04:15 AM
- at least one of column names can have some strange char, whitespace or something,
- or at least one of column type is not compatible (for example StructType)
- you can separate your code to two or more steps. First generate list of columns as some variable, than create column to concatenate than new column with sha that column. It is easier to debug and also more efficient for spark as it use lazy evolution and logical/physical plans and adaptive query execution.
My blog: https://databrickster.medium.com/
