Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-03-2024 05:59 PM
I'm glad I could help, but unfortunately, the issue wasn't resolved with that method.
As I mentioned, I encountered an error after making the change:
org.apache.spark.sql.AnalysisException: Failed to merge fields 'col10' and 'col10'. Failed to merge incompatible data types BooleanType and IntegerType.
It looks like I'll need to start from scratch and re-import all the data from the database using Kafka and a full refresh in DLT (which creates new metadata initially).
Thank you once again for all your help and insights. Have a great day!