532664
New Contributor III

@jcozar 

I'm glad I could help, but unfortunately, the issue wasn't resolved with that method.

As I mentioned, I encountered an error after making the change:

org.apache.spark.sql.AnalysisException: Failed to merge fields 'col10' and 'col10'. Failed to merge incompatible data types BooleanType and IntegerType.

It looks like I'll need to start from scratch and re-import all the data from the database using Kafka and a full refresh in DLT (which creates new metadata initially).

Thank you once again for all your help and insights. Have a great day!