532664
New Contributor III

Hi @jcozar Thanks for your opinion!


I agree with your suggestion and tried applying it. By switching to triggered mode, I was able to perform a selected refresh as you told.

After changing the struct schema value, I refreshed the silver and gold tables. Unfortunately, it didn't work as expected and resulted in an error.

org.apache.spark.sql.AnalysisException: Failed to merge fields 'col' and 'col'. Failed to merge incompatible data types BooleanType and IntegerType

Seems like I need to ponder over this more.

Thanks so much for your help.