Hi Brahma,

Thanks for the suggestion. I tried setting spark.conf.set("spark.databricks.delta.schema.autoMerge.enabled", "true") at the top of my DLT notebook as you recommended, but unfortunately, I'm still encountering the same error.

Just to clarify, my original table had a column named reference_score as INT, and it was later changed to FLOAT. Based on that, I assumed I might need to cast it back to INT, but that approach doesn't seem to work either.

I agree that dropping the target table would definitely resolve the issue, but before I take that step, I'm really curious why a schema change like this can't be handled more seamlessly.