MergeSchema Not Working

DJey
New Contributor III

Hi All,

I have a scenario where my Exisiting Delta Table looks like below:

imageNow I have an incremental data with an additional column i.e. owner:

Dataframe Name --> scdDFimageBelow is the code snippet to merge Incremental Dataframe to targetTable, but the new column is not getting added:

imagespark.conf.set("spark.databricks.delta.schema.autoMerge.enabled",True) --> this is also enabled.

But still no luck. Below is the final result which i'm currently getting:

Data looks correct, but the only issue is New Column i.e. Owner is still not merged in targetTable.image 

Someone please help.

Thanks!