Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-02-2023 06:52 AM
Hi All,
I have a scenario where my Exisiting Delta Table looks like below:
Now I have an incremental data with an additional column i.e. owner:
Dataframe Name --> scdDFBelow is the code snippet to merge Incremental Dataframe to targetTable, but the new column is not getting added:
spark.conf.set("spark.databricks.delta.schema.autoMerge.enabled",True) --> this is also enabled.
But still no luck. Below is the final result which i'm currently getting:
Data looks correct, but the only issue is New Column i.e. Owner is still not merged in targetTable.
Someone please help.
Thanks!
Labels: