cancel
Showing results for 
Search instead for 
Did you mean: 
Community Discussions
cancel
Showing results for 
Search instead for 
Did you mean: 

SQL Merge not adding new columns to delta table

Benedetta
New Contributor III
SQL Merge not adding new columns to delta table error is: cannot resolve Target AuditUpdateDate in UPDATE clause given columns in Source
this config is set: spark.conf.set("spark.databricks.delta.schema.autoMerge.enabled",True)
We're using DBR 12.2 LTS (includes Apache Spark 3.3.2, Scala 2.12)
Per the documentation below seems like new columns should be added.
What am I doing wrong? Thanks!
Benedetta
Benedetta_0-1689344912359.png
SQL query below trying to add new column: AuditUpdateDate
MERGE INTO XYZ.table1 AS Target USING cleansedTV AS Source
ON Target.col1 = Source.col1 AND Target.col2 = Source.col2
WHEN MATCHED AND( Target.col3 <> Source.col3
AND Target.col4 <> Source.col4
AND Target.col5 <> Source.col5
)
THEN UPDATE SET
Target.col3 = Source.col3
, Target.col4 = Source.col4
, Target.col5 = Source.col5
,Target.AuditUpdateDate = Source.AuditUpdateDate ----> Trying to add new column here
WHEN NOT MATCHED THEN INSERT *
0 REPLIES 0
Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.