I haven't tried but I suspect it will fail with the same message on INSERT because uc.dl_id is NULL for some rows and `whenNotMatchedInsertAll` will attempt to insert a value for dl_id field instead of generating one (as if it has been user provided...
I use 11.2 runtime. identity column created as:dl_id BIGINT NOT NULL GENERATED ALWAYS AS IDENTITYand the pyspark code is an upsert:t.alias("ec").merge(df.alias("uc"), "ec.dl_id = uc.dl_id") \
.whenMatchedUpdateAll() \
.whenN...
