topic Updates are going as insert in a databricks job in Data Engineering

Updates are going as insert in a databricks job

sashikanth — Tue, 03 Dec 2024 13:20:02 GMT

There is no code change in a databricks job or notebook but we have observed the malfunction in cdc. The records which are meant for updates are going for inserts and causing duplicacy. At the sametime we had make sure the PKs based on which merge runs are same. Note this is a streaming job from a delta table.

This does not happen very frequently but the malfunction is occuring.

Please let us know incase if anyone has come across this situation and how did we overcome.

Re: Updates are going as insert in a databricks job

Alberto_Umana — Tue, 03 Dec 2024 13:35:05 GMT

Hi @sashikanth,

I would recommend opening a case with us further investigate this behavior since this requires a code logic review and additional checks.

Please refer to:

https://docs.databricks.com/en/resources/support.html

Re: Updates are going as insert in a databricks job

sashikanth — Wed, 04 Dec 2024 13:00:08 GMT

Case has been submitted - #00576955