Merge into and data loss

lizou
Contributor III

I have a delta table with 20 M rows, Ther table is being updated dozens of times per day. The merge into is used, and the merge works fine for 1 year. But recently I begin notice some of data is deleted from merge into without delete specified. Merge into onmly do updates.

I have published a test notebook, but am unable to reproduce the issue.

I can reproduce the issue in my production database.

Have anyone encountered a similar issue before?

https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/6324...