Databricks Row filter causing duplicates in Delta Table while writing

Abhijeet2492
New Contributor II

‎09-11-2025 06:16 AM

Hello,

I have a databricks external table to which I have applied a row filter, After applying the row filter I see that the Delta merge functionality is not working as expected even with simple records ( not nested) and I have already tried execution with 16.4 and serverless clusters.

Is there any suggestion or recommendation for this?

0 Kudos

szymon_dybczak
Esteemed Contributor III

‎09-11-2025 06:45 AM

Hi @Abhijeet2492 ,

For now merge support for row filters is pretty limited. Maybe you're hitting one of following limitations:

szymon_dybczak_0-1757598309061.png

 

0 Kudos

Abhijeet2492
New Contributor II

‎09-11-2025 06:51 AM

What is the workaround to achieve the same functionality of upsert that we get with Merge?

0 Kudos

szymon_dybczak
Esteemed Contributor III
In response to Abhijeet2492

‎09-11-2025 07:02 AM

Every merge statement can be rewritten as series of INSERT, UPDATE and DELETES. So, as of now you can try to handle it that way. Or just way until this feature mature enough (I guess soon some of the limitations will be removed)

0 Kudos