topic how to implement delta load when table only has primary columns in Get Started Discussions

how to implement delta load when table only has primary columns

Puneet096 — Thu, 18 Jul 2024 06:32:51 GMT

I have a table where there are two columns and both are primary key, I want to do delta load when taking data from source to target. Any idea how to implement this?

Re: how to implement delta load when table only has primary columns

szymon_dybczak — Thu, 18 Jul 2024 06:42:35 GMT

Hi @Puneet096 ,

One approach you can use is merge into statement. Probably you can delete update part from below merge template, because it's unlikely that there will be update of values in composite primary key

MERGE INTO target_table AS target USING source_table AS source ON target.pk1 = source.pk1 AND target.pk2 = source.pk2 WHEN MATCHED AND source.last_updated > target.last_updated THEN UPDATE SET target.data = source.data, target.last_updated = source.last_updated WHEN NOT MATCHED THEN INSERT (pk1, pk2, data, last_updated) VALUES (source.pk1, source.pk2, source.data, source.last_updated);

Re: how to implement delta load when table only has primary columns

Puneet096 — Thu, 18 Jul 2024 07:16:06 GMT

Thanks for the response @szymon_dybczak, but the table has only two columns and both are primary key

Re: how to implement delta load when table only has primary columns

szymon_dybczak — Thu, 18 Jul 2024 07:39:16 GMT

But that shouldn't be a problem. In merge condition you check both keys as in example above. If combination of two keysb already exists in the table then do nothing. If there is new combination of key1 and key2 just insert it into target table.

It's that simple, or maybe I don't fully understand your problem.