how to implement delta load when table only has primary columns
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-17-2024 11:32 PM
I have a table where there are two columns and both are primary key, I want to do delta load when taking data from source to target. Any idea how to implement this?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-17-2024 11:39 PM - edited 07-17-2024 11:42 PM
Hi @Puneet096 ,
One approach you can use is merge into statement. Probably you can delete update part from below merge template, because it's unlikely that there will be update of values in composite primary key
MERGE INTO target_table AS target
USING source_table AS source
ON target.pk1 = source.pk1 AND target.pk2 = source.pk2
WHEN MATCHED AND source.last_updated > target.last_updated THEN
UPDATE SET
target.data = source.data,
target.last_updated = source.last_updated
WHEN NOT MATCHED THEN
INSERT (pk1, pk2, data, last_updated)
VALUES (source.pk1, source.pk2, source.data, source.last_updated);
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-18-2024 12:16 AM
Thanks for the response @szymon_dybczak, but the table has only two columns and both are primary key
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-18-2024 12:39 AM
But that shouldn't be a problem. In merge condition you check both keys as in example above. If combination of two keysb already exists in the table then do nothing. If there is new combination of key1 and key2 just insert it into target table.
It's that simple, or maybe I don't fully understand your problem.