Thank you for the input. Is there any particular reason why deduplication watermark makes it store everything and not just the key needed for deduplication? The 1st record has to be written to the table anyway, and its content is irrelevant as it just drops later records that get a hit.

Is there any way to control this behavior?  I know I could do a constraint on write, but that seems really excessive as the table has millions of rows  and I really need to look just a few minutes past.