Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Wednesday
Hi experts, I read an article about auto loader duplicates handling that got me bit confused. It is checkpoint that tracks what is being processed , and upon confirmation it process only the new incoming records. But, let's say I am reloading whole bulk of records again that includes previously processed records, then would auto loader checks transaction file to ensure that those records that already been processed discarded? or it silently process already processed records. Thank you.
Labels:
- Labels:
-
Delta Lake