szymon_dybczak
Esteemed Contributor III

Hi @maikel ,

It's just a trigger mechanism, so it totally depends on how you implement your pipeline that will be responsible for consuming the data.
For instance if you use structure streaming based approach (i.e autoloader) then even if your pipeline fails then you can be sure that when you re-run you won't miss any data.
If you want to implement incremental approach yourself then you need to find a proper key for a given a table and find an attribute that will help you discover only new records that appeared/changed since last loading.