Roll back to previous version of an AutoLoader checkpoint file

[DomDuf]

I know to "reset" AutoLoader, you can delete the checkpoint file entirely. I was wondering if it's possible to and how would someone :

• Get the checkpoint file to a previous version so I can reload certain files that were already processed
• Delete certain rows in the checkpoint file (by creation date for example)

Thanks for your help.

[Murthy1 · Accepted solution]

Hello @Dominic Dufour​ ,

I had the same question and Unfortunately it is not possible yet.

However you can delete the checkpoint file and use the Autoloader option - "modifiedAfter" to pick up the files after a specific time. This is a one time activity as the checkpoint will be created again to continue for the future loads. You can remove the "modifiedAfter" once your data loads are back on track.

The other work around I can think of is a bit intensive one, You can have some scheduled jobs to copy the checkpoint files to a different location frequently (with date time markers- probably in the folder name). You can restore the checkpoint files from that location when you want to reset Autoloader.

[DomDuf]

Thanks for the quick answer, this was really helpful !

[MRTN]

This would for sure be a useful feature.