Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-23-2024 10:50 PM
Auto loader will process files incrementally. Let's say you have a files in existing directory called /input_files
First time you run autoloader, it will read all files in that directory (unless you set an option includeExsistingFiles to false, like you did) and save information about what files has been read to checkpoint location.
Next run will only load new files, because auto loader knows what has been loaded previously thanks to checkpoint