kulkpd
Contributor

@esalohs @Retired_mod ,
I tried similar use-cases, my finding is even though includeExistingFiles=false and modifiedAfter configuration is included. At very first execution, autoloader will scan the entire s3 and sync up the rocks, all later executions will be fast. 
Let me know if your findings.

Try considering file-notification option instead of dir listing as you want to process only LATEST records