Thank you, Anudeep.

I plan to tune Auto Loader by increasing the maxFilesPerTrigger parameter to optimize performance. My decision to use Auto Loader is primarily driven by its built-in backup functionality via cloudFiles.cleanSource.moveDestination, which eliminates the need to maintain custom code for file cleanup.

If there is a better option to back up files after ingestion, please feel free to suggest it.

Thank you,