mattiazeni
Databricks Employee
Databricks Employee

What do you need to achieve?

Autoloader is much more efficient since it can handle a bunch of files (only new ones) in a single operation. Handling file by file, especially with a lot of files, will increase latency and increase costs.