When would you not want to use autoloader?

[StephanieRivera]

I am genuinely curious why would you ever not use Autoloader? I see it in one-off downloads of course. When you pull data from another platform, say Salesforce, is it better to append to a table without Autoloader?

There must be cases I am missing. Thoughts??

[Tharun-Kumar]

@StephanieRivera 

Auto Loader is primarily used for processing files automatically as they arrive in cloud storage.

You can refer to this document on AutoLoader - https://docs.databricks.com/en/ingestion/auto-loader/index.html

[-werners-]

Autoloader is pretty handy, but not open source.  That is one reason f.e.
Another reason is f.e. if you cannot guarantee lexicographically generated files, or you do not want to use streaming, or you do not land your raw data into a data lake (read from db directly f.e.).
I still use a lot of batch processing using /.../YYYY/MM/DD directory structure without autoloader.
Not because I don´t like it (I do use it for new setups), but because at the time it just did not exist and for a 1x/day batch job autoloader is not necessary.