Hi @ChristianRRL ,

One reason to use landing path is that you will have an easier job when you need to reload table. In such case you don't have to re-pull all the data from API's once again.

Another thing, you have separation of concerns. Extraction part is done by one part of the pipeline, loading is done by another. So you have clearly defined, logical steps.

But if you feel that in your case it's much simpler to write directly to Delta Table then go for it. 

AutoLoader is great for processing large batches of files. It allows you to land raw API responses as files (e.g., JSON), then automatically and incrementally ingest them into Delta tables. It supports scalable file discovery, handles schema drift, and enables efficient deduplication and replay. It's based on spark structured streaming so in case of failure you can gracefully recover.