You’ve already identified data duplication as a potential con of landing the data first, but there are several benefits to this approach that might not be immediately obvious:

1. Schema Inference and Evolution: AutoLoader can automatically infer the schema of your data and adapt to changes over time (e.g., new fields in the API response). This reduces manual effort and makes it easier to handle evolving data structures.
2. Incremental Loading: AutoLoader processes only new files, improving performance and reducing compute costs compared to reprocessing all data. This is especially useful if your API extractions happen frequently or in batches.
3. Reprocessing Capability: With raw files in the landing zone, you can reprocess historical data for backfills, error corrections, or new transformations without relying on the API again. If the data is updated in the source system, you'll lose a history (should you need it again), this wouldn't happen if you had a file present in storage. You can argue delta lake has time travel available and solves some of this though.
   
   I love the question, @ChristianRRL . Looking forward to seeing other responses.
   
   All the best,
   BS