Suggested Design

1. SFTP uploads from Airship go to a Landing Zone folder in ADLS Gen2.
2. Storage event triggers an orchestration pipeline (ADF/Databricks) to read and process raw files.
3. Processed data lands in the Bronze Layer Delta tables with schema enforcement.
4. Silver and Gold layers follow based on business requirements.