https://community.databricks.com/t5/data-engineering/considering-autoloader-for-bronze-to-silver-transformations/m-p/127378#M47939 <P>I’m currently implementing Auto Loader to ingest data from the source into the Bronze layer—essentially mapping the raw data into Delta tables. Now, I’ve also been considering using Auto Loader for Bronze-to-Silver transformations.</P><P>Are there any pros and cons I should consider before moving forward with this approach?</P> Mon, 04 Aug 2025 20:26:04 GMT yit 2025-08-04T20:26:04Z https://community.databricks.com/t5/data-engineering/considering-autoloader-for-bronze-to-silver-transformations/m-p/127378#M47939 <P>I’m currently implementing Auto Loader to ingest data from the source into the Bronze layer—essentially mapping the raw data into Delta tables. Now, I’ve also been considering using Auto Loader for Bronze-to-Silver transformations.</P><P>Are there any pros and cons I should consider before moving forward with this approach?</P> Mon, 04 Aug 2025 20:26:04 GMT https://community.databricks.com/t5/data-engineering/considering-autoloader-for-bronze-to-silver-transformations/m-p/127378#M47939 yit 2025-08-04T20:26:04Z https://community.databricks.com/t5/data-engineering/considering-autoloader-for-bronze-to-silver-transformations/m-p/127834#M48099 <P class="whitespace-normal break-words">Auto Loader is for loading raw files, not loading Delta Lake or Apache Iceberg tables, <A href="https://docs.databricks.com/aws/en/ingestion/cloud-object-storage/auto-loader/#supported-auto-loader-sources" target="_self">see more here</A>.</P> <P class="whitespace-normal break-words">Instead,&nbsp;<A href="https://docs.databricks.com/aws/en/structured-streaming/delta-lake#delta-table-as-a-source" target="_self">stream from a Delta Lake table</A>.</P> Fri, 08 Aug 2025 16:26:43 GMT https://community.databricks.com/t5/data-engineering/considering-autoloader-for-bronze-to-silver-transformations/m-p/127834#M48099 cgrant 2025-08-08T16:26:43Z