https://community.databricks.com/t5/data-engineering/does-autoloader-detect-new-records-in-a-snowflake-table-or-only/m-p/103359#M41417 <P>The only thing I can find with autoloader is picking up new files, nothing about new records in an existing snowflake table.</P> Fri, 27 Dec 2024 20:18:38 GMT mmceld1 2024-12-27T20:18:38Z https://community.databricks.com/t5/data-engineering/does-autoloader-detect-new-records-in-a-snowflake-table-or-only/m-p/103359#M41417 <P>The only thing I can find with autoloader is picking up new files, nothing about new records in an existing snowflake table.</P> Fri, 27 Dec 2024 20:18:38 GMT https://community.databricks.com/t5/data-engineering/does-autoloader-detect-new-records-in-a-snowflake-table-or-only/m-p/103359#M41417 mmceld1 2024-12-27T20:18:38Z https://community.databricks.com/t5/data-engineering/does-autoloader-detect-new-records-in-a-snowflake-table-or-only/m-p/103362#M41418 <P>Hi&nbsp;<a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/138073">@mmceld1</a>,&nbsp;</P> <P>Autoloader is for cloud storage files. You can achieve similar functionality by using Delta Lake and its capabilities for handling slowly changing dimensions (SCD Type 2) and change data capture (CDC)</P> Fri, 27 Dec 2024 20:24:57 GMT https://community.databricks.com/t5/data-engineering/does-autoloader-detect-new-records-in-a-snowflake-table-or-only/m-p/103362#M41418 Alberto_Umana 2024-12-27T20:24:57Z