https://community.databricks.com/t5/data-engineering/when-should-change-data-feed-be-used/m-p/26001#M18138 <P>CDF simplifies the process of identifying the set of records that are updated, inserted, or deleted with each version of a Delta table. It helps to avoid having to implement downstream 'custom' filtration to identify these changes. This makes it an ideal source for servicing changed data to downstream applications.</P><P></P><P>Some things to consider when working with CDF are around the the type and volume of changes. CDF works well where the changes are a mix of insert, update and deletes; CDF may not be the best option when changes are append-only. Also, CDF works well in situations where small fractions of the data set are being changed per batch; when a large fraction of the data set is being changed CDF may not be a good fit.</P> Mon, 07 Jun 2021 18:10:54 GMT patputnam-db 2021-06-07T18:10:54Z https://community.databricks.com/t5/data-engineering/when-should-change-data-feed-be-used/m-p/26000#M18137 <P>IHAC who has a Change Data Capture data flowing into a Delta table. They would like to propagate these changes from this table into another table downstream. Is this a good application for using Change Data Feed?</P> Mon, 07 Jun 2021 17:49:23 GMT https://community.databricks.com/t5/data-engineering/when-should-change-data-feed-be-used/m-p/26000#M18137 patputnam-db 2021-06-07T17:49:23Z https://community.databricks.com/t5/data-engineering/when-should-change-data-feed-be-used/m-p/26001#M18138 <P>CDF simplifies the process of identifying the set of records that are updated, inserted, or deleted with each version of a Delta table. It helps to avoid having to implement downstream 'custom' filtration to identify these changes. This makes it an ideal source for servicing changed data to downstream applications.</P><P></P><P>Some things to consider when working with CDF are around the the type and volume of changes. CDF works well where the changes are a mix of insert, update and deletes; CDF may not be the best option when changes are append-only. Also, CDF works well in situations where small fractions of the data set are being changed per batch; when a large fraction of the data set is being changed CDF may not be a good fit.</P> Mon, 07 Jun 2021 18:10:54 GMT https://community.databricks.com/t5/data-engineering/when-should-change-data-feed-be-used/m-p/26001#M18138 patputnam-db 2021-06-07T18:10:54Z