https://community.databricks.com/t5/data-engineering/how-to-use-dlt-module-in-streaming-pipeline/m-p/53251#M29752 <P>If anyone has example code for building a CDC live streaming pipeline generated by AWS DMS using import dlt, I'd love to see it.<BR />I'm currently able to see the parquet file starting with Load on the first full load to S3 and the cdc parquet file after that, but it doesn't work with @dlt.create_table, so I'd like to see how to do it.</P> Tue, 21 Nov 2023 08:04:41 GMT rt-slowth 2023-11-21T08:04:41Z https://community.databricks.com/t5/data-engineering/how-to-use-dlt-module-in-streaming-pipeline/m-p/53251#M29752 <P>If anyone has example code for building a CDC live streaming pipeline generated by AWS DMS using import dlt, I'd love to see it.<BR />I'm currently able to see the parquet file starting with Load on the first full load to S3 and the cdc parquet file after that, but it doesn't work with @dlt.create_table, so I'd like to see how to do it.</P> Tue, 21 Nov 2023 08:04:41 GMT https://community.databricks.com/t5/data-engineering/how-to-use-dlt-module-in-streaming-pipeline/m-p/53251#M29752 rt-slowth 2023-11-21T08:04:41Z https://community.databricks.com/t5/data-engineering/how-to-use-dlt-module-in-streaming-pipeline/m-p/102705#M41206 <P>There is a blogpost for this that includes example code that you can find <A href="https://www.databricks.com/blog/using-streaming-delta-live-tables-and-aws-dms-change-data-capture-mysql" target="_self">here</A></P> Thu, 19 Dec 2024 17:49:23 GMT https://community.databricks.com/t5/data-engineering/how-to-use-dlt-module-in-streaming-pipeline/m-p/102705#M41206 cgrant 2024-12-19T17:49:23Z