how to use dlt module in streaming pipeline
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-21-2023 12:04 AM
If anyone has example code for building a CDC live streaming pipeline generated by AWS DMS using import dlt, I'd love to see it.
I'm currently able to see the parquet file starting with Load on the first full load to S3 and the cdc parquet file after that, but it doesn't work with @dlt.create_table, so I'd like to see how to do it.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-19-2024 09:49 AM
There is a blogpost for this that includes example code that you can find here