- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-25-2025 03:21 AM - edited 08-25-2025 03:24 AM
Hey @Rainier_dw, have you read up much on CDC? https://www.databricks.com/blog/2022/04/25/simplifying-change-data-capture-with-databricks-delta-liv... . I wonder if this is an appropriate fit.
I'm currently going through the Data Engineering certification and have stumbled across the MERGE INTO command. It's also referenced in the article above. That provides a really great functionality for "when [not] matched then {delete|insert|update}: https://www.databricks.com/blog/2019/03/19/efficient-upserts-into-data-lakes-databricks-delta.html .. perhaps this is a good angle to leverage this.
I guess this doesn't address your 3rd question, I'm also interested in that one. Have you tried using the Time Travel functionality with Delta Tables? How have you found that to behave with Streaming Tables?
All the best,
BS