- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-22-2025 05:43 PM - edited 09-22-2025 05:53 PM
@Anubhav2011, here is more information on the snapshot method: https://docs.databricks.com/aws/en/dlt/cdc#how-is-cdc-implemented-with-the-auto-cdc-from-snapshot-ap...
This process efficiently determines changes in source data by comparing a series of snapshots taken in order. It then executes the necessary processing for change data capture (CDC) of the records in those snapshots. This functionality is supported exclusively by the Lakeflow Declarative Pipelines Python interface.
By the end of the year, there are plans to incorporate TTL (time to live) functionality for Delta tables (both managed and DLT tables). The timeline may change, but in the meantime, you can set up a job with vacuum.
https://docs.databricks.com/aws/en/sql/language-manual/delta-vacuum