tyler-xorbix
Databricks Partner

Hi Trilleo,

By default, a Delta Live Table acts as a materialized view, meaning it automatically updates based on its dependencies. This functionality allows for straightforward handling of data changes and dependencies without additional manual intervention.

However, your scenario seems to involve a more complex situation. This requires creating custom merge logic with a Change Data Capture (CDC) feed.

CDC is a feature in Delta Lake that helps track changes (inserts, updates, and deletes) in a table. To handle updates from your bronze table and ensure they are accurately reflected in the silver table, you will need to implement custom merge logic. This involves reading the streaming data from the bronze table and applying merge operations to update or insert records into the silver table based on a key column.

Here is a Databricks Blog overviewing CDC with custom merge logic: Change Data Capture With Delta Live Tables - The Databricks Blog.

This approach ensures that updates in the bronze table are correctly reflected in the silver table without adding duplicate entries, providing a more tailored solution to handle your specific needs.

Let me know if you need further clarification or assistance!

View solution in original post