loui_wentzel
Databricks Partner

Hello,

It's fairly common to not have the SCD col/tag in bronze layer and adding it in silver, as the granularity in bronze (often) don't have collisions on unique ID's and it is rarely something you care about in bronze. Then in silver you either add a valid_from and valid_to, where if valid_to is None, you know it's the active version, OR you have a valid flag and initial date. Both of these will give you history and easy overview of current values.

Automatically propagating deletions (and more) can be quite well handled with Change Data Feed, although it does require some logic and setup.

When you say the bronze does not have information weather it was deleted - when you get a new bronze file, how do you determine a row was deleted? Do you get full loads and can see which are not there, or do you deduce it in some other way?