Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-22-2025 05:55 AM
Hi @dalcuovidiu ,
See if this makes sense to you: given a full-snapshot source without a delete flag, keep Bronze immutable, materialize a deduplicated current snapshot in staging (e.g., via ROW_NUMBER() on ingest_ts per id), and apply an idempotent Delta MERGE in Silver with NOT MATCHED BY SOURCE to mark soft deletes (is_deleted/deleted_at), guarded by a watermark/grace window for late data, CDF enabled for downstream propagation, performance guardrails (partitioning/Z-ORDER on id, Deletion Vectors), and alert thresholds to prevent flapping.
Wiliam Rosa
Data Engineer | Machine Learning Engineer
LinkedIn: linkedin.com/in/wiliamrosa
Data Engineer | Machine Learning Engineer
LinkedIn: linkedin.com/in/wiliamrosa