Hi all — I’m running a Medallion streaming pipeline on Databricks using DLT (bronze → staging silver view → silver table). I ran into an issue and would appreciate any advice or best practices.

What I’m doing

• Ingesting streaming data into a streaming bronze table.

• Creating a staging silver view that applies transformations.

• Writing to a streaming silver table.

The problem
When I delete a record in the bronze table, it breaks downstream because deletes aren’t supported in my streaming target. To work around this I enabled options to ignore deletes / change commits, which prevents the error — but now the pipeline no longer picks up from the previous checkpoint. Instead it re-ingests all data into the silver layer (effectively a full reprocessing).

I know a full refresh fixes the checkpoint mismatch, but that’s not acceptable for us: we have very large volumes and upstream cdl_ingested_date that would be refreshed on a full reload, so a full refresh undesirable.

Questions

1. What is the recommended/best-practice approach for handling deletes in a streaming Medallion pipeline on Databricks (DLT)?

2. Is there a safe way to ignore deletes without forcing a full re-ingest / losing the previous checkpoint?

3. If rolling back or recovering streaming tables, what are considered best practise?