Hi @paolajara — in your point 5 you mentioned using Delta Lake for tracking changes. Could you point me to any official docs or examples that walk through enabling CDC / row-tracking on a Delta table?

I pull data from SharePoint via its REST endpoint, which gives me full snapshots but no explicit delete flags. Because hard-deletes aren’t surfaced directly, I need to compare each new snapshot with the previous one to spot rows that have disappeared and propagate those deletes downstream in Databricks. Do you have a recommended pattern (row-tracking + anti-join, CDF etc.) for implementing this? Any guidance or links would be much appreciated—thanks!