A Surprise Findings in Delta Live Table

[CURIOUS_DE]

While DLT has some powerful features, I found myself doing a double-take when I realized it doesn’t natively support hard deletes. Instead, it leans on a delete flag identifier to manage these in the source table. A bit surprising for a tool of its caliber, don’t you think? @Sujitha 

Let's pass it on to all Databricks Community User

[VZLA]

Thanks for your feedback !

I believe, Delta Live Tables (DLT) does not natively support hard deletes and instead uses a delete flag identifier to manage deletions, a design choice rooted in ensuring compliance with regulations like GDPR and CCPA. This approach facilitates traceability, auditing, and the "Right to be Forgotten", avoiding the complexities and risks of immediate hard deletes in large-scale data environments. Additionally, it leverages Delta Lake’s time travel and retention features to maintain data integrity, support recovery, and prevent disruptions to downstream pipelines, striking a balance between governance, compliance, and operational reliability.

https://www.databricks.com/blog/handling-right-be-forgotten-gdpr-and-ccpa-using-delta-live-tables-dl...