Are Delta tables able to support GDPR compliance?

[User16826992666]

I know that when deletes are made from a Delta table the underlying files are not actually removed. For compliance reasons I need to able to truly delete the records. How can I know which files need to be removed, and is there a way to remove them other than manually?

[sajith_appukutt · Accepted solution]

Here is a document explaining best practices for GDPR and CCPA compliance using Delta Lake.

Specifically on cleaning up stale data - you can use the VACUUM function to remove files that are no longer referenced by a Delta table and are older than a specified retention threshold - preventing the ability to view history and roll back.

For e.g.

VACUUM gdpr.customers RETAIN 100 HOURS