https://community.databricks.com/t5/data-engineering/are-delta-tables-able-to-support-gdpr-compliance/m-p/23777#M16481 <P>Here is a document explaining <A href="https://docs.databricks.com/security/privacy/gdpr-delta.html" alt="https://docs.databricks.com/security/privacy/gdpr-delta.html" target="_blank">best practices for GDPR and CCPA compliance using Delta Lake</A>. </P><P></P><P>Specifically on cleaning up stale data - you can use the&nbsp;<A href="https://docs.databricks.com/delta/delta-utility.html#delta-vacuum" alt="https://docs.databricks.com/delta/delta-utility.html#delta-vacuum" target="_blank">VACUUM</A>&nbsp;function to remove files that are no longer referenced by a Delta table and are older than a specified retention threshold - preventing the ability to view history and roll back.</P><P></P><P>For e.g.</P><PRE><CODE>VACUUM gdpr.customers RETAIN 100 HOURS</CODE></PRE><P></P> Fri, 18 Jun 2021 20:16:37 GMT sajith_appukutt 2021-06-18T20:16:37Z https://community.databricks.com/t5/data-engineering/are-delta-tables-able-to-support-gdpr-compliance/m-p/23776#M16480 <P>I know that when deletes are made from a Delta table the underlying files are not actually removed. For compliance reasons I need to able to truly delete the records. How can I know which files need to be removed, and is there a way to remove them other than manually?</P> Wed, 16 Jun 2021 17:57:59 GMT https://community.databricks.com/t5/data-engineering/are-delta-tables-able-to-support-gdpr-compliance/m-p/23776#M16480 User16826992666 2021-06-16T17:57:59Z https://community.databricks.com/t5/data-engineering/are-delta-tables-able-to-support-gdpr-compliance/m-p/23777#M16481 <P>Here is a document explaining <A href="https://docs.databricks.com/security/privacy/gdpr-delta.html" alt="https://docs.databricks.com/security/privacy/gdpr-delta.html" target="_blank">best practices for GDPR and CCPA compliance using Delta Lake</A>. </P><P></P><P>Specifically on cleaning up stale data - you can use the&nbsp;<A href="https://docs.databricks.com/delta/delta-utility.html#delta-vacuum" alt="https://docs.databricks.com/delta/delta-utility.html#delta-vacuum" target="_blank">VACUUM</A>&nbsp;function to remove files that are no longer referenced by a Delta table and are older than a specified retention threshold - preventing the ability to view history and roll back.</P><P></P><P>For e.g.</P><PRE><CODE>VACUUM gdpr.customers RETAIN 100 HOURS</CODE></PRE><P></P> Fri, 18 Jun 2021 20:16:37 GMT https://community.databricks.com/t5/data-engineering/are-delta-tables-able-to-support-gdpr-compliance/m-p/23777#M16481 sajith_appukutt 2021-06-18T20:16:37Z