Showing results for 
Search instead for 
Did you mean: 

Handling GDPR requests in databricks

Esteemed Contributor III

When dealing with GDPR requests in databricks, there are some essential things to keep in mind:

- Use a low retention period to ensure you don't keep table delta version history for tables with personal information.

- Use APPLY CHANGES to handle Slowly Changing Dimension type 1. This way, you won't track history (like in type 2) and will have it in a separate table.

- When handling customer insertion and GDPR requests, use a changed data feed in databricks. Ensure the table is declared as LIVE, not STREAM, to ensure complete data reload and avoid records for which we have received GDPR requests.




Thank you for sharing this information @Hubert_Dudek1!!!!

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.