cancel
Showing results for 
Search instead for 
Did you mean: 

Handling GDPR requests in databricks

Hubert_Dudek1
Esteemed Contributor III

When dealing with GDPR requests in databricks, there are some essential things to keep in mind:

- Use a low retention period to ensure you don't keep table delta version history for tables with personal information.

- Use APPLY CHANGES to handle Slowly Changing Dimension type 1. This way, you won't track history (like in type 2) and will have it in a separate table.

- When handling customer insertion and GDPR requests, use a changed data feed in databricks. Ensure the table is declared as LIVE, not STREAM, to ensure complete data reload and avoid records for which we have received GDPR requests.

ezgif-3-020e69a4fd.gif

1 REPLY 1

jose_gonzalez
Moderator
Moderator

Thank you for sharing this information @Hubert_Dudek1!!!!

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.