cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Handling GDPR requests in databricks

Hubert-Dudek
Esteemed Contributor III

When dealing with GDPR requests in databricks, there are some essential things to keep in mind:

- Use a low retention period to ensure you don't keep table delta version history for tables with personal information.

- Use APPLY CHANGES to handle Slowly Changing Dimension type 1. This way, you won't track history (like in type 2) and will have it in a separate table.

- When handling customer insertion and GDPR requests, use a changed data feed in databricks. Ensure the table is declared as LIVE, not STREAM, to ensure complete data reload and avoid records for which we have received GDPR requests.

ezgif-3-020e69a4fd.gif

1 REPLY 1

jose_gonzalez
Databricks Employee
Databricks Employee

Thank you for sharing this information @Hubert-Dudek!!!!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group