- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-09-2024 05:12 AM
Hi @LasseL, To manage old data in Delta Live Tables (DLT), use a combination of Delta Lake's features and DLT capabilities: start by using Delta Lake's `DELETE` command to remove outdated records, for example, by deleting entries older than one month. Integrate this deletion logic into your DLT pipeline with a Python function that filters out old data. Schedule regular maintenance tasks using Databricks Jobs to automate this process. After deletion, run the `OPTIMIZE` command to enhance query performance and use `VACUUM` to clean up unused files and free up storage space. These steps will help keep your storage usage in check and ensure efficient data management.