Trying to vacuum a table that is constantly being "createdOrReplaced"
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-27-2023 04:36 AM
and it seems that older data (From the "replaced" table) isn't being removed, long after the retention period
I'd be glad for clues on how to handle this
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-21-2023 12:36 PM
Eligibility of a file for vacuum is based on the date that the file was ,asked for deletion and not on the creation timestamp of the file.
Delta table data files are deleted according to the time they have been logically removed from Delta’s transaction log plus retention hours, not their modification timestamps on the storage system.
Can you please confirm that the files were marked for removal before the retention period to make sure that they are eligible.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-23-2023 07:05 AM
Thanks, I know that, but the table history shows 30 days, but the actual data size and number of files and all other indicators , correlate to 170 days.