Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-15-2025 07:59 PM
Hi @minhhung0507 ,
You must choose an interval that is longer than the longest running concurrent transaction and the longest period that any stream can lag behind the most recent update to the table. So that Vaccum tables cannot be corrupted when VACUUM deletes files that have not yet been committed.
And also there is a safety check to check whether there are no operations being performed on this table that take longer than the retention interval you plan to specify, you can turn off/on this safety check by setting the Spark configuration property spark.databricks.delta.retentionDurationCheck.enabled to false.
Hope this helps!!!