sajith_appukutt
Databricks Employee
Databricks Employee

If you are running VACUUM with a very short retention interval, old snapshots and uncommitted files can still be in use by concurrent readers or writers to the table and this could result in concurrent readers to fail or tables getting corrupted

Delta Lake has a safety check spark.databricks.delta.retentionDurationCheck.enabled set to true by default to prevent you from running a dangerous  vacuum command .

More details could be found here. https://docs.databricks.com/delta/delta-utility.html#remove-files-no-longer-referenced-by-a-delta-ta...

From a performance point of you, to ensure that it is not affecting the other running jobs, you could run it in a job cluster