Delete S3 files after vacuum

wwarner
New Contributor

‎12-20-2023 02:56 PM

Hi,

I'm trying to purge a table of stale data. My databricks host is on cloud.databricks.com.

I've set delta.deletedFileRetentionDuration=interval 7 days, deleted many (billions) rows, and followed up with VACUUM tablename RETAIN 168 HOURS, however my databricks filesystem is still full of stale files.

How can I get rid of those files?

Thanks --Bill

0 Kudos

shan_chandra
Databricks Employee
Databricks Employee

‎01-11-2024 12:42 PM

Hi Bill - Based on the delta delete file retention period, the files will no longer be there after 7 days .

Thanks, Shan

0 Kudos