Deleted File Retention Duration in Cluster not work

[William_Scardua]

Hi Guys,

I tried to set the retention period in my Cluster, but it`s not work

Cluster:

Notebook:

Does not remove physical files

Do you have any ideas? thanks

[Sidhant07]

Hi,

It seems you are setting the retention period to 0 hours.

You need to first use this config spark.databricks.delta.retentionDurationCheck.enabled to false.

and then set the retentention duration config as you used.

You can do a DRY RUN to check if there are any files which are ready to be vacummed.

Reference link: https://docs.databricks.com/en/sql/language-manual/delta-vacuum.html