K_Anudeep
Databricks Employee
Databricks Employee

Hello @a_user12 ,

deltaLogRetentionDuration is the interval after which the delta log files will be removed from the delta log. Delta Lake adheres to a set of internal rules to clean up the delta log when the retention duration is exceeded.

Setting delta.logRetentionDuration alone will not automatically remove delta log files. Databricks uses a specific internal cleanup logic, and log files are only deleted once both retention criteria and the required checkpoint files are present. In other words, simply configuring the retention duration does not directly trigger the removal of delta log files: Delta Lake manages delta log deletion asynchronously, and deletes only files eligible per retention rules after periodic checkpointing

Anudeep

View solution in original post