VZLA
Databricks Employee
Databricks Employee

Thanks for reporting this Sangram. Are these youtube and educational contents in the Databricks channel?

> set delta.databricks.delta.retentionDurationCheck.enabled = false. It works if I want to delete obsolete files whose lifespan is less than default retention duration.

That's fine as long as you know this also introduces a risk: any files essential for tracking data changes, maintaining historical versions, or supporting CDF operations could be deleted prematurely. This could result in unintentional data loss, such as loss of previous data states or inability to access certain changes, impacting versioning and downstream processes.