Default delta log retention interval is different than in documentation?

442027 — Thu, 06 Jul 2023 20:43:57 GMT

It notes in the documentation here that the default delta log retention interval is 30 days - however when I create checkpoints in the delta log to trigger the cleanup - historical records from 30 days aren't removed; i.e. current day checkpoint is at 2023-07-06 and historical records from 2023-04-27 are still remaining.

It's not until I specifically set:

```

ALTER TABLE <table> SET TBLPROPERTIES('delta.logRetentionDuration' = 'interval 30 days')

```

that after creating checkpoints those records from 2023-04-27 are removed as expected. Is there a different default log retention date than what is mentioned here? Or some other hidden process that I am missing?

Re: Default delta log retention interval is different than in documentation?

jose_gonzalez — Wed, 24 Jan 2024 23:50:23 GMT

you need to set SET TBLPROPERTIES ('delta.checkpointRetentionDuration' = '30 days',)

topic Default delta log retention interval is different than in documentation? in Data Engineering

Default delta log retention interval is different than in documentation?

Re: Default delta log retention interval is different than in documentation?