- 10767 Views
- 7 replies
- 0 kudos
Log files are not getting deleted automatically after logRetentionDuration internal
Hi team Log files are not getting deleted automatically after logRetentionDuration internal from delta log folder and after analysis , I see checkpoint files are not getting created after 10 commits . Below table properties using spark.sql( f""" ...
- 10767 Views
- 7 replies
- 0 kudos
- 7144 Views
- 7 replies
- 9 kudos
Clean up _delta_log files
Hello experts. We are trying to clarify how to clean up the large amount of files that are being accumulated in the _delta_log folder (json, crc and checkpoint files). We went through the related posts in the forum and followed the below:SET spark.da...
- 7144 Views
- 7 replies
- 9 kudos
- 9 kudos
What you’re seeing is expected behavior — the _delta_log folder always keeps a history of JSON commit files, checkpoint files, and CRCs. Even if you lower delta.logRetentionDuration and run VACUUM, cleanup won’t happen immediately. A couple of points...
- 9 kudos