Log files are not getting deleted automatically after logRetentionDuration internal
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-18-2023 11:43 AM
Hi team
Log files are not getting deleted automatically after logRetentionDuration internal from delta log folder and after analysis , I see checkpoint files are not getting created after 10 commits .
Below table properties using
spark.sql( f"""
ALTER TABLE Table_name
SET TBLPROPERTIES (
delta.logRetentionDuration = 'interval 1 hours',
delta.deletedFileRetentionDuration = 'interval 1 hours',
delta.checkpointRetentionDuration = '1 hours'
)
"""
No checkpoint.parquet file getting generated in delta log
Labels:
- Labels:
-
Delta Log Folder
-
TABLE Table