โ02-18-2023 11:43 AM
Hi team
Log files are not getting deleted automatically after logRetentionDuration internal from delta log folder and after analysis , I see checkpoint files are not getting created after 10 commits .
Below table properties using
spark.sql( f"""
ALTER TABLE Table_name
SET TBLPROPERTIES (
delta.logRetentionDuration = 'interval 1 hours',
delta.deletedFileRetentionDuration = 'interval 1 hours',
delta.checkpointRetentionDuration = '1 hours'
)
"""
No checkpoint.parquet file getting generated in delta log
โ02-20-2023 03:40 AM
Hi @vinay kumarโ , Could you please run "SHOW TBLPROPERTIES <table name>" command and attach a screenshot here?
โ02-20-2023 04:04 AM
Please find TBLPROPERTIES
โ02-20-2023 04:16 AM
Hi @vinay kumarโ , could you please answer the below questions as well
โ02-20-2023 04:19 AM
โ02-20-2023 04:48 AM
Hi @vinay kumarโ , It is difficult to tell what is causing this issue. I advise you to raise a support case with us to have a closer look. You can raise a support case using the link: https://help.databricks.com/
โ04-21-2023 02:26 AM
Hi @vinay kumarโ
Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.
We'd love to hear from you.
Thanks!
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group