Hi, no worries @minhhung0507 .

Check and inspect if the _delta_log files in question still reference data files that fall within the retention period. Delta Lake retains logs for files that are still active or could be required for transactional consistency and time travel.

If you notice inconsistencies across different tables, it might be due to differences in how checkpoints are created or how the retention period is managed for each table, check the table properties and default values wherever these are not set. Ensure that the configurations for retention and checkpointing are consistent across all tables. [1]

Delta Lake's log files are deleted automatically and asynchronously after checkpoint operations. If this is not happening, there might be an issue with the cleanup mechanism itself

Try running the VACUUM command on the Delta table to help in removing data files that are no longer referenced by the table. However, note that the VACUUM command does not govern the deletion of log files. Log files are managed separately and are deleted after checkpoint operations. [1]

[1] https://docs.databricks.com/en/delta/history.html#configure-data-retention-for-time-travel-queries