saurabh18cs
Honored Contributor III

Hi @minhhung0507 

behavior is not a bug but rather an inherent aspect of how the VACUUM operation functions. VACUUM do not delete from _delta_log folder, this folder has its own default retention of 30 days:

  • Delta Lake maintains a transaction log (_delta_log directory) that records all changes to the table. This log ensures ACID transactions and allows for time travel and versioning.
  • The transaction log contains metadata about the files that make up the table at any given point in time.

 

so upto you to decide how much time travel or versioning you want for your data as data files takes storage space into consideration so keeping default 7 days is good , anything greater than this incurs storage cost and lesser is the risk of having very less retention. matching to 30 days as _delta_logs is good but cost and your usecase applies here.

View solution in original post