cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Performance improvement after running VACUUM commands

User16869510359
Esteemed Contributor

How often should I run VACUUM commands? Will running the VACUUM command on a Delta table improve my read/write performance or is it just the storage benefits.

1 ACCEPTED SOLUTION

Accepted Solutions

User16869510359
Esteemed Contributor

VACUUM removes uncommitted/stale files from the Storage. The primary benefit is to save the storage cost. Ideally running VACUUM should not show any performance improvement as Delta does not list the storage directories but rather access the files directly. However in the past due to corner case issues on the storage side, we have seen performance improvement after running VACUUM. Those issues are fixed

View solution in original post

1 REPLY 1

User16869510359
Esteemed Contributor

VACUUM removes uncommitted/stale files from the Storage. The primary benefit is to save the storage cost. Ideally running VACUUM should not show any performance improvement as Delta does not list the storage directories but rather access the files directly. However in the past due to corner case issues on the storage side, we have seen performance improvement after running VACUUM. Those issues are fixed

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.