โ01-13-2022 03:05 AM
โ01-13-2022 04:44 AM
Hi @Borislav Blagoevโ ,
Vacuum cleans up files associated with a table.
Note:-
This command works differently depending on whether youโre working on a Delta or Apache Spark table.
Vacuum a Delta table (Delta Lake on Databricks)
Recursively vacuum directories associated with the Delta table and remove data files that are no longer in the latest state of the transaction log for the table and are older than a retention threshold. Files are deleted according to the time they have been logically removed from Deltaโs transaction log + retention hours, not their modification timestamps on the storage system. The default threshold is 7 days.
Vacuum a Spark table (Apache Spark)
Recursively vacuums directories associated with the Spark table and remove uncommitted files older than a retention threshold. The default threshold is 7 days.
On Spark tables, Databricks automatically triggers
VACUUM operations as data are written.
โ01-13-2022 04:44 AM
Hi @Borislav Blagoevโ ,
Vacuum cleans up files associated with a table.
Note:-
This command works differently depending on whether youโre working on a Delta or Apache Spark table.
Vacuum a Delta table (Delta Lake on Databricks)
Recursively vacuum directories associated with the Delta table and remove data files that are no longer in the latest state of the transaction log for the table and are older than a retention threshold. Files are deleted according to the time they have been logically removed from Deltaโs transaction log + retention hours, not their modification timestamps on the storage system. The default threshold is 7 days.
Vacuum a Spark table (Apache Spark)
Recursively vacuums directories associated with the Spark table and remove uncommitted files older than a retention threshold. The default threshold is 7 days.
On Spark tables, Databricks automatically triggers
VACUUM operations as data are written.
โ01-13-2022 04:55 AM
Thanks!
โ01-13-2022 05:00 AM
Hi @Borislav Blagoevโ , If that answers your question would you like to mark it as the best answer?
โ01-13-2022 05:18 AM
I can't see where is the button for that
โ01-13-2022 05:26 AM
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.