topic Is VACUUM operation recorded in the history of the delta table? in Data Engineering

Is VACUUM operation recorded in the history of the delta table?

cristianc — Mon, 21 Feb 2022 13:45:10 GMT

Greetings,

I have tried using Spark with DBR 9.1 LTS to run VACUUM on my delta table then DESCRIBE HISTORY to see the operation, but apparently the VACUUM operation was not in the history despite the things stated in the documentation from: https://docs.databricks.com/delta/delta-utility.html#3

From my understanding there should be a "VACUUM" operation in the history table. Can you please confirm if my understanding is correct, or if not can you please improve my understanding why not?

Regards,

Cristian

Re: Is VACUUM operation recorded in the history of the delta table?

AmanSehgal — Tue, 22 Feb 2022 01:39:34 GMT

To capture the logs for VACUUM operation enable vacuum logging.

spark.conf.set("spark.databricks.delta.vacuum.logging.enabled",True)

On doing so, you'll see two records in the history logs.

VACUUM START and
VACUUM END

Re: Is VACUUM operation recorded in the history of the delta table?

cristianc — Tue, 22 Feb 2022 08:44:46 GMT

That makes sense, thanks for the reply!