Retrieving Last Data and Metadata Refresh DateTimes of table in Databricks

gayatrikhatale
Contributor

‎09-18-2025 10:22 AM

Hi, 

I had a query regarding how to accurately retrieve the last data refresh datetime and last metadata refresh datetime for tables in Databricks. Currently, the only reliable approach I am aware of is using the DESCRIBE HISTORY command with filters on operations column such as WRITE, UPDATE, DELETE, MERGE, or ALTER TABLE.

However, I could not find a comprehensive list of all possible operations that can appear in the operation column of DESCRIBE HISTORY so I can separate them. The documentation here — Databricks Delta Table History — only provides examples of operation metric key values, but not the full list of supported operations.

Could you please help me.

Thank you!

0 Kudos

szymon_dybczak
Esteemed Contributor III

‎09-18-2025 10:46 AM

Hi @gayatrikhatale,

I think you already found correct place in documentation. It lists all supported operations in the first column.

https://docs.databricks.com/aws/en/delta/history?#operation-metrics-key

0 Kudos

gayatrikhatale
Contributor
In response to szymon_dybczak

‎09-18-2025 10:53 AM

Hi @szymon_dybczak ,

Thank you for your response.

If this is the correct documentation, then why am I not able to see operations like ADD COLUMNS or DROP COLUMNS listed there? I have observed these operation types in the output of DESCRIBE HISTORY, but they don’t seem to be documented in the link.

Thank you!

szymon_dybczak
Esteemed Contributor III
In response to gayatrikhatale

‎09-18-2025 12:13 PM

Thanks for information. To be honest I've always thought that this table contains all supported operations. I learn everyday something new 🙂

0 Kudos

siva-anantha
Contributor

‎09-18-2025 11:08 AM

Hi, 

I had a similar need once and referred the delta-io git repository for full list of metrics. It is the not the best way but we get an idea of what all is available in the metrics. Hope it helps.

Regards,
Siva

szymon_dybczak
Esteemed Contributor III
In response to siva-anantha

‎09-18-2025 01:22 PM

Hi @siva-anantha ,

This is super useful, thanks for sharing. I've prepared list of all distinct operation based on source code for reference:

 

  • WRITE
  • STREAMING UPDATE
  • DELETE
  • UPDATE
  • MERGE
  • TRUNCATE
  • CREATE TABLE
  • REPLACE TABLE
  • ADD COLUMNS
  • DROP COLUMNS
  • RENAME COLUMN
  • CHANGE COLUMN
  • CHANGE COLUMNS
  • REPLACE COLUMNS
  • UPDATE SCHEMA
  • UPDATE COLUMN METADATA
  • SET TBLPROPERTIES
  • UNSET TBLPROPERTIES
  • UPGRADE PROTOCOL
  • DROP FEATURE
  • REMOVE COLUMN MAPPING
  • OPTIMIZE
  • REORG
  • VACUUM START
  • VACUUM END
  • COMPUTE STATS
  • RESTORE
  • CLONE
  • CONVERT
  • ADD CONSTRAINT
  • DROP CONSTRAINT
  • CLUSTER BY
  • UPGRADE UNIFORM PROPERTIES
  • ROW TRACKING BACKFILL
  • ROW TRACKING UNBACKFILL
  • Manual Update
  • Empty Commit
  • Deletion Vector Tombstones
  • Domain Metadata Cleanup

 

View solution in original post

gayatrikhatale
Contributor

‎09-19-2025 02:44 AM

Thank you @szymon_dybczak , @siva-anantha !