topic Re: Need to track the schema changes/column renames/column drops in Data bricks Unity Catalog in Data Engineering

Need to track the schema changes/column renames/column drops in Data bricks Unity Catalog

data_learner1 — Wed, 18 Jun 2025 16:48:01 GMT

Hi Team,

We are getting data from third party vendor to the databricks unity Catalog. They are doing schema changes frequently and we would like to track that. Just wanted to know if I can do this using audit table on the system catalog. As we only have read permissions, we may not have an access but wanted to ask them to create a view for us. does wanted to understand if the audit table logs can help us in this scenario? please let us know if anyone of you has a better solution or how you are doing it. Please provide me some insights on this.

Thank you

Re: Need to track the schema changes/column renames/column drops in Data bricks Unity Catalog

KaranamS — Wed, 18 Jun 2025 18:14:32 GMT

Hi @data_learner1 ,

Audit logs focus on security and usage monitoring such as user access, table read/write events. They don't track schema level changes.

To track schema level changes, delta transaction logs will be the best to use. The transaction log files are typically stored in a _delta_log directory within the table's root directory. Here is a sample code snippet you can use to query the log files for a table.

df = spark.read.json("/path/to/your/delta/table/_delta_log/*.json") df.select("commitInfo","metaData").display()

Hope this helps!

Re: Need to track the schema changes/column renames/column drops in Data bricks Unity Catalog

data_learner1 — Fri, 20 Jun 2025 11:19:29 GMT

@KaranamS : for suppose, i have list of tables in databricks unity catalog, with the read access I wanted to find out if they are doing any renames to the existing tables? how can we do this?

Thank you for your suggestions.

Re: Need to track the schema changes/column renames/column drops in Data bricks Unity Catalog

KaranamS — Wed, 25 Jun 2025 17:47:46 GMT

@data_learner1 , Other alternative to track the table renames would be snapshots. You can run 'SHOW TABLES' on a catalog and save the output on daily basis and compare current vs previous to find out dropped or renamed tables

Re: Need to track the schema changes/column renames/column drops in Data bricks Unity Catalog

CURIOUS_DE — Wed, 25 Jun 2025 18:02:34 GMT

@data_learner1

Unity Catalog logs all data access and metadata operations (including schema changes) into the audit logs — which are stored in the system catalog tables, such as:

system.access.audit

You mentioned you only have read access — and likely no access to system.access.audit, which is only visible to metastore admins or users with elevated privileges.