Need to track the schema changes/column renames/column drops in Data bricks Unity Catalog
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-18-2025 09:48 AM
Hi Team,
We are getting data from third party vendor to the databricks unity Catalog. They are doing schema changes frequently and we would like to track that. Just wanted to know if I can do this using audit table on the system catalog. As we only have read permissions, we may not have an access but wanted to ask them to create a view for us. does wanted to understand if the audit table logs can help us in this scenario? please let us know if anyone of you has a better solution or how you are doing it. Please provide me some insights on this.
Thank you
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-18-2025 11:14 AM
Hi @data_learner1 ,
Audit logs focus on security and usage monitoring such as user access, table read/write events. They don't track schema level changes.
To track schema level changes, delta transaction logs will be the best to use. The transaction log files are typically stored in a _delta_log directory within the table's root directory. Here is a sample code snippet you can use to query the log files for a table.
df = spark.read.json("/path/to/your/delta/table/_delta_log/*.json")
df.select("commitInfo","metaData").display()
Hope this helps!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-20-2025 04:19 AM
@KaranamS : for suppose, i have list of tables in databricks unity catalog, with the read access I wanted to find out if they are doing any renames to the existing tables? how can we do this?
Thank you for your suggestions.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-25-2025 10:47 AM
@data_learner1 , Other alternative to track the table renames would be snapshots. You can run 'SHOW TABLES' on a catalog and save the output on daily basis and compare current vs previous to find out dropped or renamed tables
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-25-2025 11:02 AM
Unity Catalog logs all data access and metadata operations (including schema changes) into the audit logs — which are stored in the system catalog tables, such as:
system.access.audit
You mentioned you only have read access — and likely no access to system.access.audit, which is only visible to metastore admins or users with elevated privileges.