Persisting query history data

WWoman
Databricks Partner

Hello,

I am looking for a way to persist query history data. I have not have direct access to the system tables. I do have access to a query_history view created by selecting from the system.query.history and system.access.audit system tables. I want to create a simple process that will incrementally load the data that I access through the view into a table..

What is the simplest way to achieve this?

Thanks

yuriy
Databricks Employee
Databricks Employee

Hi @WWoman you can set up a simple append only DLT pipeline or just schedule a job that will run merge statement insert * only. In both cases you will get a table that will accumulate history of additions in your view over time and in both cases it will be incremental

View solution in original post

syed_sr7
New Contributor II

Is any system table there for query history?