cancel
Showing results for 
Search instead for 
Did you mean: 
Lakebase Discussions
Ask questions, share challenges, and connect with others working on Lakebase. From troubleshooting to best practices, this is where conversations happen.
cancel
Showing results for 
Search instead for 
Did you mean: 

Inquiry regarding Query History and Audit Logs for Databricks Lakebase

POCUSER
New Contributor II
We are using Lakebase Data API (HTTP Endpoint) to execute queries and need to verify the audit log capabilities for compliance. Could you please clarify:
  1. Query Text Logging: Does Databricks capture the full SQL statement text and the actual user identity for every request sent via the Lakebase Data API?
  2. Log Location: Where are these Data API history logs stored (e.g., in system.query.history, Audit Logs in Storage Bucket, or via Unity Catalog)?
  3. Error Logging: Are failed queries (Query Errors) and their detailed error messages recorded in these logs as well?
2 REPLIES 2

balajij8
Contributor III

You can follow below

  • Query History Tracking - You can use the pg_stat_statements entity for getting the historical SQL query execution details and other runtime metrics in the Lakebase instance. Every successful or failed query sent through the Lakebase is parameterized, normalized and logged alongside its total runtime, execution time and other tracking metrics. This entity serves as the primary ledger for tracking precisely what query texts were run. You can periodically offload these metrics if you require persistent retention beyond the life of the compute cluster as the data lives within the database's shared memory, More details here
  • Active Query Monitoring - You can use pg_stat_activity to monitor live requests, active transactions and current connections hitting the Lakebase in real time for monitoring. This entity exposes the immediate state of the database showing he currently executing queries, client IP addresses, query, backend type and operational states. It helps in finding which accounts are running active, long running or blocked statements at any given second.
  • Account Level Audit Logs - You can use system.access.audit for account level governance, infrastructure compliance and broader security perspective. All major activities related to the database - such as provisioning, configuration changes etc are recorded in the system.access.audit table. To isolate these specific events use the service name databaseInstances or postgres for getting the related details.

Ashwin_DSA
Databricks Employee
Databricks Employee

Hi @POCUSER,

I’m not aware of any public documentation that explicitly confirms that every request sent through the Lakebase Data API is captured in system.query.history or exposed through a dedicated per-request audit log.

A practical way to validate it is to run a simple test query through the Data API and then check system.query.history. The public docs for that table currently describe coverage for SQL warehouses and serverless compute, so testing is the best way to confirm whether Lakebase Data API activity appears there in your environment.

It is also worth checking system.access.audit together with the audit log reference to see whether any Lakebase-related control-plane events are emitted there.

If you need a definitive answer specifically for compliance, such as whether full query text, caller identity, and failed request details are logged for Lakebase Data API requests, I’d recommend opening a Databricks support case. Some of the Lakebase observability surfaces are still evolving, so support is the best route for confirmed guidance on the current supported behaviour.

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.

 

Regards,
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***