topic Re: Service Principal Can’t Access Full Query History via queryHistory/list API in Data Governance

Service Principal Can’t Access Full Query History via queryHistory/list API

lio-illumex — Mon, 28 Jul 2025 13:58:13 GMT

I have a weird issue with Databricks.
I'm using the query history API:
https://docs.databricks.com/api/workspace/queryhistory/list
When I'm using the API with Personal Access Token, it works great.

When i'm using the API with Service Principal, i'm getting only the history of the service principal.
I made sure that Service Principal has "Can Manage" Permission to the SQL Warehouse.
In addition, I added the Service Principal to the "admins" group but it didn't help.

This is very critical for one of our customers.

Thanks!

Re: Service Principal Can’t Access Full Query History via queryHistory/list API

SP_6721 — Tue, 29 Jul 2025 16:47:06 GMT

Hi @lio-illumex ,

Have you applied the filter while making the API call?
Try adding all target user IDs to the filter_by.user_ids field in your request body—this might help retrieve the full query history you're expecting.

Re: Service Principal Can’t Access Full Query History via queryHistory/list API

nayan_wylde — Tue, 29 Jul 2025 17:55:30 GMT

@lio-illumex I think the API gets a paginated response. you need to hit the API recursively untill it reached the last page. Loop in until you get next_page_token. I would say the best way is to use the databricks SDK. It is simpler then the APIs.

https://databricks-sdk-py.readthedocs.io/en/stable/workspace/sql/query_history.html

Re: Service Principal Can’t Access Full Query History via queryHistory/list API

jhonm_839 — Wed, 30 Jul 2025 04:23:01 GMT

The Query History API is scoped to the calling identity, so even with admin rights, service principals can only access their own queries—cross-user access isn't supported.

Re: Service Principal Can’t Access Full Query History via queryHistory/list API

shkolar — Wed, 30 Jul 2025 15:17:28 GMT

@SP_6721 I don't filter by user ids in both scenarios, it shouldn't matter
@nayan_wylde I also tried with the SDK but it's the same API call behind the scenes. I made sure it's not a pagination issue.
@jhonm_839 This is so weird. PAT and SP have the same admin rights.... and the admin PAT can see the SP queries.... I don't understand how it's not possible the other way around

Re: Service Principal Can’t Access Full Query History via queryHistory/list API

nayan_wylde — Thu, 31 Jul 2025 01:24:09 GMT

Is your workspace Unity Catalog Enabled? If yes try to give the SPN metastore admin permissions. Hopefully it should resolve the issue.