Duplicate queryId in the query history api response

tanmoy-km
New Contributor

‎11-12-2023 08:01 AM

Hello Experts,

I was using the `/api/2.0/sql/history/queries` end-point to find the historical stats for queries on the warehouse endpoint.

I am surprised to see exact rows duplicated in the response, even though they have different `next_page_token` 

How do you suggest to avoid this, so that I get the actual stats about queries that leverage SQL endpoint

Reference link - https://docs.databricks.com/api/workspace/queryhistory/list

0 Kudos

Emil_Kaminski
Contributor II

‎11-12-2023 02:31 PM

Hi

you can use SDK from your DataBricks notebook. Here comes example how can you do that:

 
 

 

%pip install databricks-sdk --upgrade<div><span>Restart Python kernel<li-code lang="markup">dbutils.library.restartPython()<p>List your queries history:<li-code lang="markup">from databricks.sdk import WorkspaceClient
w = WorkspaceClient()

for q in w.query_history.list():
    print(f"Query: {q.as_dict()['query_id']} \nStart Time: {q.as_dict()['query_start_time_ms']} \
        \nQuery text: {q.as_dict()['query_text']}\n")

 

Everything in the same notebook. 

Good luck

 

** You might also want to subscribe to Warsaw Databricks YT channel: https://www.youtube.com/channel/UC1-u_2nI97cNHtu_FQ3HP_A
0 Kudos

Emil_Kaminski
Contributor II

‎11-12-2023 02:33 PM

Sorry for formatting. This editor drives me crazy. Python code once again:

#Install SDK first
%pip install databricks-sdk --upgrade

#Restart Python kernel
dbutils.library.restartPython() 

#List your queries history:
from databricks.sdk import WorkspaceClient 
w = WorkspaceClient() 
for q in w.query_history.list(): 
    print(f"Query: {q.as_dict()['query_id']} \nStart Time: {q.as_dict()['query_start_time_ms']} \nQuery text: {q.as_dict()['query_text']}\n")
** You might also want to subscribe to Warsaw Databricks YT channel: https://www.youtube.com/channel/UC1-u_2nI97cNHtu_FQ3HP_A
0 Kudos