topic get saved query by name using rest API or databricks SDK in Data Engineering

get saved query by name using rest API or databricks SDK

mahfooziiitian — Fri, 09 Aug 2024 15:21:26 GMT

Hi All,

I want to get the saved query by name using rest API or databricks SDK. So It do not find any direct end point or method which can give us the saved query by name.

I have one solution as given below:

get the list all queries
filter the my queries from list of queries

But databricks is saying that if you are calling list all queries end point (List queries ) multiple then it might ban temporarily.

Can you suggest us how can I get the query without using list all queries end point?

Re: get saved query by name using rest API or databricks SDK

szymon_dybczak — Sat, 10 Aug 2024 16:53:26 GMT

Hi @mahfooziiitian ,

The answer is no, currently you can get saved query only by id. If your are afraid of exceeding concurrent calls, then design a process that as a first step will use list queries endpoint to extract endpoint IDs and names and save that information to a Delta Table. Then you can lookup endpoint names, IDs in that table and use it to download specific query in later steps of the process. That's how you can avoid exceeding API limits

Re: get saved query by name using rest API or databricks SDK

mahfooziiitian — Sun, 11 Aug 2024 12:18:14 GMT

I have tried this solution but problem with this solution is that new query details will not be available until we are preloading the data again. I am thinking of using pagination by using attributes page_token,next_page_token,page_size and loop through it until we find the query. This might not exceed concurrent API Limit. I need to just verify it with databricks team.

Re: get saved query by name using rest API or databricks SDK

szymon_dybczak — Mon, 12 Aug 2024 13:59:23 GMT

Cool that you found workaround that works for you.