cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Pulling list of running jobs using JOBS API 2.1

celerity12
New Contributor II

I need to find out all jobs which are currently running and not get other jobs

The below command fetches all the jobs

curl --location --request GET 'https://xxxxxx.gcp.databricks.com/api/2.1/jobs/list?active_only=true&expand_tasks=true&run_type=JOB_RUN&completed_only=false&verbose=TRUE& offset=0' \

--header 'Authorization: Bearer xxxxxxxx'

active_only=true doesn't seem to work. Any help is appreciated

7 REPLIES 7

AmanSehgal
Honored Contributor III

Work around -

You can add a filter for start_time and end_time to get jobs that were executed after a specific date (eg. T-1 day) to current time. And once you get the response, parse the JSON output to filter out jobs that don't have end time (implying that they're still running).

Hubert-Dudek
Esteemed Contributor III

Strange. I've tested your curl with Azure databricks, and everything worked perfectly. What distribution are you using?

celerity12
New Contributor II

Thanks @Hubert Dudekโ€‹  I am working with GCP Databricks. I do not see any start time and end time

image

Atanu
Esteemed Contributor
Esteemed Contributor

@Sumit Rohatgiโ€‹ can you try with limit and see if that works. are you still not getting it worked?Please let me know if still this is not listing. Thanks. also, just try running https://xxxxxx.gcp.databricks.com/api/2.1/jobs/list?active_only=true and see if the output.

Hi @Sumit Rohatgiโ€‹,

Just a friendly follow-up. Did Atanu's response help you? please let us know if you still need help.

User16764241763
Honored Contributor

Hi @Sumit Rohatgiโ€‹ 

It seems like active_only=true only applies to jobs/runs/list API and not to jobs/list.

Can you please try the jobs/runs/list API?

HHi @Sumit Rohatgiโ€‹,

Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.