cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

How to get the list of all jobs available for a particular user?

Maverick1
Valued Contributor II

As of now, if I try to list the jobs via "list job" API then there is a limit of 25 jobs only.

Is there a way to list all the available/visible jobs to a user?

1 ACCEPTED SOLUTION

Accepted Solutions

Maverick1
Valued Contributor II

@Kaniz Fatma​  @Arvind Ravish​ :

Yes the user can generate the API token but the limit is on the "get jobs" API level in version 2.1.

One workaround that I found is to create a while loop function and pass the offset param variable to "list job" API with value as 25.

Then reiterate and check the json response at every iteration. If the response have a key value pair as {"has_more: False"} then break the loop. Also, at every iteration, keep on appending the job_name or job_id or any other relevant info. from json response in a list.

Finally you will be having a list of all jobs that is visible to the current user.

Alternatively you can use "list job" API version 2.0. This version doesn't have the limit of 25 but it won't return the multi-tasks inside the job and has a hard limit of max 2000 jobs.

View solution in original post

4 REPLIES 4

User16764241763
Honored Contributor

Hello @Saurabh Verma​ 

Can the user generate the API token in the workspace and try to use the API?

Kaniz_Fatma
Community Manager
Community Manager

Hi @Saurabh Verma​, We haven’t heard from you on the last response from @Arvind Ravish​, and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please share it with the community as it can be helpful to others.

Maverick1
Valued Contributor II

@Kaniz Fatma​  @Arvind Ravish​ :

Yes the user can generate the API token but the limit is on the "get jobs" API level in version 2.1.

One workaround that I found is to create a while loop function and pass the offset param variable to "list job" API with value as 25.

Then reiterate and check the json response at every iteration. If the response have a key value pair as {"has_more: False"} then break the loop. Also, at every iteration, keep on appending the job_name or job_id or any other relevant info. from json response in a list.

Finally you will be having a list of all jobs that is visible to the current user.

Alternatively you can use "list job" API version 2.0. This version doesn't have the limit of 25 but it won't return the multi-tasks inside the job and has a hard limit of max 2000 jobs.

Awesome @Saurabh Verma​!

Thank you for sharing the great workaround and choosing the best answer.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!