โ06-16-2022 08:16 AM
As of now, if I try to list the jobs via "list job" API then there is a limit of 25 jobs only.
Is there a way to list all the available/visible jobs to a user?
โ06-23-2022 09:40 AM
@Kaniz Fatmaโ @Arvind Ravishโ :
Yes the user can generate the API token but the limit is on the "get jobs" API level in version 2.1.
One workaround that I found is to create a while loop function and pass the offset param variable to "list job" API with value as 25.
Then reiterate and check the json response at every iteration. If the response have a key value pair as {"has_more: False"} then break the loop. Also, at every iteration, keep on appending the job_name or job_id or any other relevant info. from json response in a list.
Finally you will be having a list of all jobs that is visible to the current user.
Alternatively you can use "list job" API version 2.0. This version doesn't have the limit of 25 but it won't return the multi-tasks inside the job and has a hard limit of max 2000 jobs.
โ06-17-2022 04:22 AM
Hello @Saurabh Vermaโ
Can the user generate the API token in the workspace and try to use the API?
โ06-23-2022 07:27 AM
Hi @Saurabh Vermaโ, We havenโt heard from you on the last response from @Arvind Ravishโ, and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please share it with the community as it can be helpful to others.
โ06-23-2022 09:40 AM
@Kaniz Fatmaโ @Arvind Ravishโ :
Yes the user can generate the API token but the limit is on the "get jobs" API level in version 2.1.
One workaround that I found is to create a while loop function and pass the offset param variable to "list job" API with value as 25.
Then reiterate and check the json response at every iteration. If the response have a key value pair as {"has_more: False"} then break the loop. Also, at every iteration, keep on appending the job_name or job_id or any other relevant info. from json response in a list.
Finally you will be having a list of all jobs that is visible to the current user.
Alternatively you can use "list job" API version 2.0. This version doesn't have the limit of 25 but it won't return the multi-tasks inside the job and has a hard limit of max 2000 jobs.
โ06-23-2022 09:45 AM
Awesome @Saurabh Vermaโ!
Thank you for sharing the great workaround and choosing the best answer.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group