API Call to return more than 100 jobs

dpc
Contributor III

‎11-26-2025 07:42 AM

Hello

 

I have around 150 jobs and this is likely to increase.

I use this call to get all the jobs and write them into a list called json.

My logic here is to match a name to a job id and run the job using the job id.

 

response = requests.get(hostHTTPS, json = additional_parameters, headers=auth).json()
 
HostHTTPS is basically this:
hostHTTPS = ''.join([databricksHost, "/api/2.1/jobs/list?limit=100"])
 
That goes off and returns 100 jobs - all good
 
If I change limit to more than 100, I get (e.g. 200):
 
{'error_code': 'INVALID_PARAMETER_VALUE', 'message': 'Invalid limit 200 - it has to be no more than 100.', 'details': [{'@type': 'type.googleapis.com/google.rpc.RequestInfo'
 
How do I increase this as 100 is a pretty small number?
 
Thanks
 
0 Kudos

bianca_unifeye
Databricks MVP

‎11-26-2025 07:53 AM

Please refer to https://docs.databricks.com/api/workspace/jobs/list?

The official documentation for the Jobs API (GET /api/2.2/jobs/list) states that the limit parameter controls how many jobs are returned per call, and the API enforces a cap (100).

Because you have ~150 jobs and may have more later, you should loop over subsequent pages until you've retrieved all jobs. The API supports pagination: the response will include a next_page_token (or an equivalent mechanism) when there are more jobs to fetch.

https://docs.databricks.com/aws/en/reference/jobs-api-2-2-updates

View solution in original post

dpc
Contributor III

‎11-26-2025 08:33 AM

Thanks. I will try that

0 Kudos

dpc
Contributor III

‎12-03-2025 12:48 AM

Looping using next_page_token works well, thanks @bianca_unifeye