we have noticed that more than 1000 jobs are in jobs list in shard . Due to which we are getting 'error_code': 'QUOTA_EXCEEDED', when submitting new jobs using job API

User16753724663 — Wed, 16 Jun 2021 17:25:19 GMT

Re: we have noticed that more than 1000 jobs are in jobs list in shard . Due to which we are getting 'error_code': 'QUOTA_EXCEEDED', when submitting new jobs using job API

User16753724663 — Wed, 16 Jun 2021 17:26:33 GMT

We can use the below api to list out the jobs and then use the delete job api:

https://docs.databricks.com/dev-tools/api/latest/jobs.html#list

List

Endpoint HTTP Method

2.0/jobs/list GET

Once we list out the jobs, then we can use below API to delete them:

https://docs.databricks.com/dev-tools/api/latest/jobs.html#delete

Delete

Endpoint HTTP Method

2.0/jobs/delete POST

Here is the Json that, we need to use:

{

"job_id": 1

}

Here is the sample python code for listing the job:

import requests
 
DOMAIN = '<your-domain>.cloud.databricks.com'
 
TOKEN = '<api-token>'
 
 
 
response = requests.get(
 
'https://%s/api/2.0/jobs/list' % (DOMAIN),
 
headers={'Authorization': 'Bearer %s' % TOKEN},
 
json={
 
}
 
)
 
 
 
if response.status_code == 200:
 
print(response.json())
 
else:
 
print("Error launching cluster: %s: %s" % (response.json()["error_code"], response.json()["message"]))

Here is the KB article as well on the same:

https://kb.databricks.com/jobs/howto-jobsdeleterestapi.html

topic we have noticed that more than 1000 jobs are in jobs list in shard . Due to which we are getting 'error_code': 'QUOTA_EXCEEDED', when submitting new jobs using job API in Machine Learning

we have noticed that more than 1000 jobs are in jobs list in shard . Due to which we are getting 'error_code': 'QUOTA_EXCEEDED', when submitting new jobs using job API

Re: we have noticed that more than 1000 jobs are in jobs list in shard . Due to which we are getting 'error_code': 'QUOTA_EXCEEDED', when submitting new jobs using job API