cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

we have noticed that more than 1000 jobs are in jobs list in shard . Due to which we are getting 'error_code': 'QUOTA_EXCEEDED', when submitting new jobs using job API

User16753724663
Valued Contributor
 
1 REPLY 1

User16753724663
Valued Contributor

We can use the below api to list out the jobs and then use the delete job api:

https://docs.databricks.com/dev-tools/api/latest/jobs.html#list

List

Endpoint HTTP Method

2.0/jobs/list GET

Once we list out the jobs, then we can use below API to delete them:

https://docs.databricks.com/dev-tools/api/latest/jobs.html#delete

Delete

Endpoint HTTP Method

2.0/jobs/delete POST

Here is the Json that, we need to use:

{

"job_id": 1

}

Here is the sample python code for listing the job:

import requests
 
DOMAIN = '<your-domain>.cloud.databricks.com'
 
TOKEN = '<api-token>'
 
 
 
response = requests.get(
 
'https://%s/api/2.0/jobs/list' % (DOMAIN),
 
headers={'Authorization': 'Bearer %s' % TOKEN},
 
json={
 
}
 
)
 
 
 
if response.status_code == 200:
 
print(response.json())
 
else:
 
print("Error launching cluster: %s: %s" % (response.json()["error_code"], response.json()["message"]))

Here is the KB article as well on the same:

https://kb.databricks.com/jobs/howto-jobsdeleterestapi.html

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!