cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

429 XHR requests against jobs endpoint (RESOURCE_EXHAUSTED)

Malthe
Valued Contributor II

The Databricks UI sends thousands of repeated queries on the form:

  • /ajax-api/2.0/jobs/get?include_acls=true&job_id=<redacted>

Getting a 429 Too Many Requests response. It seems to rotate on a list of job ids.

It just keeps trying ...

4 REPLIES 4

Ashwin_DSA
Databricks Employee
Databricks Employee

Hi @Malthe,

Databricks enforces perโ€‘workspace rate limits on Jobs endpoints (including what the UI internally calls via /ajax-api/2.0/jobs/get, which wraps GET /api/2.0/jobs/get). For example, the documented limit for jobs/get is 20 requests/second per workspace. Do you know if the browser (or a script in the browser) is crossing the limits?

 

jobgetslimit.png

 

Could this be a reason for the issue, or have I misunderstood your question? 

If this answer resolves your question, could you mark it as โ€œAccept as Solutionโ€? That helps other users quickly find the correct fix.

Regards,
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***

Malthe
Valued Contributor II

It's an error in the Databricks UI. If it hits a 429, it just keeps going, which means it'll keep going over the limit ... that's my best guest.

To fix this, first make sure your UI doesn't repeatedly ask for a resource that previously returned a 429. Back off for some time.

Ashwin_DSA
Databricks Employee
Databricks Employee

Hi @Malthe,

Got it. I thought you were running some script or automation that was hitting the Jobs API limit. If this is purely due to normal UI usage, itโ€™s worth treating it as a potential UI issue.

Could you try opening a clean browser session (no custom JS in the console, no extensions that inject requests, ideally an incognito/private window) and do a minimal, repeatable set of clicks in the Jobs UI that leads to the 429s.

If you still see the UI repeatedly calling /ajax-api/2.0/jobs/get and getting 429 responses in that clean setup, thatโ€™s good evidence this is a Databricks UI bug. At that point Iโ€™d open a Databricks support ticket, attach a short HAR/network capture. That should give the Databricks team enough to investigate and, if needed, add proper backoff in the UI.

If this answer resolves your question, could you mark it as โ€œAccept as Solutionโ€? That helps other users quickly find the correct fix.

Regards,
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***

Malthe
Valued Contributor II

Just notify your engineers, they'll know how to handle this and won't need more information. It's a no-brainer to fix this.