topic Re: serverless workflow Compute became unresponsive. Compute is likely out of memory. in Data Engineering

serverless workflow Compute became unresponsive. Compute is likely out of memory.

lizou1 — Sat, 09 Aug 2025 23:34:00 GMT

I set up run 10 notebooks at same time in serverless workflow

I got this error:

serverless workflow Compute became unresponsive. Compute is likely out of memory.

Is there a quota in serverless compute I can set in zure databricks?

These notebooks are processing a few tables with 100 G in size each

Re: serverless workflow Compute became unresponsive. Compute is likely out of memory.

Advika — Mon, 11 Aug 2025 13:44:48 GMT

Hello @lizou1!

Yes, serverless compute has a quota, measured in Databricks Units (DBUs) per hour. For notebooks, jobs, or pipelines, there’s a built-in hourly spending upper limit.
To check your account’s serverless scale-up limits, you need to contact your Azure Databricks account team. You can also request a quota increase through the Azure Portal.

https://learn.microsoft.com/en-us/azure/databricks/admin/account-settings/serverless-quotas

Re: serverless workflow Compute became unresponsive. Compute is likely out of memory.

lizou1 — Mon, 15 Sep 2025 00:06:42 GMT

The issue is new, and azure cloud providers are also not quite sure the details, will get more info later

Re: serverless workflow Compute became unresponsive. Compute is likely out of memory.

lizou1 — Mon, 15 Sep 2025 18:25:59 GMT

Sorry just FYI

I am having this issue randomly every day, in environment 4 , I also have same issue in environment 2,3 as well

Run just waiting for hours

2031.048s info gc GC(22) Pause Young (Allocation Failure) 1190M - 273M(3172M) 19.415ms