Can't use job cluster for scheduled jobs ADD_NODES_FAILED : Failed to add 9 containers to the cluster. Will attempt retry: false. Reason: Azure Quota Exceeded Exception

B_J_Innov
New Contributor III

Hi everyone,

I've been using my all purpose cluster for scheduled jobs and I've been told that it's a suboptimal thing to do and that using a job cluster for the scheduled jobs cuts costs by half.

Unfortunately, when I tried to switch clusters on my existing job, an unexpected error appeared : Unexpected failure while waiting for the cluster to be ready : is in unexpected state Terminated: AZURE_QUOTA_EXCEEDED_EXCEPTION(CLIENT_ERROR):

The operation couldn't be completed. Databricks quota estimation shows that you don't have sufficient CPU core for instance type Standard_DS3_v2 to launch the cluster. Consider increasing quota for the instance type, decreasing quota usage, trying other instance types and trying again later,databricks_error_message:The operation couldn't be completed. Databricks quota estimation shows that you don't have sufficient CPU core for instance type Standard_DS3_v2 to launch the cluster. Consider increasing quota for the instance type, decreasing quota usage, trying other instance types and trying again later.

I don't know why this happens, I'm supposed to have a job cluster and I know that nobody uses it.

It seems like nobody had this error in this forum that's why I post this issue, thanks in advance for your help !