cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Unable to create Compute

sanketpatel
New Contributor

Hello Community Support,

I am not able to create the compute resources with my Azure Databricks subscription. Can anyone help?

I was on a 14-day free premium trial plan. Now I am on Premium (+ Role-based access controls) plan.  My azure account is pay as you go subscription plan.

 
My Workspace ID: 7405616306325796
OpenSharing ID: azure:eastus:6bad6940-d3cf-4e41-bf37-a18c6d615f22
2 REPLIES 2

ShamenParis
Contributor III

Hi @sanketpatel ,

I think the problem is that your Azure subscription simply doesn't have enough vCPU core quota allocated for the StandardDCASv5Family in the East US region (According to the images). Even though you are on a Pay-As-You-Go plan, Azure enforces default core limits on specific VM families (which is currently set to 0 on your account) to prevent accidental overspending.

You can try requesting a quota increase directly through the Azure Portal. It's usually a very quick process.

Here are the steps to fix it:

  1. Sign in to the Azure portal and type Quotas in the top search bar.

  2. On the left menu or overview page, click on Compute.

  3. Filter the location to East US and search for StandardDCASv5Family (the family for your standard_DC4as cluster).

  4. Check the box next to it and click New Quota Request at the top.

  5. Select Enter a new limit. The error mentions you need a minimum of 4 cores, but it is usually a good idea to request 8 or 16 so you have a little buffer for future clusters.

  6. Click Submit.

Azure often approves these automatically within a few minutes. If it requires manual review, it might prompt you to open a quick support ticket from that same page.

Link: Increase VM-family vCPU quotas (Azure Documentation)

Louis_Frolio
Databricks Employee
Databricks Employee

Greetings @sanketpatel, I did some digging and here is what I found.

ShamenParis nailed it. This is an Azure subscription quota issue, not a problem with your workspace or Premium plan. From the screenshots, your standardDCASv5Family quota in East US is 0, and a Standard_DC4as_v5 node needs 4 vCPUs. Classic clusters run VMs inside your own Azure subscription, so Azure blocks the launch.

You've got a few ways out:

  1. Switch VM types. DCasv5 is Azure's confidential computing family. Unless you need confidential VMs, pick a general-purpose size like Standard_D4ds_v5 or Standard_DS3_v2. Pay-as-you-go subscriptions usually have some default quota for the common D-series families, so this may fix it with no request at all.

  2. Request more quota. In the Azure portal, open Quotas (or Subscriptions > Usage + quotas), choose Compute, and filter for East US. Check both the family quota and Total Regional vCPUs, since both have to cover you. Size the request for the whole cluster: the driver and every worker count, plus autoscaling headroom. Four vCPUs only covers a single-node cluster. If you use spot instances, those draw from a separate low-priority quota. If the quota is non-adjustable or the request gets rejected, open an Azure support request for Service and subscription limits (quotas).

  3. Use serverless. Serverless notebooks and jobs run on Databricks-managed compute, so your subscription quota doesn't apply. It needs Unity Catalog (typically on by default for newer workspaces). In a notebook, open the Connect drop-down and pick Serverless. If it's grayed out, an account admin can enable it in the account console.

If the error sticks around, check the compute event log and post the exact Azure error text, VM size, and region here. One small thing: I'd edit your original post to remove the workspace and sharing IDs. They aren't credentials, but there's no need for them on a public forum.

Public references:

Hope this helps.

Regards, Louis.