cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

databricks cluster creation error

kunaldeb
New Contributor III

Hi Databricks Community,

My Databricks workspace created on Azure pay-as-go subscription.

I am facing two sided challenges.

First>>   I am not able to create Delta-Live-table pipeline or any other all-purpose multi-node cluster as it is throwing below error (although I am allocating very limited amount of resource).

Clusters are failing to launch. Cluster launch will be retried.

Details for the latest failure: Error: Error code: QuotaExceeded, error message: Operation could not be completed as it results in exceeding approved LowPriorityCores quota. Additional details - Deployment Model: Resource Manager, Location: CentralIndia, Current Limit: 3, Current Usage: 0, Additional Required: 8, (Minimum) New Limit Required: 8. Submit a request for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/{xxx... by specifying parameters listed in the ‘Details’ section for deployment to succeed.

Second>> To resolve that when I am trying to increase Quota from Azure (for “Total Regional Low-priority vCPUs”) it is not happening and it is showing “We were unable to increase your quota. Submit a support ticket so that a support engineer can assist you in increasing your quota for your Total Regional Low-priority vCPUs in Central India for Azure subscription 1.

Point is can anyone pls suggest easy and quick way for this problem statement. I guess creating cluster in Databricks not that troublesome.

Please suggest something to move on quickly. Please let me know if you need any more inputs.

3 REPLIES 3

jose_gonzalez
Moderator
Moderator

Are you able to create a support ticket with the Azure team? if you already try to increase the quota, but still getting the error, then the best solution will be to report this issue with the Azure support team, so they can take a loot at this issues for you

karthik_p
Esteemed Contributor

@kunal debnath​ this issue happens some times due to kind of quota limit support that has on aws pay as you go. you need to change your plan from pay as you go to the subscription and then increase quota and you should be able to use you DLT clusters

kunaldeb
New Contributor III

HI All,

Thanks for your reply.

Just to update you, I am now able to to create DLT pipeline as well as all purpose multi-node cluster with minimum resources. This is due to Quota limit and I was able to increase it. But observation is, if I try to use more resources then problem will surely occur and looks like pay-as-you-go account has some limitations as well for quota increase.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group