databricks cluster creation error

[kunaldeb]

Hi Databricks Community,

My Databricks workspace created on Azure pay-as-go subscription.

I am facing two sided challenges.

First>>   I am not able to create Delta-Live-table pipeline or any other all-purpose multi-node cluster as it is throwing below error (although I am allocating very limited amount of resource).

Clusters are failing to launch. Cluster launch will be retried.

Details for the latest failure: Error: Error code: QuotaExceeded, error message: Operation could not be completed as it results in exceeding approved LowPriorityCores quota. Additional details - Deployment Model: Resource Manager, Location: CentralIndia, Current Limit: 3, Current Usage: 0, Additional Required: 8, (Minimum) New Limit Required: 8. Submit a request for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/{xxx... by specifying parameters listed in the ‘Details’ section for deployment to succeed.

Second>> To resolve that when I am trying to increase Quota from Azure (for “Total Regional Low-priority vCPUs”) it is not happening and it is showing “We were unable to increase your quota. Submit a support ticket so that a support engineer can assist you in increasing your quota for your Total Regional Low-priority vCPUs in Central India for Azure subscription 1.”

Point is can anyone pls suggest easy and quick way for this problem statement. I guess creating cluster in Databricks not that troublesome.

Please suggest something to move on quickly. Please let me know if you need any more inputs.

[jose_gonzalez]

Are you able to create a support ticket with the Azure team? if you already try to increase the quota, but still getting the error, then the best solution will be to report this issue with the Azure support team, so they can take a loot at this issues for you

[karthik_p]

@kunal debnath​ this issue happens some times due to kind of quota limit support that has on aws pay as you go. you need to change your plan from pay as you go to the subscription and then increase quota and you should be able to use you DLT clusters

[kunaldeb]

HI All,

Thanks for your reply.

Just to update you, I am now able to to create DLT pipeline as well as all purpose multi-node cluster with minimum resources. This is due to Quota limit and I was able to increase it. But observation is, if I try to use more resources then problem will surely occur and looks like pay-as-you-go account has some limitations as well for quota increase.