DLT pipelines always spin up job compute, and Azure is strict about SKU availability per region & per subscription.

Most common causes

Quota for that VM family is set to 2 vCPUs

Databricks shows:

“Estimated available: 2”

“QuotaExceeded”

The SKU exists in Azure CLI but Azure has no capacity for it in UK South

This is very common for older families like DS_v2 and F-series.

The Pipeline UI hides Advanced Options

This normally happens when Databricks can’t find any valid SKUs for job compute under your subscription constraints.

What to check

In Azure Portal → Subscription → Usage + quotas

Filter by:

Region: UK South

VM family: Dsv2, F-series, Dv3, Dv5, etc.

You will typically see vCPU limits like “2/2 used”.

Request a quota increase for at least:

Standard Dv3 Family vCPUs

Standard Dv5 Family vCPUs

(These have much better regional availability.)

Alternatively try a workspace in UK South 2 or North Europe, where clusters often provision successfully.