available virtual machines in databricks

bill-kotronis
New Contributor

2 weeks ago

My team is investigating the migration of older and now non-adjustable azure skus to newer versions such as v5. I see in azure Usage+Quotas the Standard Dldsv5 and Dldsv6 families with available quota. However, when creating compute in Databricks these families are not in the available options. I want the 'l' feature for the reduced memory to vcpu ratio to cleanly migrate from older Fsv2 families.

0 Kudos

GabFernandes
Contributor

2 weeks ago

Hi @bill-kotronis ,

The behavior you're seeing is expected — not all Azure VM families are exposed in Databricks, even if you have quota for them.

Databricks maintains a curated, validated subset of Azure instance types. Having available quota in Azure Portal (Usage + Quotas) is a prerequisite but not sufficient — the SKU also needs to be on Databricks' supported list for your region.

Why Dldsv5/Dldsv6 don't appear: These "l" (low memory-to-vCPU) families haven't been onboarded by Databricks yet. You can confirm programmatically what's available in your workspace via the REST API:

GET /api/2.0/clusters/list-node-types

This returns the full list of instance types your workspace can use.

Alternatives for a low memory-to-vCPU ratio (Fsv2 replacement):

  • Fdsv2 — already supported in Databricks, compute-optimized (2 GiB/vCPU), and is the direct successor to Fsv2. This should be the cleanest migration path for your use case.
  • Ddsv5 (without the "l") — general purpose (4 GiB/vCPU), higher memory but widely available.

To request Dldsv5/v6 support: Open a feature request via the Databricks Ideas Portal or file a support ticket. New VM families are added periodically based on demand.

0 Kudos