topic Re: For sql classic, does number of driver also increase when auto scaled up? in Warehousing & Analytics

For sql classic, does number of driver also increase when auto scaled up?

alexw4561 — Wed, 24 Sep 2025 00:23:33 GMT

I know for spark clusters, the number of driver will remain the same and only the number of workers increases when auto scaling. But for sql classic warehouse I'm a bit confused, since it's scaling number of cluster not driver.

Re: For sql classic, does number of driver also increase when auto scaled up?

szymon_dybczak — Wed, 24 Sep 2025 06:21:54 GMT

Hi @alexw4561 ,

I would assume that. They don’t scale by resizing a single cluster. Instead, they scale by adding or removing whole clusters. Each of those clusters has its own driver and workers. So if you configure min = 1 cluster, max = 10 clusters, Databricks can spin up to 10 separate clusters. The SQL Warehouse load balancer routes queries to one of these clusters. This is horizontal scaling at the cluster level.

Re: For sql classic, does number of driver also increase when auto scaled up?

jebouwman — Wed, 24 Sep 2025 11:13:27 GMT

Hi @alexw4561 ,

@szymon_dybczak is indeed right as this page explains: https://docs.databricks.com/aws/en/compute/sql-warehouse/warehouse-behavior#queuing-and-autoscaling-logic

We had the same question in the past and found out it indeed just adds an extra cluster.

Re: For sql classic, does number of driver also increase when auto scaled up?

nayan_wylde — Wed, 24 Sep 2025 16:30:12 GMT

SQL Classic Warehouses:

Scaling is done by adding or removing entire clusters, not just workers.
Each cluster has its own driver and set of workers.
So when autoscaling happens, more clusters (each with a driver + workers) are added to handle concurrent query load.

Suppose if you are creating a M size cluster and you set the min scale at 1 and max at 5. The calculation will be:

At low load, you run 1 cluster → 1 driver + 8 workers.
At high load, Databricks may scale up to 5 clusters → 5 drivers + 40 workers.