For sql classic, does number of driver also increase when auto scaled up?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-23-2025 05:23 PM
I know for spark clusters, the number of driver will remain the same and only the number of workers increases when auto scaling. But for sql classic warehouse I'm a bit confused, since it's scaling number of cluster not driver.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-23-2025 11:21 PM
Hi @alexw4561 ,
I would assume that. They don’t scale by resizing a single cluster. Instead, they scale by adding or removing whole clusters. Each of those clusters has its own driver and workers. So if you configure min = 1 cluster, max = 10 clusters, Databricks can spin up to 10 separate clusters. The SQL Warehouse load balancer routes queries to one of these clusters. This is horizontal scaling at the cluster level.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-24-2025 04:13 AM
Hi @alexw4561 ,
@szymon_dybczak is indeed right as this page explains: https://docs.databricks.com/aws/en/compute/sql-warehouse/warehouse-behavior#queuing-and-autoscaling-...
We had the same question in the past and found out it indeed just adds an extra cluster.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-24-2025 09:30 AM
SQL Classic Warehouses:
- Scaling is done by adding or removing entire clusters, not just workers.
- Each cluster has its own driver and set of workers.
- So when autoscaling happens, more clusters (each with a driver + workers) are added to handle concurrent query load.
Suppose if you are creating a M size cluster and you set the min scale at 1 and max at 5. The calculation will be:
- At low load, you run 1 cluster → 1 driver + 8 workers.
- At high load, Databricks may scale up to 5 clusters → 5 drivers + 40 workers.