anshu_roy
Databricks Employee
Databricks Employee

Hello,
Databricks job cluster autoscaling makes decisions from Spark scheduler signals (pending/queued tasks vs available task slots and idleness windows), not raw CPU% alone. Enhanced autoscaling uses task queue size and task slot utilization. Autoscaling is available for jobs compute, and can be set as described here