Databricks Community

Is the enhanced autoscalar https://docs.databricks.com/aws/en/ldp/auto-scaling#what-is-enhanced-autoscaling available for job clusters as well?I do not see the option to set the autoscaling mode for job clusters

Can you share the metrics databricks uses during job cluster auto scaling?Is Databricks  looking at queued tasks, slot utilization etc or just looking at CPU utilizations?The autoscaling docuemnt https://docs.databricks.com/aws/en/compute/configure?u...

noticed that the limit parameter on the list job runs api: https://docs.databricks.com/api/workspace/jobs/listrunsdoesnt work. no matter what limit i set, i always get all the runs which can be too many if i want all runs.If i set offset as > 1, then...

This simple query takes 50seconds for me on a X-Small warehouse.select * from SYSTEM.access.workspaces_latest where workspace_id = '442224551661121'Can the team comment on why querying on system tables takes so long? I also dont see any primary keys ...

We have a few serverless dlt pipelines that we want to optimize for cost as we are ok with an increased latency. Where can I change the pipeline to run on cost optimized mode. I dont see this option in UI or API