Debayan
Databricks Employee
Databricks Employee

Hi, You can check https://docs.databricks.com/en/compute/cluster-config-best-practices.html for reference for cluster configurations best practises. About distributing workloads , could you please elaborate how is this expected? In a general scenario one run  is designed to use a whole cluster until it's healthy.