Comment
Databricks Employee
Databricks Employee

A really good optimization guide - https://www.databricks.com/discover/pages/optimize-data-workloads-guide.

Apart from this, if you see from the dashboard that for certain jobs both CPU and memory is under utilized, you can just use lower compute and the job might run exactly the same even on the lower sized compute.