RajeshReddy
New Contributor II

you might want to take a look at job clusters for cost optimization 

https://learn.microsoft.com/en-us/azure/databricks/clusters/cluster-config-best-practices