Cost Optimization

[christo_M]

How can I optimize the cost on our Databricks platform ? Despite some optimization actions I've taken so far it's still difficult to lower the cost. I tried different technics like Vacuum , or shutting down a cluster running after 30 mins but still don't any significant improvement.    

[RajeshReddy]

you might want to take a look at job clusters for cost optimization 

https://learn.microsoft.com/en-us/azure/databricks/clusters/cluster-config-best-practices

[JulioCastro]

Give me more info on what you are doing. Is it something you can run once a day? Run on schedule. Are you cleaning data? Or is it inference? Really does depend

[rodrigosanchesz]

Have you try to use Job Cluster?

You also can expurge no more use tables in data store and optimize table for querys can execute more fast.

[erigaud]

Make sure you're using a cluster that is the right size for your workload. You can greatly reduce the costs by using smaller clusters.