How do I know which worker type to choose when creating my cluster?

[User16826992666]

I am new to using Databricks and want to create a cluster, but there are many different worker types to choose from.

How do I know which worker type is the right type for my use case?

[sajith_appukutt · Accepted solution]

For delta workloads, where you could benefit from caching it is recommended to use storage optimized instances that come with NVMe SSDs.

For other workloads, it would be a good idea to check Ganglia metrics to see whether your workload is Cpu/Memory bound and choose a cpu/mem optimized instance appropriately.