cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

How do I know which worker type to choose when creating my cluster?

User16826992666
Valued Contributor

I am new to using Databricks and want to create a cluster, but there are many different worker types to choose from.

How do I know which worker type is the right type for my use case?

1 ACCEPTED SOLUTION

Accepted Solutions

sajith_appukutt
Honored Contributor II

For delta workloads, where you could benefit from caching it is recommended to use storage optimized instances that come with NVMe SSDs.

For other workloads, it would be a good idea to check Ganglia metrics to see whether your workload is Cpu/Memory bound and choose a cpu/mem optimized instance appropriately.

View solution in original post

1 REPLY 1

sajith_appukutt
Honored Contributor II

For delta workloads, where you could benefit from caching it is recommended to use storage optimized instances that come with NVMe SSDs.

For other workloads, it would be a good idea to check Ganglia metrics to see whether your workload is Cpu/Memory bound and choose a cpu/mem optimized instance appropriately.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.