When should I set the cluster mode to High Concurrency vs Standard?

[User16826992666]

How do I know which mode I should be using when creating a cluster?

[Ryan_Chynoweth · Accepted solution]

Generally speaking, if you are using Scala then you must use a Standard cluster. If you are using another language then typically the high concurrency is a better option as it does a better job sharing compute with multiple users. If you are looking to create a single user cluster and only one person uses it at a time then go with Standard anyways.

Note that standard is best for single users, but it is not restricted meaning multiple users can still use a standard cluster.. not recommended.

[sajith_appukutt]

High Concurrency clusters are ideal for groups of users who need to share resources or run ad-hoc jobs - for example data scientists sharing a cluster. They come with Query Watchdog, a process which keeps disruptive queries in check by automatically pre-empting rogue queries.

Standard clusters are ideal for developing / testing / debugging batch ETL / streaming workloads