https://community.databricks.com/t5/data-engineering/when-should-i-set-the-cluster-mode-to-high-concurrency-vs/m-p/24090#M16713 <P>Generally speaking, if you are using Scala then you must use a Standard cluster. If you are using another language then typically the high concurrency is a better option as it does a better job sharing compute with multiple users. If you are looking to create a single user cluster and only one person uses it at a time then go with Standard anyways.</P><P></P><P>Note that standard is best for single users, but it is not restricted meaning multiple users can still use a standard cluster.. not recommended. </P> Fri, 18 Jun 2021 21:16:06 GMT Ryan_Chynoweth 2021-06-18T21:16:06Z https://community.databricks.com/t5/data-engineering/when-should-i-set-the-cluster-mode-to-high-concurrency-vs/m-p/24089#M16712 <P>How do I know which mode I should be using when creating a cluster?</P> Wed, 16 Jun 2021 04:06:59 GMT https://community.databricks.com/t5/data-engineering/when-should-i-set-the-cluster-mode-to-high-concurrency-vs/m-p/24089#M16712 User16826992666 2021-06-16T04:06:59Z https://community.databricks.com/t5/data-engineering/when-should-i-set-the-cluster-mode-to-high-concurrency-vs/m-p/24090#M16713 <P>Generally speaking, if you are using Scala then you must use a Standard cluster. If you are using another language then typically the high concurrency is a better option as it does a better job sharing compute with multiple users. If you are looking to create a single user cluster and only one person uses it at a time then go with Standard anyways.</P><P></P><P>Note that standard is best for single users, but it is not restricted meaning multiple users can still use a standard cluster.. not recommended. </P> Fri, 18 Jun 2021 21:16:06 GMT https://community.databricks.com/t5/data-engineering/when-should-i-set-the-cluster-mode-to-high-concurrency-vs/m-p/24090#M16713 Ryan_Chynoweth 2021-06-18T21:16:06Z https://community.databricks.com/t5/data-engineering/when-should-i-set-the-cluster-mode-to-high-concurrency-vs/m-p/24091#M16714 <P>High Concurrency clusters are ideal for groups of users who need to share resources or run ad-hoc jobs - for example data scientists sharing a cluster. They come with Query Watchdog, a process which keeps disruptive queries in check by automatically pre-empting rogue queries.</P><P></P><P>Standard clusters are ideal for developing / testing / debugging batch ETL / streaming workloads </P> Fri, 18 Jun 2021 22:17:49 GMT https://community.databricks.com/t5/data-engineering/when-should-i-set-the-cluster-mode-to-high-concurrency-vs/m-p/24091#M16714 sajith_appukutt 2021-06-18T22:17:49Z