Cluster Modes

[Anonymous]

Given that there are three different kinda of cluster modes, when is it appropriate to use each one?

[User16826994223]

Standard clusters

A Standard cluster is recommended for a single user. Standard clusters can run workloads developed in any language: Python, SQL, R, and Scala.

High Concurrency clusters

A High Concurrency cluster is a managed cloud resource. The key benefits of High Concurrency clusters are that they provide fine-grained sharing for maximum resource utilization and minimum query latencies.

A Single Node cluster has no workers and runs Spark jobs on the driver node.

In contrast, a Standard cluster requires at least one Spark worker node in addition to the driver node to execute Spark jobs.

Databricks Documents- https://docs.databricks.com/clusters/configure.html

[peter_mcnally]

@Kunal Gaurav​ If I want to use R and there are multiple users, I see two options, creating a separate 'single user' cluster for each R user or using the 'no isolation shared' cluster option for multiple users. I would rather use a single cluster, but i am not sure of the implications.

Advice?