How do I choose between a standard cluster and a shared cluster in Databricks?

[Suheb]

When should I use a single-user cluster, and when should I use a multi-user/shared cluster? What’s the difference and how do I pick the right one?

[K_Anudeep · Accepted solution]

Hello @Suheb ,

Use single-user (Dedicated) clusters when you need privileged machine access or features not supported on shared clusters. These clusters run under a single identity (user or group) and behave like a traditional Spark cluster with full access to the VM.

Use multi-user/shared (Standard) clusters when multiple users need to attach notebooks or SQL and run workloads concurrently with strong isolation and Unity Catalog governance. This mode is recommended for most workloads because it gives better cost efficiency, simplified management, and per-user access control while isolating user code.

Refer to offical docs: https://docs.databricks.com/aws/en/compute/configure

Please let me know if this answers your question

[Raman_Unifeye]

Multi-user cluster is always our go-to choice for all of our work. I never got a need to use dedicated one (yet).

[Coffee77]

Agree with previous responses BUT consider using a dedicated cluster if using it under an "automation security principal" identity for a subset of your jobs, if that makes sense. To summarize, you need to run a job, under a non-human identity and you need an active cluster, not possible to wait for creating a jobs compute cluster (it takes minutes despite using pools) and serverless compute is non an option for you (maybe for security or other reasons it's not enabled yet in your company).