Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Showing results for 
Search instead for 
Did you mean: 

Do shared clusters have multiple drivers?

New Contributor


I know that with single clusters, theres a single driver node and one driver per cluster. 

With shared clusters, multiple jobs can run concurrently. Does this still run on a single driver container or multiple driver containers run per application? 


Community Manager
Community Manager

Hi @SirCrayon

In the context of shared clusters, where multiple jobs can run concurrently, the behavior regarding driver containers differs from that of single clusters.

  1. Single Clusters:

    • In a single cluster, there is indeed a single driver node responsible for managing the Spark application.
    • This driver node runs in a single driver container.
    • All tasks and computations are coordinated by this single driver.
  2. Shared Clusters:

    • In shared clusters, multiple jobs can indeed run concurrently.
    • However, each job still has its own driver.
    • Therefore, multiple driver containers can run concurrently, one for each Spark application.
    • Each driver manages its specific job’s execution, task scheduling, and communication with the cluster.

In summary, shared clusters allow multiple jobs to coexist, but each job still operates with its dedicated driver container. This design ensures isolation and proper management of resources for each application. 🚀

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.