cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

If 2 users run the same notebook on the same cluster, will they share the same Spark session?

fijoy
Contributor

Databricks docs here:

https://docs.databricks.com/notebooks/notebook-isolation.html

state that "Every notebook attached to a cluster has a pre-defined variable named spark that represents a SparkSession."

What if 2 users run the same notebook on the same cluster? Will they share the session? Or is the Spark session user-specific also in addition to cluster- & notebook-specific?

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

Lakshay
Esteemed Contributor
Esteemed Contributor

The spark session is isolated at the notebook level and is not isolated at the user level. So, two users accessing the same notebook will be using the same spark session

View solution in original post

4 REPLIES 4

Anonymous
Not applicable

Hi @Fijoy Vadakkumpadan​ 

Great to meet you, and thanks for your question!

Let's see if your peers in the community have an answer to your question. Thanks.

Lakshay
Esteemed Contributor
Esteemed Contributor

The spark session is isolated at the notebook level and is not isolated at the user level. So, two users accessing the same notebook will be using the same spark session

Thanks @Lakshay Goel​ for the response. Follow up question: can the same notebook be attached to 2 different clusters at the same time by 2 different users?

Lakshay
Esteemed Contributor
Esteemed Contributor

No, we cannot attach the same notebook to 2 different clusters.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!