state that "Every notebook attached to a cluster has a pre-defined variable named spark that represents a SparkSession."
What if 2 users run the same notebook on the same cluster? Will they share the session? Or is the Spark session user-specific also in addition to cluster- & notebook-specific?
The spark session is isolated at the notebook level and is not isolated at the user level. So, two users accessing the same notebook will be using the same spark session
The spark session is isolated at the notebook level and is not isolated at the user level. So, two users accessing the same notebook will be using the same spark session
Thanks @Lakshay Goelโ for the response. Follow up question: can the same notebook be attached to 2 different clusters at the same time by 2 different users?
Welcome to Databricks Community: Lets learn, network and celebrate together
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.