cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Shared cluster

shiva1212
New Contributor II

Hello Community,

At my company, we use Databricks extensively for workflow development. We operate on a shared cluster that multiple users access concurrently, and we utilize a shared workspace, where one folder is accessible to a designated group. Each user manages their data through their individual secret scope.

However, I have observed a significant issue: when User A executes a code cell, User B is able to execute the following cell, effectively continuing from where User A left off. This situation compromises the isolation between user sessions, posing a potential security risk.

Can anyone suggest a way to ensure better isolation between user sessions in this shared environment?

 

1 ACCEPTED SOLUTION

Accepted Solutions

Aviral-Bhardwaj
Esteemed Contributor III

No Shiva, it don't work like that,one notebook one session simple, collaborative notebook means one development notebook with multiple user ,but multiple session wont work like that .

AviralBhardwaj

View solution in original post

3 REPLIES 3

Aviral-Bhardwaj
Esteemed Contributor III

it should not happen , if the user is using different notebooks , if they are using same notebook, it is expected

AviralBhardwaj

shiva1212
New Contributor II

Thank you for your response, Aviral.

I agree that if users are using different notebooks, there shouldn’t be any security threat. However, the concern arises even when two users are using the same notebook; it should ideally be different sessions for each user.

Each user has their own OpenAI token. For instance, if User A runs a cell to extract their token, and then User B runs the next cell, User B ends up using User A’s token. This is undesirable behavior and poses a significant security risk.

From my understanding, a shared compute cluster should keep sessions separate for User A and User B. This means that while they share the code, they shouldn’t be sharing the variable values. Unfortunately, this separation is not happening in our environment.

Has anyone else experienced this, or does anyone have suggestions on how to enforce better session isolation?

Aviral-Bhardwaj
Esteemed Contributor III

No Shiva, it don't work like that,one notebook one session simple, collaborative notebook means one development notebook with multiple user ,but multiple session wont work like that .

AviralBhardwaj

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group