cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks shared workspace

nskiran1
New Contributor II
We have a Self service portal through which users can launch databricks clusters of different configurations. 
 
This portal is set up to work in Dev, Sandbox and Prod environments. We have configured databricks workspaces only for Sandbox and Prod portals only. So, users can launch databricks clusters through Sandbox and Prod portals. No databricks workspace available for Dev Portal.
 
Self service portal invokes different databricks APIs like list/delete/register instance profiles, create/delete clusters etc using Python. We have set up Service Principals for all the databricks workspaces and invoke databricks APIs with respective service principals.
 
Recently, our management decided to route databricks cluster launch requests from Dev portal to Sandbox databricks workspace on urgent basis as we do not have databricks workspace for Dev portal
 
Is it possible to have databricks 'shared' workspace that can be tied to multiple AWS accounts? Can someone share documentation on IAM permissions policies on how to configure multiple AWS accounts for shared workspace please?
2 REPLIES 2

Alberto_Umana
Databricks Employee
Databricks Employee

Hello @nskiran1,

Yes, it is possible to have a 'shared' workspace that can be tied to multiple AWS accounts. This can be achieved by associating multiple VPCs (Virtual Private Clouds) across different AWS accounts with a single Databricks account

You should associate your VPCs and create a cross account IAM role

https://docs.databricks.com/en/admin/account-settings-e2/credentials.html#step-1-create-a-cross-acco...

https://docs.databricks.com/en/security/network/classic/customer-managed-vpc.html

https://docs.databricks.com/en/admin/account-settings-e2/credentials.html

nskiran
New Contributor III

@Alberto_Umana Thanks for sharing doc links

We have exact same set up to support shared databricks workspace. But still Im facing issue while adding instance profile

I am trying to add AWS Instance Profile created in source AWS Account (No databricks workspace) to a target AWS Account to which databricks workspace set up available. Is this possible?

I have added required IAM permissions for both instance profile as well as cross account role. What else am I missing here?

{"error_code":"DRY_RUN_FAILED","message":"Verification of the instance profile failed. AWS error: You are not authorized to perform this operation.","details":[{"@type":"type.googleapis.com/google.rpc.ErrorInfo","reason":"CM_API_ERROR_SOURCE_CALLER_ERROR","domain":""}]}

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group