cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Different GCP Service Account for cluster (compute) creation?

LeoRickli
New Contributor II

I have a Databricks workspace that is attached to a GCP Service Account from a project named "random-production-data". I want to create a cluster (compute) on Databricks that uses a different Service Account from another project for isolation purposes. Whenever I try to create this cluster, it returns me as invalid. So here is the question:

Using Databricks on GCP, is it possible to create a cluster using a Service Account from another project that was not attached to the workspace when it was created?

1 REPLY 1

jennie258fitz
New Contributor II

@LeoRickli wrote:

I have a Databricks workspace that is attached to a GCP Service Account from a project named "random-production-data". I want to create a cluster (compute) on Databricks that uses a different Service Account from another project for isolation purposes. Whenever I try to create this cluster, it returns me as invalid. So here is the question: Receiptify

Using Databricks on GCP, is it possible to create a cluster using a Service Account from another project that was not attached to the workspace when it was created?


In Databricks on GCP, clusters are typically created using the service account that is attached to the workspace. This service account is configured at the workspace level and is generally not interchangeable with service accounts from other GCP projects.

If you want to create a cluster using a different service account from another project, you'll need to ensure that the desired service account has the appropriate permissions and roles associated with the Databricks workspace. However, as of my last update, you cannot directly use a service account from a different project for a Databricks cluster unless that service account is explicitly granted access to the Databricks workspace.

To achieve your goal of isolating environments using different service accounts, you might consider the following approaches:

Create a New Workspace: Set up a new Databricks workspace using the desired service account. This way, you can have complete control over the service account used for clusters.

Service Account Permissions: If you want to stay within the same workspace, ensure that the service account from the other project has the necessary permissions to access the resources required by your Databricks cluster. You may need to adjust IAM roles and permissions.

Cross-Project Access: If using resources across projects, ensure that any necessary permissions are configured for the service accounts involved.

Unfortunately, you can't just switch the service account for a cluster once the workspace is created. You might want to check the latest Databricks and GCP documentation for any updates on this functionality.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group