Infrastructure question

[horatiug]

We've noticed that the GKE worker nodes which are automatically created when Databricks workspace is created inside GCP project are using the default compute engine SA which's not the best security approach, even Google doesn't recommend using default resources inside production environments. Is there any option when creating the workspace to avoid using the Google default service account ?

[Kaniz_Fatma]

Hi @horatiug, there is an option to avoid using the default service account when creating a Databricks workspace in a GCP project. You can create your workspaces in an existing customer-managed Virtual Private Cloud (VPC) that you make in your account. This allows you to exercise more control over your network configurations to comply with specific cloud security and governance standards that your organization may require. While creating a workspace, Databricks makes a service account and grants a role with the permissions needed to manage your workspace. If your workspace uses a customer-managed VPC, it does not need as many permissions. The part that Databricks creates omits permissions such as creating, updating, and deleting objects such as networks, routers, and subnets. 

Please note that you must specify the customer-managed VPC when you create the workspace through the account console. You cannot move an existing workspace with a Databricks-managed VPC to your own VPC.

Also, you cannot change which customer-managed VPC the workspace uses after workspace creation.

[horatiug]

Hi @Kaniz_Fatma 

we are already using custom VPC but the service account used is the the default compute engine. Expectation from our security team would be that when deploying the workspace a new SA is created in the project with required rights and not using the default compute engine SA.