Databricks workspace creation using Terraform and storage configuration IAM Role Arn

sampo — Sat, 02 Mar 2024 18:56:23 GMT

Hi,

When creating a new AWS Databricks workspace in account console manually, it appears that IAM Role ARN is mandatory parameter when configuring a Storage Configuration

However in the Terraform databricks_mws_storage_configurations resource there is apparently no way to set the IAM Role arn.

This then leads to failure when creating workspace using databricks_mws_workspaces resource

Has anyone else seen this problem?

Re: Databricks workspace creation using Terraform and storage configuration IAM Role Arn

sampo — Tue, 05 Mar 2024 07:44:15 GMT

Investigated this further... Terraform code to create the workspace

resource "databricks_mws_workspaces" "databricks_ws" { provider = databricks.mws account_id = var.databricks_account_id aws_region = var.region workspace_name = "databricks-workspace-${local.env}" credentials_id = databricks_mws_credentials.databricks_credential_conf.id storage_configuration_id = databricks_mws_storage_configurations.databricks_storage_conf.id network_id = databricks_mws_networks.databricks_network_conf.id token { comment = "Terraform" } }

From the Terraform log it looks like account_id string is added to credentials_id, network_id, and storage_configuration_id values

POST /api/2.0/accounts/<account_id>/workspaces { "account_id": "<account_id>", "aws_region": "eu-west-1", "credentials_id": "<account_id>/bbf46eba-67e2-4538-8d19-426e75136ead", "is_no_public_ip_enabled": true, "network_id": "<account_id>/e9d9fb8a-3b9d-4ea3-8823-72c62e7c8f6d", "storage_configuration_id": "<account_id>/06bfb24a-4426-4343-b474-6fe593e20746", "workspace_name": "databricks-workspace-test" } < HTTP/2.0 400 Bad Request

Looking at the Rest API sample for new workspace creation

There is no account_id in the credentials_id, network_id, and storage_configuration_id values

I replaced account_id string with empty string in the Terraform code and now workspace creation works without any errors

resource "databricks_mws_workspaces" "databricks_ws" { provider = databricks.mws account_id = var.databricks_account_id aws_region = var.region workspace_name = "databricks-workspace-${local.env}" credentials_id = replace(databricks_mws_credentials.databricks_credential_conf.id, "${var.databricks_account_id}/", "") storage_configuration_id = replace(databricks_mws_storage_configurations.databricks_storage_conf.id, "${var.databricks_account_id}/", "") network_id = replace(databricks_mws_networks.databricks_network_conf.id, "${var.databricks_account_id}/", "") token { comment = "Terraform" } }

topic Databricks workspace creation using Terraform and storage configuration IAM Role Arn in Administration & Architecture

Databricks workspace creation using Terraform and storage configuration IAM Role Arn

Re: Databricks workspace creation using Terraform and storage configuration IAM Role Arn