topic Re: Cannot use Terraform to create Databricks Storage Credential in Administration & Architecture

Cannot use Terraform to create Databricks Storage Credential

AlbertWang — Mon, 23 Sep 2024 02:27:37 GMT

Hi all,

When I use Terraform in an Azure DevOps pipeline to create Databricks Storage Credential, I got the following error. Has anybody met the same error before? Or is there any idea how to debug it?

Error: cannot create storage credential: failed during request visitor: default auth: azure-cli: cannot get account info: exit status 1.

My implementation.

(1) Below is my `main.tf`. It works well on my local using my own Azure account.

terraform { required_providers { azurerm = { source = "hashicorp/azurerm" version = "4.2.0" } databricks = { source = "databricks/databricks" version = "1.52.0" } } } provider "azurerm" { features {} subscription_id = "${var.AZURE_SUBSCRIPTION_ID}" } provider "databricks" { host = var.DATABRICKS_HOST retry_timeout_seconds = 600 } data "azurerm_databricks_access_connector" "unity_catalog_access_connector" { name = "unity-catalog-access-connector" resource_group_name = "rg-dbr-managed-${var.ENVIRONMENT}" } resource "databricks_storage_credential" "dbr_strg_cred" { name = "dbr_strg_cred_${var.ENVIRONMENT}" azure_managed_identity { access_connector_id = data.azurerm_databricks_access_connector.unity_catalog_access_connector.id } }

(2) However, I got an error when I apply the Terraform file in an Azure DevOps pipeline. Below is my `azure-pipelines.yaml`.

trigger: none pool: vmImage: ubuntu-latest variables: - group: vg-dbr-dev stages: - stage: setup jobs: - job: setup displayName: "Set up Databricks workspace using Terraform" steps: - script: | echo "##vso[task.setvariable variable=TF_VAR_DATABRICKS_HOST]$DATABRICKS_HOST" echo "##vso[task.setvariable variable=TF_VAR_AZURE_SUBSCRIPTION_ID]$AZURE_SUBSCRIPTION_ID" echo "##vso[task.setvariable variable=TF_VAR_ENVIRONMENT]$ENV" displayName: 'Set up environment variables' - script: env | sort displayName: 'Environment / Context' - task: UsePythonVersion@0 displayName: 'Use Python 3.12' inputs: versionSpec: 3.12 - script: | python -m pip install wheel displayName: 'Install dependencies' - task: TerraformInstaller@1 displayName: Install Terraform 1.9.2 inputs: terraformVersion: 1.9.2 - task: TerraformTaskV4@4 displayName: Initialize Terraform inputs: provider: 'azurerm' command: 'init' backendServiceArm: '$(DEVOPS_SERVEICE_CONNECTION)' backendAzureRmResourceGroupName: '$(TERRAFORM_STATE_STORAGE_RESOURCE_GROUP_NAME)' backendAzureRmStorageAccountName: '$(TERRAFORM_STATE_STORAGE_ACCOUNT_NAME)' backendAzureRmContainerName: '$(TERRAFORM_STATE_STORAGE_CONTAINER_NAME)' backendAzureRmKey: 'state.tfstate' - task: TerraformTaskV4@4 name: terraformPlan displayName: Create Terraform Plan inputs: provider: 'azurerm' command: 'plan' commandOptions: '-out main.tfplan' environmentServiceNameAzureRM: '$(DEVOPS_SERVEICE_CONNECTION)' # Only runs if the 'terraformPlan' task has detected changes the in state. - task: TerraformTaskV4@4 displayName: Apply Terraform Plan condition: eq(variables['terraformPlan.changesPresent'], 'true') inputs: provider: 'azurerm' command: 'apply' commandOptions: 'main.tfplan' environmentServiceNameAzureRM: '$(DEVOPS_SERVEICE_CONNECTION)'

(3) In the Azure DevOps pipeline, I use a DevOps Service Connection, which refers to a Microsoft Entra ID app (a service principal). I have added this service principal to the Databricks account and the Databricks workspace, and give the service principal the account admin and workspace admin permissions.

However, the pipeline reports error in the last step.

/opt/hostedtoolcache/terraform/1.9.2/x64/terraform apply -auto-approve main.tfplan databricks_storage_credential.dbr_strg_cred: Creating... ╷ │ Error: cannot create storage credential: failed during request visitor: default auth: azure-cli: cannot get account info: exit status 1. Config: host=https://adb-123123123123123.1.azuredatabricks.net, azure_client_id=***, azure_tenant_id=d123123-1234-1234-1234-123123123. Env: DATABRICKS_HOST, ARM_CLIENT_ID, ARM_TENANT_ID │ │ with databricks_storage_credential.dbr_strg_cred, │ on main.tf line 33, in resource "databricks_storage_credential" "dbr_strg_cred": │ 33: resource "databricks_storage_credential" "dbr_strg_cred" { │ ╵ ##[error]Error: The process '/opt/hostedtoolcache/terraform/1.9.2/x64/terraform' failed with exit code 1

Does anybody have any idea? Thank you.

Re: Cannot use Terraform to create Databricks Storage Credential

AlbertWang — Tue, 24 Sep 2024 00:26:40 GMT

Still struggling with this issue. Can anyone kindly help?

Re: Cannot use Terraform to create Databricks Storage Credential

AlbertWang — Tue, 24 Sep 2024 20:41:01 GMT

I found the reason. Because I did not configure the `auth_type` for the Terraform Databricks provider, it uses the default auth type `azure-cli`. However, in my pipeline, I did not log in Azure CLI using `az login`. Therefore, the authentication of the Terraform Databricks provider does not work.

Re: Cannot use Terraform to create Databricks Storage Credential

MichaelFu — Wed, 26 Feb 2025 10:40:58 GMT

How exactly do you need to configure auth_type in this case? I tried different options but nothing seems to work. I also would like to use the Service Connection from Azure DevOps Pipeline to deploy Databricks via TerraformTaskV4@4.

Re: Cannot use Terraform to create Databricks Storage Credential

AlbertWang — Wed, 26 Feb 2025 19:35:40 GMT

I used the following configure in `main.tf`.

provider "databricks" { auth_type = "pat" }

Then, in my Azure Pipeline, I configure the following:

...... - task: AzureCLI@2 displayName: 'Get Databricks access token' inputs: azureSubscription: $(DEVOPS_SERVEICE_CONNECTION) scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | echo "Getting access token..." DATABRICKS_TOKEN=$(az account get-access-token --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d --query "accessToken" -o tsv) echo "##vso[task.setvariable variable=DATABRICKS_TOKEN]$DATABRICKS_TOKEN" ...... - task: TerraformTaskV4@4 name: terraformPlan displayName: Create Terraform Plan inputs: provider: 'azurerm' command: 'plan' commandOptions: '-out main.tfplan' environmentServiceNameAzureRM: '$(DEVOPS_SERVEICE_CONNECTION)' ......

I hope this is helpful.

Re: Cannot use Terraform to create Databricks Storage Credential

MichaelFu — Thu, 27 Feb 2025 06:51:04 GMT

Thank you for your reply! Where is $DATABRICKS_TOKEN then used in the terraform template?

Re: Cannot use Terraform to create Databricks Storage Credential

AlbertWang — Thu, 27 Feb 2025 20:04:22 GMT

echo "##vso[task.setvariable variable=DATABRICKS_TOKEN]$DATABRICKS_TOKEN"

The DATABRICKS_TOKEN is set as an environment variable. As mentioned, https://docs.databricks.com/aws/en/dev-tools/auth/pat and databricks_token | Resources | databricks/databricks | Terraform | Terraform Registry. The DATABRICKS_TOKEN will be used by Terraform for auth because

auth_type = "pat"

Re: Cannot use Terraform to create Databricks Storage Credential

PhatAdam — Tue, 06 May 2025 21:20:34 GMT

Thank you so much for posting, what is this --resource id? the managed account? the access connector or another app registration? I have tried the resource provider in ADO, it doesnt seem to matter. I can't auth no matter what i try.

Re: Cannot use Terraform to create Databricks Storage Credential

AlbertWang — Tue, 06 May 2025 22:37:26 GMT

You are welcome, PhatAdam.

This ID is the programmatic ID for Azure Databricks, or the unique resource ID for the Azure Databricks service. It is not my company's resource id, but Databricks Service's id. So I did not mask it when posting 🙂

You can find more details from this article.

Get Microsoft Entra ID tokens for service principals - Azure Databricks | Microsoft Learn