topic Creating an Azure-Keyvault-backed secret scope with terraform in Data Engineering

Creating an Azure-Keyvault-backed secret scope with terraform

VicS — Thu, 14 Nov 2024 13:08:45 GMT

We want to create an Azure-Keyvault-backed secret scope with terraform - while we are able to do it via the UI with the URL https://adb-xxxxxxxx.x.azuredatabricks.net/?o=xxxxxxxxxxxxxx#secrets/createScope, I'm unable to do it with Terraform.

resource "databricks_secret_scope" "this" { name = "my-keyvault-name" keyvault_metadata { resource_id = "/subscriptions/x/resourceGroups/x/providers/Microsoft.KeyVault/vaults/my-keyvault-name" dns_name = "my-keyvault-name.vault.azure.net/" } }

In case it's relevant: while running Terraform we authenticate with a browser-pop up with our ActiveDirectory to authenticate against and deploy the Azure ressources.

Re: Creating an Azure-Keyvault-backed secret scope with terraform

szymon_dybczak — Thu, 14 Nov 2024 13:55:06 GMT

Could you share with us what error message you get?

Re: Creating an Azure-Keyvault-backed secret scope with terraform

VicS — Thu, 14 Nov 2024 14:22:53 GMT

Sorry I forgot, of course - Terraform plan goes through without a problem, but during the apply phase, I get

│ Error: cannot create secret scope: Scope with Azure KeyVault must have userAADToken defined! │ │ with databricks_secret_scope.this, │ on main_secret_scope_and_keyvault_acl.tf line 15, in resource "databricks_secret_scope" "this": │ 15: resource "databricks_secret_scope" "this" { │

Re: Creating an Azure-Keyvault-backed secret scope with terraform

JamesMrukHughes — Tue, 15 Apr 2025 10:51:21 GMT

I'm getting the same error message when trying to use the REST API as well and have tried about every combination I could think of to get the payload correct. I have tried AAD tokens for a user account, service principal, made those accounts full owners on the Key Vault, tried different names for the AAD Token with no success. Anyone out there, get a working solution?

url = f"{databricks_instance}/api/2.0/secrets/scopes/create"

headers = {

"Authorization": f"Bearer {token}",

"Content-Type": "application/json"

}

payload = {

"scope": scopeName,

"scope_backend_type": "AZURE_KEYVAULT",

"backend_azure_keyvault": {

"resource_id": keyVaultResourceId,

"dns_name": keyVaultDnsName

"user_aad_token": aadToken

}

response = requests.post(url, headers=headers, json=payload)

{"error_code":"INVALID_PARAMETER_VALUE","message":"Scope with Azure KeyVault must have userAADToken defined!","details":[{"@type":"type.googleapis.com/google.rpc.RequestInfo","request_id":"de4c03b9-10c1-959b-95e8-dbeabb046fa1","serving_data":""}]}

Re: Creating an Azure-Keyvault-backed secret scope with terraform

ElijahFord — Tue, 15 Apr 2025 13:37:42 GMT

I am having the same issue.

Re: Creating an Azure-Keyvault-backed secret scope with terraform

J-Bradlee — Thu, 22 May 2025 06:43:10 GMT

I am also having the same issue. I am deploying the Azure backed secrets across 3 different workspaces in my TF deployment. Strangley enough it works for 2/3 of my deployments but then I get the same error:

Scope with Azure KeyVault must have userAADToken defined!