Issue Creating Metastore Using Terraform with Service Principal Authentication

[jv_v]

I'm encountering an issue when attempting to create a metastore using Terraform with service principal authentication. Below is the error message I receive:

Error:

"module.metastore_and_users.databricks_metastore.this: error: cannot create metastore: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method"

I've ensured that my service principal is set up correctly and that the necessary permissions are granted. However, the authentication still seems to be failing. I've followed the instructions on the provided documentation link, but the problem persists.

Here is the relevant portion of my Terraform code:

"
terraform {
required providers {
azurerm = {
source = "hashicorp/azurerm"
}
databricks = {
source = "databricks/databricks"
}
}
}

provider "azurerm"{
skip_provider_registration = true
features {}
subscription_id = var.subscription_id
tenant_id = var.tenant_id
client_id = var.client_id
client_secret = var.client_secret
}
// Provider for databricks account
provider "databricks" {
alias = "azure_account"
host = "https://accounts.azuredatabricks.net"
account_id = var.account_id
client_id = var.databricks_clientid
client_secret = var.databricks_clientsecret

}

// Provider for databricks workspace
provider "databricks" {
alias = "Workspace"
host = local.databricks_workspace_host
client_id = var.databricks_clientid
client_secret = var.databricks_clientsecret
}

resource "databricks_metastore" "this" {
name = var.metastore_name
storage_root = format("abfss://%s@%s.dfs.core.windows.net/",
azurerm_storage_container.unity_catalog.name,
azurerm_storage_account.unity_catalog.name)
force_destroy = true
owner = var.owner
}
output "output_metastore" {
value = databricks_metastore.this.metastore_id
}
"

Could anyone help me identify what might be going wrong or provide any suggestions on how to resolve this authentication issue?

Thank you in advance for your assistance!

[daniel_sahal]

@jv_v Since you're using two providers (one on account level, second on workspace level), make sure to specify the provider when trying to create a resource.
https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication

[jacovangelder · Accepted solution]

You need to add the provider alias to the databricks_metastore resource, i.e.:

resource "databricks_metastore" "this" {
provider = databricks.azure_account
name = var.metastore_name
storage_root = format("abfss://%s@%s.dfs.core.windows.net/",
azurerm_storage_container.unity_catalog.name,
azurerm_storage_account.unity_catalog.name)
force_destroy = true
owner = var.owner
}