Issue Creating Metastore Using Terraform with Service Principal Authentication

jv_v — Thu, 20 Jun 2024 16:29:08 GMT

I'm encountering an issue when attempting to create a metastore using Terraform with service principal authentication. Below is the error message I receive:

Error:

"module.metastore_and_users.databricks_metastore.this: error: cannot create metastore: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method"

I've ensured that my service principal is set up correctly and that the necessary permissions are granted. However, the authentication still seems to be failing. I've followed the instructions on the provided documentation link, but the problem persists.

Here is the relevant portion of my Terraform code:

"
terraform {
required providers {
azurerm = {
source = "hashicorp/azurerm"
}
databricks = {
source = "databricks/databricks"
}
}
}

provider "azurerm"{
skip_provider_registration = true
features {}
subscription_id = var.subscription_id
tenant_id = var.tenant_id
client_id = var.client_id
client_secret = var.client_secret
}
// Provider for databricks account
provider "databricks" {
alias = "azure_account"
host = "https://accounts.azuredatabricks.net"
account_id = var.account_id
client_id = var.databricks_clientid
client_secret = var.databricks_clientsecret

}

// Provider for databricks workspace
provider "databricks" {
alias = "Workspace"
host = local.databricks_workspace_host
client_id = var.databricks_clientid
client_secret = var.databricks_clientsecret
}

resource "databricks_metastore" "this" {
name = var.metastore_name
storage_root = format("abfss://%s@%s.dfs.core.windows.net/",
azurerm_storage_container.unity_catalog.name,
azurerm_storage_account.unity_catalog.name)
force_destroy = true
owner = var.owner
}
output "output_metastore" {
value = databricks_metastore.this.metastore_id
}
"

Could anyone help me identify what might be going wrong or provide any suggestions on how to resolve this authentication issue?

Thank you in advance for your assistance!

Re: Issue Creating Metastore Using Terraform with Service Principal Authentication

daniel_sahal — Fri, 21 Jun 2024 06:29:26 GMT

@jv_v Since you're using two providers (one on account level, second on workspace level), make sure to specify the provider when trying to create a resource.
https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication

Re: Issue Creating Metastore Using Terraform with Service Principal Authentication

jacovangelder — Fri, 21 Jun 2024 15:47:36 GMT

You need to add the provider alias to the databricks_metastore resource, i.e.:

resource "databricks_metastore" "this" { provider = databricks.azure_account name = var.metastore_name storage_root = format("abfss://%s@%s.dfs.core.windows.net/", azurerm_storage_container.unity_catalog.name, azurerm_storage_account.unity_catalog.name) force_destroy = true owner = var.owner }

topic Re: Issue Creating Metastore Using Terraform with Service Principal Authentication in Data Governance

Issue Creating Metastore Using Terraform with Service Principal Authentication

Re: Issue Creating Metastore Using Terraform with Service Principal Authentication

Re: Issue Creating Metastore Using Terraform with Service Principal Authentication