cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Issue Creating Metastore Using Terraform with Service Principal Authentication

jv_v
New Contributor III

I'm encountering an issue when attempting to create a metastore using Terraform with service principal authentication. Below is the error message I receive:

Error:

"module.metastore_and_users.databricks_metastore.this: error: cannot create metastore: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method"

I've ensured that my service principal is set up correctly and that the necessary permissions are granted. However, the authentication still seems to be failing. I've followed the instructions on the provided documentation link, but the problem persists.


Here is the relevant portion of my Terraform code:

"
terraform {
required providers {
azurerm = {
source = "hashicorp/azurerm"
}
databricks = {
source = "databricks/databricks"
}
}
}

provider "azurerm"{
skip_provider_registration = true
features {}
subscription_id = var.subscription_id
tenant_id = var.tenant_id
client_id = var.client_id
client_secret = var.client_secret
}
// Provider for databricks account
provider "databricks" {
alias = "azure_account"
host = "https://accounts.azuredatabricks.net"
account_id = var.account_id
client_id = var.databricks_clientid
client_secret = var.databricks_clientsecret

}

// Provider for databricks workspace
provider "databricks" {
alias = "Workspace"
host = local.databricks_workspace_host
client_id = var.databricks_clientid
client_secret = var.databricks_clientsecret
}

resource "databricks_metastore" "this" {
name = var.metastore_name
storage_root = format("abfss://%s@%s.dfs.core.windows.net/",
azurerm_storage_container.unity_catalog.name,
azurerm_storage_account.unity_catalog.name)
force_destroy = true
owner = var.owner
}
output "output_metastore" {
value = databricks_metastore.this.metastore_id
}
"

Could anyone help me identify what might be going wrong or provide any suggestions on how to resolve this authentication issue?

Thank you in advance for your assistance!

1 ACCEPTED SOLUTION

Accepted Solutions

jacovangelder
Honored Contributor

You need to add the provider alias to the databricks_metastore resource, i.e.:

 

resource "databricks_metastore" "this" {
provider = databricks.azure_account
name = var.metastore_name
storage_root = format("abfss://%s@%s.dfs.core.windows.net/",
azurerm_storage_container.unity_catalog.name,
azurerm_storage_account.unity_catalog.name)
force_destroy = true
owner = var.owner
}

 



View solution in original post

2 REPLIES 2

daniel_sahal
Esteemed Contributor

@jv_v Since you're using two providers (one on account level, second on workspace level), make sure to specify the provider when trying to create a resource.
https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication

jacovangelder
Honored Contributor

You need to add the provider alias to the databricks_metastore resource, i.e.:

 

resource "databricks_metastore" "this" {
provider = databricks.azure_account
name = var.metastore_name
storage_root = format("abfss://%s@%s.dfs.core.windows.net/",
azurerm_storage_container.unity_catalog.name,
azurerm_storage_account.unity_catalog.name)
force_destroy = true
owner = var.owner
}

 



Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group