Issue with Service Principal and Grants in Databricks

[jv_v]

Hi,

We created a service principal in Databricks as per the documentation here.

However, when we execute the following SQL query, we are unable to see the service principal:

SHOW GRANTS testservice ON METASTORE

error:
[RequestId=564cbcf9-e8b7-476d-a4db-96e6dccb87ad ErrorClass=PRINCIPAL_DOES_NOT_EXIST] Could not find principal with name testservice

We believe this might be the reason why we are unable to create an external location and also unable to grant the "CREATE EXTERNAL LOCATION" permission using the following Terraform code:

resource "databricks_grants" "grant" {

depends_on = [databricks_metastore.this]

  metastore = "databricks_metastore.this.id"

  grant {

    principal = "testservice"

    privileges = ["CREATE_EXTERNAL_LOCATION","CREATE_CATALOG"]

  }

}

// Task011 Create external location to be used as root storage by dev catalog

resource "databricks_external_location" "dev_location" {

  provider = databricks.Workspace

  name = var.external_location_name

  #url="abfss://strgconr@demorkstrgac.dfs.core.windows.net"

  url = format("abfss://%s@%s.dfs.core.windows.net/",azurerm_storage_container.dev_catalog.name,

  module.metastore_and_users.azurerm_storage_account_unity_catalog.name)

  credential_name = databricks_storage_credential.external_mi.id

  owner           = var.owner

  comment         = "External location used by dev catalog as root storage"

}

error: cannot create grants: metastore_id must be empty or equal to the metastore id assigned to the workspace.

Can you help us understand why the service principal is not appearing in the query results and how we can resolve this issue to create an external location and grant the necessary permissions?

[Kaniz_Fatma]

Hi @jv_v, 

• Ensure that the service principal testservice was created successfully. You can check this in the Azure portal under Azure Active Directory > App registrations.
• Make sure the service principal has the necessary permissions to access the Databricks workspace and the metastore. This includes roles like Contributor or Owner on the Databricks resource group.
• Verify that the metastore ID used in your Terraform script matches the one assigned to your Databricks workspace. You can find the metastore ID in the Databricks UI under Data > Metastores.
• Double-check your Terraform configuration to ensure that the metastore ID is correctly referenced.

If these steps don’t resolve the issue, please provide more details or any additional error messages you encounter, and we can further troubleshoot together.