Greetings @jzu , I did some digging around with internal docs and references and put together some helpful tips and things to consider.
This is a common authorization issue related to permission propagation delays and ownership configuration when managing Databricks Unity Catalog metastores with Terraform using service principals or managed identities.
Root Cause
The "User not authorized" error occurs after the initial successful deployment because of how Databricks handles metastore ownership and permission caching. When the metastore is created, the `owner` parameter transfers ownership away from the creating identity (the managed identity) to the specified owner. Once this happens, the managed identity running the pipeline no longer has implicit permissions to read or manage the metastore, causing subsequent Terraform runs to fail when trying to read the metastore state.
Additionally, metastore admin assignment changes can take up to 30 seconds to propagate across the account, and may take even longer to take effect in workspaces due to caching protocols.
Solutions
Make the Managed Identity the Metastore Owner
Instead of setting a different owner, keep the managed identity as the owner of the metastore. Modify the resource configuration:
```hcl+
resource "databricks_metastore" "metastore" {
name = var.metastore_name
region = var.location
owner = azuread_service_principal.pipeline_identity.application_id
provider = databricks.accounts
}
```
This ensures the managed identity retains ongoing permissions to manage the metastore.
Assign Metastore Admin Role
If the managed identity needs to use a different owner, explicitly grant the managed identity the Metastore Admin role. This can be done through the Azure Databricks account console or via Terraform after the metastore is created:
```hcl
resource "databricks_metastore" "metastore" {
name = var.metastore_name
region = var.location
owner = var.metastore_owner
provider = databricks.accounts
}
resource "databricks_grant" "metastore_admin" {
metastore = databricks_metastore.metastore.id
principal = azuread_service_principal.pipeline_identity.application_id
privileges = ["CREATE_CATALOG", "CREATE_STORAGE_CREDENTIAL", "CREATE_EXTERNAL_LOCATION"]
provider = databricks.accounts
}
```
Account admins who create metastores become the initial metastore admin automatically, but when ownership is transferred, explicit permissions are needed.
Use a Dedicated Group for Metastore Administration
Databricks recommends using an Azure AD group as the metastore owner/admin rather than individual identities. Add the managed identity to this group:
1. Create an Azure AD group for Databricks administrators
2. Add the managed identity to this group
3. Set the group as the metastore owner
This approach provides better permission management and reduces propagation issues.
Account-Level vs Workspace-Level Provider
While using the account-level provider (`databricks.accounts`) is correct for metastore creation, be aware that some operations may work better with workspace-level providers. However, for metastore creation specifically, the account provider is the appropriate choice.
Verify Account Admin Status
Ensure the managed identity is properly configured as an Account Admin in the Databricks account console. Being an account admin alone may not be sufficient if ownership is transferred without explicit ongoing permissions.
Hope this helps, Louis.
