Create account group with terraform without account admin permissions

[dofrey]

I’m trying to create an account-level group in Databricks using Terraform. When creating a group via the UI, it automatically becomes an account-level group that can be reused across workspaces. However, I’m struggling to achieve the same using Terraform without account admin privileges.

Current Terraform Configuration:

provider "databricks" {
  alias      = "mws"
  host       = "https://accounts.azuredatabricks.net"
  account_id = "<account-id>"
}

resource "databricks_group" "ws_admins" {
  provider = databricks.mws
  display_name = "ws_dev_project_admins"
}

Issue:
When executing this script without account admin rights, I get the following error:

│ Error: cannot read group: failed during request visitor: default auth: azure-cli: cannot get access token: WARNING: Could not retrieve credential from local cache for service principal *** under tenant common. Trying credential under tenant xyz, assuming that is an app credential.
│ ERROR: AADSTS50059: No tenant-identifying information found in either the request or implied by any provided credentials. Trace ID: abc Correlation ID: def Timestamp: 2025-03-05 14:40:57Z
│ Interactive authentication is needed. Please run:
│ az login
│ . Config: host=https://accounts.azuredatabricks.net, account_id=uvw, azure_tenant_id=common
│ 
│   with databricks_group.ws_admins,
│   on databricks_groups.tf line 9, in resource "databricks_group" "ws_admins":
│    9: resource "databricks_group" "ws_admins" {

The group is only created successfully if the service account has account admin privileges, which I want to avoid.

What I’ve Tried:

• Using the workspace provider instead of databricks.mws – This only creates workspace-level groups, not account-level ones.
• Creating the group via the UI – This works as expected, automatically creating an account-level group.

Since it works via the UI, there must be a way to do this through the API. Does anyone know how to achieve this without requiring account admin privileges? Any guidance would be greatly appreciated.

[fazetu01]

I am also interested in the solution for this! Workspace-level groups cannot be used to grant permissions on Unity Catalog resources so I also need to be able to create account-level groups in terraform while not being an account admin.

[mark_ott · Accepted solution]

You cannot create account-level groups in Databricks with Terraform unless your authentication mechanism has account admin privileges. This is a design limitation of both the Databricks API and Terraform provider, which require admin-level permissions for managing resources at the account scope, including account-level groups.

Key Points

• Account-Level Group Creation: Only users or service principals with "account admin" privileges in Databricks can create or manage account-level groups via the API or Terraform provider. Workspace-level admin does not suffice.

• UI vs API Behavior: The UI automatically escalates your permissions if you are an account admin, which is why it works there. Without account admin privileges, the API and Terraform will enforce stricter access controls and error out.

• Azure AD Authentication Errors: The error you see (AADSTS50059, missing tenant-identifying info) is due to authentication failing for account-level admin APIs because your credentials lack the required scope.

• Workspace vs Account Scope: Using the workspace-scoped provider only lets you create workspace-only groups, not reusable account-level groups.

Possible Workarounds

• Service Account with Admin Privileges: If you require automation, you must perform group creation with a service principal or user configured as an account admin within Databricks.

• Manual Creation: You can request your Databricks account admin to create the account-level group via the UI or API, then reference that group in workspace provisioning scripts or Terraform as needed.

• RBAC Review: Some organizations can temporarily grant account admin privileges to service accounts for specific automation tasks, then revoke them after provisioning is complete.

Why the Restriction Exists

Databricks enforces this restriction for security reasons to prevent broad account-level changes by non-admins. Only account admins can manage entities shared between workspaces (groups, users, service principals) to safeguard cross-workspace governance.

Summary Table

Method	Account Admin Required?	Scope Created
Databricks UI	Yes	Account-level
API (Terraform Provider)	Yes	Account-level
Workspace provider	No	Workspace-level

Next Steps

To automate account-level group creation, ensure your automation identity is granted account admin rights in Databricks. Otherwise, delegate group creation to an existing admin or perform it manually via the UI.

If Databricks' permissions or API surfaces change in the future to allow delegated account-level group creation, review the Terraform provider documentation and Databricks release notes for updates.