How to make an Entra group available for GRANT commands inside a workspace?

[mzs]

We're using Azure Databricks and automatic identity management. Users and groups sync over automatically.

If I want to grant permissions to an Entra group to a schema, I can't just run something like this in a workspace notebook:

GRANT USE SCHEMA ON SCHEMA xxx.yyy TO `Example Entra Group`;

It can't find the group. If I go to workspace settings -> security -> groups -> add group, and start typing the name of the group, it populates in a dropdown. If I click on the group it found, the browser makes a GraphQL call to "GetOrCreateIdpGroup". Once that's happened, I can cancel out of the "add group" dialog, and my GRANT query above starts working.

How can I do this programmatically using the API? Ideally at the workspace level.

I'd like to automate the assignment of groups to schemas using a job and service principal. I don't particularly care if the groups are in Entra or Databricks, but I'm trying to avoid using Databricks account-level APIs as they don't have fine-grained permissions: whatever process or job is creating a group in the Databricks account would need full admin privileges at the account level. I figured I could create the groups in Entra instead, because apps in Entra can be given access just to create groups. The group seems like it syncs over, but I can't use it in a GRANT command in a workspace until I go through the "add group" workspace UI above.

[Ashwin_DSA]

Hi @mzs,

To make sure we suggest the right option, can you share a bit more about your setup?

• Cloud & workspace type: Are you on Azure Databricks, and is the workspace identity-federated (Unity Catalog enabled)?
• Identity sync: Are you using Automatic Identity Management (AIM) with Entra, SCIM, or both?
• How are you managing groups today? Terraform, REST API, or just SQL GRANT from notebooks?
• What principal is running the automation? (service principal vs. user; workspace-admin vs. limited role)

[mzs]

Hi Ashwin, yes, it's Azure Databricks, Unity Catalog is enabled, and automatic identity management is enabled. We are not using SCIM.

I'm looking for ways to automate group creation using a fairly limited-privilege service principal. Either using the REST API from outside Databricks, or maybe a job running as a service principal inside Databricks that I could then trigger from outside. If I do it inside Databricks, I figure I could use GRANT and/or the REST API with WorkspaceClient() and the default credentials available to the job.

This would be using a service principal, probably one that we'd create at either the account or workspace level in Databricks. I would give it the minimum privileges possible to do what it needs to do: create a schema and assign a group to the schema.

But I was testing GRANT manually in the SQL Editor as an Entra user with workspace admin privileges.

[Ashwin_DSA · Accepted solution]

Hi @mzs,

Given that you’re on Azure Databricks + UC + AIM (no SCIM), here are some recommendations.

Firstly, I think that you should create/manage the group in Entra only. You should avoid creating the groups in Databricks directly. AIM treats Entra as the source of truth.

You can then programmatically activate the Entra group in Databricks (the UI’s GetOrCreateIdpGroup) using IAM v2.

Workspace-level endpoint (works from a job or from outside):

POST https://<workspace-host>/api/2.0/identity/groups/resolveByExternalId
Authorization: Bearer <workspace_admin_SP_OAuth_token>
Content-Type: application/json

{ "external_id": "<entra_group_object_id>" }

This ensures the Entra group exists as a Databricks principal (created if needed) and is referenceable by display name in the GRANT/UC APIs in that workspace.

You can create a schema and assign permissions with the same SP using REST 

# Create schema
POST https://<workspace-host>/api/2.1/unity-catalog/schemas
Authorization: Bearer <token>
Content-Type: application/json

{
  "name": "my_schema",
  "catalog_name": "main",
  "comment": "…"
}

and then either use SQL

GRANT USE SCHEMA ON SCHEMA main.my_schema TO `Example Entra Group`;

or UC grants API (/api/2.1/unity-catalog/permissions/…).

You can do this inside a Databricks job (recommended) by configuring a workspace-admin service principal for the workspace (minimal Databricks side privilege you need today to call IAM v2 + create schemas):

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()  # job runs as SP via OAuth

entra_group_id = "<entra_group_object_id>"
group_display_name = "Example Entra Group"

# 1. Activate group from Entra
w.iamv2.workspace_iam_v2.resolve_group_proxy(external_id=entra_group_id)

# 2. Create schema (if needed)
w.unity_catalog.create_schema(
    name="my_schema",
    catalog_name="main",
)

# 3. Grant to the group by display name
w.sql.statements.execute(
    warehouse_id="<sql_warehouse_id>",
    statement=f"GRANT USE SCHEMA ON SCHEMA main.my_schema TO `{group_display_name}`",
    wait=True,
)

From outside Databricks you just trigger this job with the schema + group parameters.

In terms of privileges for the Databricks SP... For workspace, make it workspace admin (that’s currently the practical minimum to call identity/groups/resolveByExternalId, and create UC schemas). Fo unity catalog, grant this SP USE CATALOG and CREATE SCHEMA (and OWN/GRANT on the schema if you want it to manage further grants).

That gives you no account-admin SCIM or group-creation, groups fully owned in Entra and a single, limited-scope SP that can activate Entra groups via resolveByExternalId, create schemas, and run GRANTs, either as a Databricks job or from an external script using the same REST calls.

Try this out and let me know how it goes. If you encounter any issues, let me know. 

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.

[mzs]

Hi Ashwin,

Thanks for your reply. WorkspaceClient doesn't seem to have an iamv2 or workspace_iam_v2 attribute - I get an AttributeError.

w = WorkspaceClient()  # job runs as SP via OAuth

resolve_group_response = w.iamv2.workspace_iam_v2.resolve_group_proxy(external_id=entra_group_id)

AttributeError: 'WorkspaceClient' object has no attribute 'iamv2'

I'm using Serverless - environment v5:

Thanks!

[mzs]

Genie Code did a decent job fixing this up for me. Thanks again, Ashwin!

from databricks.sdk import WorkspaceClient
from databricks.sdk.service.catalog import PermissionsChange, Privilege
import re

catalog_name="dataeng_us"

# Job parameters:
dbutils.widgets.text("entra_group_id", "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "Entra Group ID")
dbutils.widgets.text("schema_name", "xxx", "Schema Name")
entra_group_id = dbutils.widgets.get("entra_group_id")
schema_name = dbutils.widgets.get("schema_name")

# Validate entra group ID looks like a UUID (case insensitive)
if not re.match(r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$", entra_group_id):
    raise ValueError(f"Invalid entra_group_id '{entra_group_id}'. Must be a valid UUID.")


# Unity Catalog schema names must match: ^[a-zA-Z][a-zA-Z0-9_]{2,254}$
if not re.match(r"^[a-zA-Z][a-zA-Z0-9_]{2,254}$", schema_name):
    raise ValueError(f"Invalid schema name '{schema_name}'. Must start with a letter and contain only letters, numbers, and underscores (3-255 chars).")

print("Starting WorkspaceClient")

w = WorkspaceClient()  # job runs as SP via OAuth

print("Resolving group from Entra")

# 1. Activate group from Entra
# https://databricks-sdk-py.readthedocs.io/en/stable/workspace/iamv2/workspace_iam_v2.html
resolve_group_response = w.workspace_iam_v2.resolve_group_proxy(external_id=entra_group_id)
print(resolve_group_response)
print(resolve_group_response.group)
print(f"Resolved group {resolve_group_response.group.group_name}")

# 2. Create schema (if needed)
full_schema_name = f"{catalog_name}.{schema_name}"
try:
    w.schemas.get(full_name=full_schema_name)
    print(f"Schema '{schema_name}' already exists in catalog '{catalog_name}'.")
except Exception:
    w.schemas.create(
        name=schema_name,
        catalog_name=catalog_name,
    )
    print(f"Schema '{schema_name}' created in catalog '{catalog_name}'.")

# 3. Grant to the group using Unity Catalog API
# TODO: more permissions will be needed.
w.grants.update(
    securable_type="schema",
    full_name=full_schema_name,
    changes=[
        PermissionsChange(
            add=[Privilege.USE_SCHEMA],
            principal=resolve_group_response.group.group_name,
        )
    ],
)

[Ashwin_DSA]

That's great. I was planning to respond later today, but I'm glad it's resolved now.