Programmatically activate groups in account

[Sven_Relijveld]

Hi,

I am currently trying to use the Accounts SDK to add External groups from Entra ID to functional groups within Databricks. I expect thousands of groups in Entra and I want to add these groups programmatically (for example) to a group in Databricks that has access to an Endpoint, or a Dashboard or give them consumer access by default. 

Example: thousand of Entra groups called 'projectgroup_projectcode_external_managed_automatic' into a non-Entra databricks group 'Databricks Vector Search Readers'. This way I can manage the permissions of functional groups within databricks and the organizational groups and their members are managed in Entra, outside my scope by another team.

Our Entra groups have a very standardized structure, with which i can filter it down to the correct set of groups
List group details. | Account Groups API | REST API reference | Azure Databricks

filter=displayName co "foo" and displayName co "bar"

Now I noticed that untill the groups are activated, I cannot find them with the API or SDK. I can find them through the UI however with Automatic Identity Management, and i notice this is powered by GraphQL, probably directly querying the Microsoft Graph API.

How can I programmatically 'activate' these External Entra groups within databricks account, such that i can manage them from there?

[Louis_Frolio · Accepted solution]

Hey @Sven_Relijveld , I did some digging/research and here is a summary of what I uncovered:

• There is currently no public Databricks Accounts API that lets you pre-activate or bulk-import Entra groups directly by object ID or filter. JIT provisioning via assignment is the only way for AIM.
• You can automate bulk initial activation by scripting permission/group/resource assignments in the UI or via account/workspace assignment APIs, if your environment has access.
• For direct Entra-to-Databricks group sync and management, configure a SCIM connector and manage assignments in Entra.
• After activation, all group operations, including permissions, access assignment, and consumption in group-based policies/workflows, can be performed programmatically via the Account Groups API, SDKs, or the Terraform provider.

Let me know if this is helpful.

Cheers, Louis.

[SvenRelijveld]

Hi!

I've been working on setting up the bulk initial activation at the creation time of the Entra groups. This seems to work. 

I missed the maximum number of groups in the account however, which seems to be 5K. That will likely be too low for my clients use-case. Is this a technical limit or something that can be adjusted?

Best,

Sven

[Louis_Frolio]

Hi @Sven_Relijveld — great to hear that your bulk-initial activation workflow is working as expected. Thanks for the update.

Regarding the 5K external group limit you’re seeing:

That is the current default soft quota for Azure Databricks accounts. It exists to prevent accidental large-scale syncs that could cause performance and governance challenges. That said, we do support environments that exceed this threshold — especially for enterprise-scale Entra-driven identity architectures like yours.

To move forward, the right next step is to file a support ticket with the details of your use case, scale projections, and identity topology. Our engineering team will review and can increase the limit where appropriate.

Cheers, Louis.

[SvenRelijveld]

Great, thank you Louis, for the quick and detailed response! We'll get the account team to go over the use-case with us.

Cheers, Sven