topic Re: Programmatically activate groups in account in Administration & Architecture

Programmatically activate groups in account

Sven_Relijveld — Thu, 25 Sep 2025 13:42:41 GMT

Hi,

I am currently trying to use the Accounts SDK to add External groups from Entra ID to functional groups within Databricks. I expect thousands of groups in Entra and I want to add these groups programmatically (for example) to a group in Databricks that has access to an Endpoint, or a Dashboard or give them consumer access by default.

Example: thousand of Entra groups called 'projectgroup_projectcode_external_managed_automatic' into a non-Entra databricks group 'Databricks Vector Search Readers'. This way I can manage the permissions of functional groups within databricks and the organizational groups and their members are managed in Entra, outside my scope by another team.

Our Entra groups have a very standardized structure, with which i can filter it down to the correct set of groups
List group details. | Account Groups API | REST API reference | Azure Databricks

filter=displayName co "foo" and displayName co "bar"

Now I noticed that untill the groups are activated, I cannot find them with the API or SDK. I can find them through the UI however with Automatic Identity Management, and i notice this is powered by GraphQL, probably directly querying the Microsoft Graph API.

How can I programmatically 'activate' these External Entra groups within databricks account, such that i can manage them from there?

Re: Programmatically activate groups in account

Louis_Frolio — Fri, 26 Sep 2025 16:15:08 GMT

Hey @Sven_Relijveld , I did some digging/research and here is a summary of what I uncovered:

There is currently no public Databricks Accounts API that lets you pre-activate or bulk-import Entra groups directly by object ID or filter. JIT provisioning via assignment is the only way for AIM.
You can automate bulk initial activation by scripting permission/group/resource assignments in the UI or via account/workspace assignment APIs, if your environment has access.
For direct Entra-to-Databricks group sync and management, configure a SCIM connector and manage assignments in Entra.
After activation, all group operations, including permissions, access assignment, and consumption in group-based policies/workflows, can be performed programmatically via the Account Groups API, SDKs, or the Terraform provider.

Let me know if this is helpful.

Cheers, Louis.

Re: Programmatically activate groups in account

SvenRelijveld — Fri, 31 Oct 2025 15:32:58 GMT

Hi!

I've been working on setting up the bulk initial activation at the creation time of the Entra groups. This seems to work.

I missed the maximum number of groups in the account however, which seems to be 5K. That will likely be too low for my clients use-case. Is this a technical limit or something that can be adjusted?

Best,

Sven

Re: Programmatically activate groups in account

Louis_Frolio — Fri, 31 Oct 2025 17:10:10 GMT

Hi @Sven_Relijveld — great to hear that your bulk-initial activation workflow is working as expected. Thanks for the update.

Regarding the 5K external group limit you’re seeing:

That is the current default soft quota for Azure Databricks accounts. It exists to prevent accidental large-scale syncs that could cause performance and governance challenges. That said, we do support environments that exceed this threshold — especially for enterprise-scale Entra-driven identity architectures like yours.

To move forward, the right next step is to file a support ticket with the details of your use case, scale projections, and identity topology. Our engineering team will review and can increase the limit where appropriate.

Cheers, Louis.

Re: Programmatically activate groups in account

SvenRelijveld — Sun, 02 Nov 2025 11:57:27 GMT

Great, thank you Louis, for the quick and detailed response! We'll get the account team to go over the use-case with us.

Cheers, Sven