Hi @discuss_darende,
@KartikBhatnagar is on the right track in the replies above. Let me expand on this with the full picture.
The "InternalError" you are seeing when trying to add users to an Azure AD-synced group via SCIM is expected behavior. Groups that are synced from Microsoft Entra ID (formerly Azure AD) into Databricks are classified as "External" groups, and by default their membership is immutable within Databricks. Any attempt to modify membership of these groups -- whether through the SCIM API, the account console UI, or the workspace admin settings -- will fail because Databricks enforces that external group membership must be managed in your identity provider, not in Databricks.
Let me walk through the details.
WHY THIS HAPPENS
When you sync a group from Microsoft Entra ID to Databricks using the SCIM provisioning connector (or automatic identity management), that group gets a source type of "External" in Databricks. You can verify this by looking at the Source column in the Groups list under your account console or workspace admin settings.
External groups are treated as read-only in Databricks by default. A feature called "Immutable external groups preview" is enabled by default in the account console, which prevents any membership changes to external groups from within Databricks. This is intentional -- it ensures that your identity provider (Microsoft Entra ID) remains the single source of truth for group membership, and that local edits in Databricks do not cause sync conflicts.
When you send a SCIM PATCH request to add a member to one of these external groups, Databricks returns an InternalError because the operation is blocked at the platform level.
HOW TO PROPERLY MANAGE MEMBERSHIP OF AZURE AD-SYNCED GROUPS
Option 1: Manage membership in Microsoft Entra ID (Recommended)
The correct approach is to add or remove users from the group in Microsoft Entra ID. The SCIM provisioning connector will automatically sync the updated membership to Databricks within 20-40 minutes. To force an immediate sync, go to your enterprise application in the Azure Portal under Manage > Provisioning and select "Clear current state and restart synchronization."
This is the approach Databricks recommends because it keeps your identity provider as the authoritative source for group membership.
Option 2: Disable immutable external groups (Not recommended for most cases)
If you have a specific need to manage external group membership directly in Databricks, an account admin can disable the "Immutable external groups preview" toggle. To do this:
1. Log in to the Databricks account console at https://accounts.azuredatabricks.net
2. Navigate to the Preview page
3. Find and disable "Immutable external groups preview"
Important caveats about this approach:
- Any membership changes you make in Databricks may be overwritten by the next SCIM sync from Entra ID, which could cause confusion.
- If you are using Automatic Identity Management (enabled by default for accounts created after August 1, 2025), external groups CANNOT be updated in Databricks even when the immutable external groups preview is disabled.
- This approach can lead to drift between your identity provider and Databricks, which is difficult to troubleshoot.
Option 3: Create a Databricks-native account group instead
If you need a group whose membership is managed entirely within Databricks, create a new "Account" group (not synced from Entra ID) directly in the Databricks account console or via the Account Groups API. Account-native groups are fully editable. You can then nest this group or the external group as needed to achieve the access structure you want.
ACCOUNT-LEVEL VS WORKSPACE-LEVEL SCIM
There is another common source of errors to be aware of. Databricks has two levels of SCIM:
- Account-level SCIM (recommended): Provisions users and groups to the Databricks account. Uses the endpoint /api/2.1/accounts/{account_id}/scim/v2/. Groups created this way are account groups.
- Workspace-level SCIM (legacy, Public Preview): Provisions users and groups directly to a single workspace. This is a legacy approach.
If your workspace has identity federation enabled (which is the default for most workspaces), then workspace-level SCIM API calls will fail if they involve account groups or external groups. If you are making SCIM API calls against the workspace endpoint (e.g., {workspace-domain}/api/2.0/preview/scim/v2/Groups/) to modify a group that was synced at the account level, that would also produce an error.
Make sure you are using the correct API endpoint:
- Account admins: {account-domain}/api/2.1/accounts/{account_id}/scim/v2/
- Workspace admins/group managers: {workspace-domain}/api/2.0/account/scim/v2/
AUTOMATIC IDENTITY MANAGEMENT (NEWER ALTERNATIVE TO SCIM)
If your Databricks account was created after August 1, 2025, or if you have opted in, you may have Automatic Identity Management enabled. This is a newer approach that does not require configuring an enterprise application in Entra ID. It uses Microsoft Entra ID as the source of record directly and supports features that SCIM does not, such as nested groups and service principal syncing.
With automatic identity management, external groups are strictly managed by Entra ID and cannot be modified in Databricks at all, regardless of the immutable external groups preview setting.
SUMMARY / RECOMMENDED STEPS
1. Confirm the group source: In the account console under User Management > Groups, check the Source column for the group in question. If it says "External," that is why your SCIM call fails.
2. Add users to the group in Microsoft Entra ID: Go to the Azure Portal, find the enterprise application or the Entra ID group, and add the users there. Wait for the next sync cycle (20-40 minutes) or trigger a manual sync.
3. If you must manage the group in Databricks: Consider creating a new Databricks-native account group instead of trying to modify the external group.
4. Check your API endpoint: Make sure you are calling the account-level SCIM API, not the workspace-level SCIM API, if your workspace uses identity federation.
RELEVANT DOCUMENTATION
- Configure SCIM provisioning using Microsoft Entra ID:
https://learn.microsoft.com/en-us/azure/databricks/admin/users-groups/scim/aad
- Groups overview (explains External vs Account vs Workspace-local groups):
https://learn.microsoft.com/en-us/azure/databricks/admin/users-groups/groups
- Manage groups (includes the note about immutable external groups):
https://learn.microsoft.com/en-us/azure/databricks/admin/users-groups/manage-groups
- SCIM provisioning overview and migration from workspace-level to account-level:
https://docs.databricks.com/en/admin/users-groups/scim/
- Identity management best practices:
https://docs.databricks.com/en/admin/users-groups/best-practices.html
- Automatic identity management (newer alternative to SCIM for Azure):
https://learn.microsoft.com/en-us/azure/databricks/admin/users-groups/automatic-identity-management
Hope this helps. The short version is: external groups synced from Azure AD are immutable in Databricks by design. Add users to the group in Microsoft Entra ID and let the sync propagate the change.
* This reply used an agent system I built to research and draft this response based on the wide set of documentation I have available and previous memory. I personally review the draft for any obvious issues and for monitoring system reliability and update it when I detect any drift, but there is still a small chance that something is inaccurate, especially if you are experimenting with brand new features.
