Adding service principal with Microsoft Entra ID fails

[Daan_Fostier]

Hi,

I am trying to add a service principal using Microsoft Entre ID, but I encounter an issue as described in the following documentation: https://learn.microsoft.com/en-us/azure/databricks/dev-tools/auth/oauth-m2m.

I followed the instructions step by step, but at step 3, I receive the following error:

"Failed to add existing service principal"

"Successfully created new service principal but failed to add the new service principal to this workspace. Error fetching user"

Identity federation is enabled, as the option to add service principals is available, and adding Databricks specific service principals works.

Under "Groups" and "Service Principals," I can see the service principal I'm trying to add, but it does not appear under "Users," and it doesn't seem possible to add the user manually.

How can I successfully add these users to the workspace?

[mark_ott]

The error message you encountered—“Successfully created new service principal but failed to add the new service principal to this workspace. Error fetching user”—along with the service principal's absence in “Users,” typically points to a synchronization or permissions issue between Microsoft Entra ID (formerly Azure AD) and Azure Databricks when identity federation is enabled.​

Key Observations

• You can successfully add Databricks-managed service principals, but not Microsoft Entra (Azure AD) service principals.

• The principal appears in “Groups”/“Service Principals,” but not in “Users.”

• Manual addition in the “Users” interface is not available for this entity.

Possible Causes

• Azure Entra ID Service Principal Missing Required Permissions: The service principal may not have the required roles or permissions in Azure, or isn’t assigned to the workspace.

• Propagation/Synchronization Delay: Sometimes, after changes in Entra ID, it can take a few minutes for the object to appear in Databricks.

• Identity Federation Misconfiguration: There could be subtle misconfigurations with how identity federation (Entra ID to Databricks) is set up.

• API/CLI Tooling Mismatch: Using older CLI/SDK versions or unsupported authentication flows can cause add-user API calls to fail.

Recommended Steps

1. Confirm Service Principal Assignment

• Make sure the service principal is assigned to the Databricks workspace in the Azure portal. This is necessary for the principal to be visible and manageable within Databricks.​

  • In Azure Portal, go to your Databricks workspace → Access Control (IAM) → Add role assignment → Assign the service principal with the required role (e.g., Contributor).

2. Ensure Service Principal Permissions

• The service principal must have the correct permissions, not just in Azure but also inside Databricks.​

  • Account admin privileges may be required for initial addition.

  • Check in the Databricks account console that the principal is assigned to the workspace and has at least “Can Use” or “Can Manage” permissions.

3. Synchronize/Propagate Changes

• Wait several minutes and try again—there may be directory synchronization delays.

• If using any automation or scripts, verify that token caches are not interfering.

4. Use the Latest Tools and APIs

• Make sure you are using the latest Databricks UI, CLI (v0.205+ for identity federation), or the current REST API versions for user/service principal management.​

5. Add via Databricks Account Console

• Attempt to add the service principal from the Databricks Account Console, as opposed to in-workspace UI.

  • Go to the account console → Manage service principals → Add to workspace.

6. Troubleshoot Errors

• If you have logs, look for errors related to “user not found” or “failed to fetch user” in both Azure and Databricks logs.

• Run a network test/lookup for the service principal’s object ID to make sure it resolves in both Azure AD and Databricks.

Special Cases

• If the principal stays missing in “Users” but shows in “Groups”/“Service Principals,” it often means that identity federation recognizes it at the account level but workspace assignment hasn’t completed properly.

• If manual assignment (via UI or API) still fails, you might need a Databricks support ticket, as this can sometimes involve backend propagation or permissions issues outside normal user control.

---

Quick Checklist

Action	Details
Assign principal to Databricks workspace (in Azure IAM)	Must assign via Azure Portal
Ensure proper roles (“Contributor” at minimum)	In Azure Portal and Databricks
Check “Service Principals” and “Users” views	Appearing in only one may signal sync problems
Wait for propagation (up to 30 mins)	Azure and Databricks sync delay possible
Add via Databricks Account Console, not just workspace UI	Go to Account Console → Principals → Add to workspace
Use latest CLI, SDK, or REST API versions	Outdated tools may not support identity federation properly
Review error and access logs	Details often appear in the audit trail or browser/CLI error output

---

If, after trying all the above, the service principal still cannot be added and does not appear in “Users,” then it’s likely a deeper propagation or identity federation issue requiring Databricks support intervention.​

For reference, always ensure your workflow aligns with the latest official documentation for OAuth M2M and service principal management.​