Error connecting Databricks Notebook using managed identity from Azure Data Factory

[Prashanth24]

I am trying to connect Databricks Notebook using managed identity authentication type from Azure Data Factory. Below are the settings done. Error message is appended at the bottom of this message. With the same settings but with different authentication type like "Basic", "Service Prinicipal" is working fine. Please let me know anything i am missing out here.

URL => https://adb-xxxxxxxxxxxxxxxxxx.xx.azuredatabricks.net/api/2.0/jobs/runs/submit

Method => POST

Body => {"existing_cluster_id":"xxxx-xxxxxx-xxxxxxxx","notebook_task":{"notebook_path":"/Workspace/Users/<user email id>/workflows/primary_notebook","base_parameters":{"param_1":"First Web Activity"}}}

Authentication => System-assigned Managed Identity

Resources => https://adb-xxxxxxxxxxxxxxxx.xx.azuredatabricks.net/

Error is as below

Get access token from MSI failed for Datafactory data-factory1-us-east-2, region eu2. Please verify resource url is valid and retry. Details: Failed to get MI access token. The error message is: Acquire MI token from AAD failed. ErrorCode: invalid_resource, Message: AADSTS500011: The resource principal named https://adb-xxxxxxxxxxxxxxx.xx.azuredatabricks.net/ was not found in the tenant named xxxxxxxxxxxxxx. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You might have sent your authentication request to the wrong tenant.

[Kaniz_Fatma]

Hi @Prashanth24, To resolve this, ensure the resource URL is correctly set, grant the Data Factory Managed Identity Contributor role in the Databricks workspace, verify the Databricks workspace is registered in the correct Azure AD tenant, confirm the Managed Identity is properly set up and has necessary permissions, and double-check the Linked Service configuration in Azure Data Factory, making sure the authentication type is "Managed Service Identity" and the workspace resource ID is correct. 

If the problem persists, please let me know, and we can explore further troubleshooting steps.