Databricks Community

Shubh5073 · a week ago

Hi Databricks Community,

I am trying to enable Unity Catalog in my Azure Databricks account. As part of the setup, I created an Azure Databricks Access Connector and assigned its managed identity the required permissions on my ADLS Gen2 storage account.

I then provided the Access Connector Resource ID and ADLS path in the Databricks Account Console. However, I am getting the following error:

Azure Managed Identity Credential with Access Connector Id /subscriptions/7c585268-20a0-4c57-b952-bf6a47dd315a/resourceGroups/shubh_rg/providers/Microsoft.Databricks/accessConnectors/shubh_connect and Managed Identity ID: for Account Id: 1b687630-77a8-4ff1-888e-a2837de30bf8 could not be found.

Configuration

Azure Subscription ID: 7c585268-20a0-4c57-b952-bf6a47dd315a
Resource Group: shubh_rg
Access Connector: shubh_connect
Access Connector Resource ID: /subscriptions/7c585268-20a0-4c57-b952-bf6a47dd315a/resourceGroups/shubh_rg/providers/Microsoft.Databricks/accessConnectors/shubh_connect
Databricks Account ID: 1b687630-77a8-4ff1-888e-a2837de30bf8
Storage: Azure Data Lake Storage Gen2
Managed Identity: System-assigned managed identity
ADLS Role: Storage Blob Data Contributor

The Access Connector has been successfully created in Azure, and the managed identity has been assigned the required permissions on the ADLS storage account.

However, the Databricks Account Console is unable to find or validate the Access Connector.

Questions

Why is the Databricks Account Console unable to find the Access Connector?
Is there any account-level configuration or prerequisite required before creating the Unity Catalog metastore?
Does the Access Connector need to be associated with the Databricks workspace/account in any additional way?
Is there any issue with the managed identity configuration or Access Connector Resource ID?

Any guidance on how to resolve this issue would be greatly appreciated.

Thank you.

ShamenParis · a week ago

Hi @Shubh5073 ,

As per my understanding of the issue you are facing, the error indicates that Databricks cannot locate the Access Connector resource in Azure, rather than a problem with your ADLS permissions (since you already correctly assigned Storage Blob Data Contributor).

The most common cause for this specific "could not be found" error is missing Azure permissions. The identity you are using to log into the Databricks Account Console must have the Reader role on the Access Connector resource in Azure, otherwise Databricks cannot verify its existence (I would say check your access on the Access Connector). Additionally, ensure the Resource ID is copied exactly from the Azure Portal's JSON view, as it is strictly case-sensitive.

To address your configuration question: If you are setting up the root storage for a completely new Metastore, this must be done at the Account level. However, if you are simply creating an External Location for an existing Metastore, you can create the Storage Credential directly within the Workspace UI. You only need the appropriate metastore admin privileges to create the credential, which you can then use to configure your external locations.

For more information:

Manage External Locations: https://learn.microsoft.com/en-us/azure/databricks/connect/unity-catalog/cloud-storage/manage-extern...

Access Connector: https://learn.microsoft.com/en-us/azure/databricks/connect/unity-catalog/cloud-storage/azure-managed...

Aparajita_Nag · yesterday

Hi,

I need help.

While creating metastore in UI, I have created Access connector with system assigned managed identity as I can see in the JSON view. And storage blob data contribitor is also given to the access connector which I created in Azure portal. The issue is: Azure Managed Identity Credential with Access Connector Id /subscriptions/3d41ae7a-6c82-440f-8ec8-d1df27199109/resourceGroups/resourcesuntitycatalog/providers/Microsoft.Databricks/accessConnectors/AccessConnectorUnityCatalog and Managed Identity ID: for Account Id: 5d77c2be-ed5c-45c8-b5ee-44e20f21d1f9 could not be found.

How to solve this?

ShamenParis · yesterday

Hi @Aparajita_Nag

The "could not be found" error indicates Databricks cannot locate the Access Connector resource in Azure. This happens before any ADLS permissions are checked. Because I lack full information on your setup, you can verify the below list (if you have already confirmed some of these things, there is no need to do them again):

Azure RBAC Role: The account signed into the Databricks Account Console must have at least the Reader role assigned on the Access Connector resource itself. Assigning "Storage Blob Data Contributor" on the storage account is a separate requirement and does not grant access to the connector.
Resource ID Accuracy: In your second error message, the Resource Group is named resourcesuntitycatalog. Ensure the full Access Connector resource ID is copied exactly from the Azure portal. Any typo or case-sensitivity mismatch will cause this failure.
Entra ID Tenant Alignment: Confirm the Azure Subscription hosting the Access Connector is in the exact same Microsoft Entra ID tenant as your Databricks account. A connector in a different tenant is invisible to the account.
Identity Type Selection: The empty space in the error string and Managed Identity ID: for Account Id: confirms Databricks is attempting to resolve a system-assigned identity. Verify that the system-assigned managed identity is enabled on the Access Connector.

Aparajita_Nag · 13 hours ago

Hi ShamenParis,

Confirming the details from my end.

1. Given reader role to the account signing into Databricks. This user adrinilmitraazure_gmail.com#EXT#@adrinilmitraazuregmail.onmicrosoft.com has Reader role in storage account IAM alongside Access Connector has Storage Blob Data Contributor Role.

2. My ADLS Gen2 Path is: abfss://unitycatalogcont@stgaccuntitycatalog.dfs.core.windows.net/metadirectoryadri

and Access Connector ID is copied from Access Connector Resource ID: /subscriptions/3d41ae7a-6c82-440f-8ec8-d1df27199109/resourceGroups/resourcesuntitycatalog/providers/Microsoft.Databricks/accessConnectors/AccessConnectorUnityCatalog

3. Step 1: Check the Tenant ID of the Access Connector Subscription- Checked subscription(hosting the Access Connector) ID: 3d41ae7a-6c82-440f-8ec8-d1df27199109 has parent management group as b2a24d99-4b47-403d-8bc4-359568eed7b7.

Step 2: Check the Tenant ID of your Databricks Account

I can confirm that the related workspace in databricks is using the same subscription 3d41ae7a-6c82-440f-8ec8-d1df27199109 where the Tenant ID is same b2a24d99-4b47-403d-8bc4-359568eed7b7.

But still getting the same issue:

Azure Managed Identity Credential with Access Connector Id /subscriptions/3d41ae7a-6c82-440f-8ec8-d1df27199109/resourceGroups/resourcesuntitycatalog/providers/Microsoft.Databricks/accessConnectors/AccessConnectorUnityCatalog and Managed Identity ID: for Account Id: 5d77c2be-ed5c-45c8-b5ee-44e20f21d1f9 could not be found.

Please let me know what to do now?

ShamenParis · 11 hours ago

Hi @Aparajita_Nag

Thank you for verifying those details! Looking closely at your first point, I spotted the exact reason why this is failing.

You mentioned that your user has the Reader role in the Storage Account IAM. However, to create the metastore, your user account must have the Reader role on the Access Connector resource itself.

Because your user lacks permission to "see" the Access Connector in Azure, the Databricks Account Console cannot verify its existence when you click create, resulting in the "could not be found" error.

How to Fix This:

Go to the Access Connector in Azure: Open the Azure Portal and navigate to your resource group (resourcesuntitycatalog), then click on your Access Connector (AccessConnectorUnityCatalog).
Assign the Role:
- Click on Access control (IAM) on the left menu.
- Click Add > Add role assignment.
- Select the Reader role (or Contributor).
- Assign it directly to your user account (adrinilmitraazure_gmail.com#EXT#@adrinilmitraazuregmail.onmicrosoft.com).
Refresh Your Session: This is a crucial step. After assigning the role, log out of the Databricks Account Console entirely and log back in. This forces Azure to generate a new authentication token that includes your new permissions.

Once you log back in with the fresh token, try creating the metastore again. The Account Console should now be able to successfully locate the Access Connector.

Aparajita_Nag · 9 hours ago

Hi ShamenParis,

I have done exactly as you have said to provide reader role to EXT user account in Access Control IAM. But getting the same error.

Azure Managed Identity Credential with Access Connector Id /subscriptions/3d41ae7a-6c82-440f-8ec8-d1df27199109/resourceGroups/resourcesuntitycatalog/providers/Microsoft.Databricks/accessConnectors/AccessConnectorUnityCatalog and Managed Identity ID: for Account Id: 5d77c2be-ed5c-45c8-b5ee-44e20f21d1f9 could not be found.

I have logged out of databricks and logged in again to create metastore.

But alas, getting the same error again.

Please help. I am not getting why this happening.