Using user-assigned managed identity inside notebook

[shervinmir]

Hi team,

I am interested in using a user-assigned managed identity within my notebook. I've come across examples using system-assigned managed identities or leveraging the Access Connector for Azure Databricks via Unity Catalog. However, as I do not have access to my Azure subscription as a Global Administrator, Unity Catalog isn't an option for me.

I've noticed that it's possible to run the notebook as the current user or add a service principal to the workspace. I've already added the user-assigned managed identity to my workspace. However, when attempting to run the notebook using this imported managed identity, I encounter authentication errors. My goal is to connect to a Gen 2 Storage account.

I've seen similar issues discussed in the community, but I haven't found any recent solutions. I'm reaching out to inquire if there's a way to achieve my objective.

Any assistance you can provide would be greatly appreciated.

Thanks for your time,

Shervin

[shervinmir]

Thanks for your reply. The user-managed identity is not the issue because I use it on ADF to pull data from the same storage account. I have followed the documentation for adding the managed identity to workspace. Perhaps my issue is the syntax code used inside the notebook. Maybe I need to get a token using a library? Any example or documentation would help.

I have tried

df = spark.read.format("delta").option("header", "true").load("abfss://{base folder}@{storage account name}.dfs.core.windows.net/{path to file}")

df.show()

i have also tried to use storage SDK and also tried to mount the storage but no success

from azure.storage.filedatalake import DataLakeServiceClient

from azure.identity import ManagedIdentityCredential credential = ManagedIdentityCredential()

service_client = DataLakeServiceClient( account_url=f"https://{storage_account_name}.dfs.core.windows.net", credential=credential )

# List filesystems (containers)

for filesystem in service_client.list_file_systems(): print(filesystem.name)

Can you point me to the correct documentation? I think my issues is what is inside the notebook.

Thanks again

[damour-jrs]

---

I have followed the documentation for adding the managed identity to workspace.

---

you didn't link the guides you followed.   but if it was https://learn.microsoft.com/en-us/azure/databricks/dev-tools/auth/azure-mi or https://learn.microsoft.com/en-us/azure/databricks/dev-tools/azure-mi-auth those are all about granting a UMI the ability to access a workspace...not about letting a workspace access other azure resources so they won't help AFAIK

[shervinmir]

Hi team,

Just wondering if anyone has any suggestions. We are still unable to use User Assigned managed identity inside the a notebook in Databricks to connect to a external Gen 2 storage 

[damour-jrs]

i also came across https://community.databricks.com/t5/data-engineering/how-to-assign-user-managed-identity-to-dbr-clus... which says...u have to use unity catalog's storage credential abstraction to make use of a umi...that makes a lot of sense as it abstracts datarbicks from the specific provider..eg UMI means nothing in AWS.