Databricks Community

zsucic1 · ‎03-21-2024

Hello everyone,

I have a few questions about MI capabilites:

Is it possible to define a managed identity for Azure Databricks Service resource and use it for e.g.:
1. Writing to Azure SQL Server database
2. Authenticating to Azure Devops in order to download a repo
Is it possible to define a managed identity for Azure App Service resource and use it to download MLFLOW models from Databricks (Workspace/unity catalog based) model registry, using

azure-identity Managed Identity library in python code to authenticate? If yes, are you aware of any additional steps to take, other than:

Turning on managed identity on App Service
Adding that managed identity to Databricks (as a service principal, which is required by Databricks)

When I did those steps, I got 403 authentication error, is it likely to simply be a mistake on our part?

Is it possible to use a service principal to download a Azure DevOps repo from a Databricks job, by linking a Databricks Git credentials entry to it, as opposed to a mere user like user@gmail.com?

Any answers, links, tips and comments are greatly appreciated.

Thanks in advance!

zsucic1 · ‎03-27-2024

Hi Kaniz,

Thank you immensely for this thorough response!
You really helped us a lot
and enabled us to start resolving this problem strategically.
May I just ask you a few follow ups:

1. Regarding your 2nd point, do you know specifically if it is possible
to download Mlflow models as a Azure Managed Identity? If it is then
we made a mistake somewhere and we can systematically try to resolve it.

2. Regarding your 3rd point, thank you,
service principal is certainly an improvement over user account,
but do you know if this could be further improved by downloading
Azure DevOps Repo as a "Azure Databricks Service Managed identity",
instead of as a service principal. A corrollary question would be:
is it possible to run Databricks jobs as a
"Azure Databricks Service Managed identity",
instead of as a service principal.