AzureDevOps and Databricks Connection using managed identity or service principal

[TitaMn]

Hi All!  Im in a project where i need to connect azure devops and databricks using managed identity to avoid the using of service account, PAT, etc.

The thing is i cant move forward with the connection since i cannot take the ownership of the files when i clone the repos, i need the managed identity to be owner of the files clones from azure devops

Is there a way to do that?

Thank you in advance

[Kaniz_Fatma]

Hi @TitaMn, Connecting Azure DevOps and Azure Databricks using managed identity is a great approach to enhance security and avoid using service accounts or personal access tokens (PATs).

Let’s explore some options:

1. Azure Managed Identity for Databricks:

   • User-Assigned Managed Identity: Databricks recommends using user-assigned managed identities for Azu...¹. You can create a user-assigned managed identity in Azure and associate it with your Databricks workspace.
   • However, please note that there are limitations when it comes to using managed identities with Databricks. For example, you cannot directly use Databricks Access Connector or Managed Identity for this purpos...².

2. Access Connector for Azure Databricks:

   • While managed identity is a good option, you might also consider using the Access Connector for Azure Databricks. However, be aware that some users find it less secure³.
   • To set up the Access Connector:
     • Log in to the Azure Portal as a Contributor or Owner of a resource group.
     • Search for “Access Connector for Azure Databricks” and create it.
     • Configure the necessary fields and create the connector.

3. Data Factory Integration:

   • If you’re using Azure Data Factory (ADF) to orchestrate your data workflows, you can set up managed identity authentication for Azure Databricks within ADF.
   • Navigate to the “Manage” pane in Data Factory, create a new linked service under “compute,” and select “Azure Databricks.”
   • Choose the appropriate cluster type (e.g., existing interactive cluster) and set the “authentication...⁴.

If you encounter any specific issues during the setup, feel free to seek further assistance. Good luck with your project! 😊🚀