topic Re: Run Task as Service Principal with Code in Azure DevOps Repo in Data Engineering

Run Task as Service Principal with Code in Azure DevOps Repo

pgruetter — Wed, 19 Apr 2023 06:05:44 GMT

Hi all

I have a task of type Notebook, source is Git (Azure DevOps). This task runs fine with my user, but if I change the Owner to a service principal, I get the following error:

Run result unavailable: run failed with error message

Failed to checkout Git repository: PERMISSION_DENIED: Invalid Git provider credentials.Go to User Settings > Git Integration to ensure that: ...

I assume the error is a bit misleading and the problem is actually missing authorizations on the DevOps side.

How exactly can I give access to the SP in DevOps? The SP is already added to the DevOps organization with Basic access level. I can't add it to the specific Repo somehow.

Thanks

Re: Run Task as Service Principal with Code in Azure DevOps Repo

User16752242622 — Wed, 19 Apr 2023 08:57:33 GMT

Hello, Thank you for reaching out to us.

The error could be due to misconfiguration while setting up the Service principal for Azure DevOps.

have you checked this doc:

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/ci-cd/ci-cd-sp

Re: Run Task as Service Principal with Code in Azure DevOps Repo

User16752242622 — Wed, 19 Apr 2023 08:59:25 GMT

Please feel free to share the complete error message here. Thank you!

Re: Run Task as Service Principal with Code in Azure DevOps Repo

pgruetter — Wed, 19 Apr 2023 09:32:27 GMT

Thanks a lot for the link, I'll have to check in detail. I first thought that's only for access from DevOps to Databricks but now I see it's also for Repo access in the other direction.

From quickly skimming the article, I'm not sure what is meant by "username associated with your Git provider" though. Is it the display name of the SP ?

Re: Run Task as Service Principal with Code in Azure DevOps Repo

User16752242622 — Wed, 19 Apr 2023 09:51:10 GMT

The username would be the name associated with Azure DevOps. Check this for Azure DevOps integration

https://learn.microsoft.com/en-gb/azure/databricks/repos/get-access-tokens-from-git-provider#--azure-devops-services

Re: Run Task as Service Principal with Code in Azure DevOps Repo

pgruetter — Wed, 19 Apr 2023 09:57:46 GMT

But the link describes how my personal user is connected to Azure DevOps. Since I want to use a service principal, it must be something else, right ?

Re: Run Task as Service Principal with Code in Azure DevOps Repo

Anonymous — Tue, 25 Apr 2023 03:43:16 GMT

@pgruetter :

To enable a service principal to access a specific Azure DevOps repository, you need to grant it the necessary permissions at both the organization and repository levels.

Here are the steps to grant the service principal the necessary permissions:

Navigate to your Azure DevOps organization and select the project that contains the repository you want to grant access to.
Select the "Settings" menu and then "Permissions" under "Repositories".
Select the repository you want to grant access to.
Click "Add" to add a new user or group.
In the "Add users and groups" dialog, search for the name of the service principal that you want to grant access to the repository.
In the "Assign permissions" dialog, select the appropriate permission level for the service principal. You may want to start with "Read" access and then grant additional permissions as needed.
Click "Add" to save the changes.

After you've granted the necessary permissions to the service principal at the repository level, you'll also need to make sure that it has the necessary permissions to access the code in the repository. You can do this by adding the service principal's credentials to the notebook task's Git repository connection.

Here's how to add the service principal's credentials:

In the Azure portal, navigate to the resource group that contains the Azure DevOps project.
Select the Azure DevOps project resource.
In the "Overview" tab, select "Service connections".
Select the connection for the Git repository that the notebook task is using.
Click "Edit" to edit the connection.
In the "Edit service connection" dialog, select "Service principal (manual)" as the authentication method.
Enter the service principal's client ID and client secret.
Click "Save" to save the changes.

After you've granted the necessary permissions and added the service principal's credentials, you should be able to run the notebook task with the service principal as the owner without encountering the "PERMISSION_DENIED" error.

Re: Run Task as Service Principal with Code in Azure DevOps Repo

jannemanson — Mon, 15 Jan 2024 10:28:26 GMT

Hello @pgruetter , any chance you figures this issue out? We followed the steps of Anonymous, but were not successful.

Cheers