11-02-2021 10:05 AM
Databricks Repos best-practices recommend using the Repos REST API to update a repo via your git provider. The REST API requires authentication, which can be done one of two ways:
Using a user access token authenticates the REST API as the user, so all repos actions are performed as the user identity. This isn't desirable for automation, as all automation tasks are tied to a specific user account. In this case, a service principal would be preferable. As far as I can tell, the service principal doesn't work in Azure DevOps, because the service principal doesn't have access to the Azure DevOps git repo.
Has anyone had success getting a service principal access to Azure DevOps? If not, what alternatives have people used to integrate Databricks Repos with Azure DevOps CI/CD (apart from using personal access tokens)?
07-31-2023 03:28 AM
How did you solve this? Where did you find a way to create PAT tokens for Service Principals? The other comments don't make it that clear either for me
07-31-2023 05:03 AM
Unfortunately I didn't find any solution to this. 🙁
07-31-2023 07:50 AM
Did you manage to run the workflow integrated with Git with a service principal eventually?
04-03-2024 07:38 AM
Getting this when using Postman on api/2.0/token-management/on-behalf-of/tokens
{
"error_code": "ENDPOINT_NOT_FOUND",
"message": "No API found for 'GET /token-management/on-behalf-of/tokens'"
}
Using the databricks cli (databricks token-management create-obo-token) I get the following error
Error: On-behalf-of token creation for service principals is not enabled for this workspace
01-10-2023 01:52 PM
Hello,
My use case is to create dbt task inside of a databricks workflow.
It needs to specify `git_source`, and my workflow is run under a service principal account.
Unfortunately, from the beginning of the workflow run. an error is raised like:
"Failed to checkout Git repository: PERMISSION_DENIED: Encountered an error with your Azure Active Directory credentials. Please try logging out of Azure Active Directory (https://portal.azure.com) and logging back in."
But I've no where to grant to the service principal the Azure DevOps checkout permission.
Replacing the service principal with an standard user account works, but we can not use user account in production.
01-10-2023 02:41 PM
Are you able to use Azure DevOps Personal Access token for this instead of Active Directory credentials? Using DevOps PATS directly should work with service principals.
01-10-2023 03:03 PM
yes, I tried adding PAT in the git_source, it doesn't work neither.
I'm sure the PAT and the URL is OK, as I can `git clone (git_source url value with PAT inside)` locally without any issue.
tested with runtime: 10.4.x-cpu-ml-scala2.12
02-23-2023 06:55 AM
@Vaibhav Sethi "first add the Git PAT token for the service principal via the Git Credential API. " - is this possible? On the Git Credentials API guide I only see the option to set the credentials for the current user (not a service principal)
02-23-2023 02:38 PM
yes, it is. you could find a quick tuto from this link:
Repos configuration for Azure Service Principal (databricks.com)
04-03-2024 07:30 AM
The link is broken
08-23-2023 12:58 PM
I am also having this issue.
03-07-2024 07:09 AM - edited 03-07-2024 07:12 AM
Since last year, Azure offers workload identity federation, which can be used to to authenticate SPs against Azure DevOps using Microsoft Entra ID. Any chances Databricks might leverage this to implement OAuth for authenticating service principals against Azure DevOps?
04-03-2024 07:10 AM
Having the exact same problem. Did you find a solution @michael_mehrten ?
In my case Im using a managed identity so the solution some topics suggest on generating an access token from a Entra ID service principal is not applicable.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group