Service Principal for remote repository in workflow/job expiring token

[diego_poggioli]

I would like to create a databricks Job where the 'Run as' field is set to a ServicePrincipal. The Job points to notebooks stored in Azure DevOps.

The step I've already performed are:

I created the Service Principal and I'm now able to see it into the "Run as" dropdown menù of the Job

1) I created the ADO PAT (DevOps personal access token)

2) I created a AAD token for ServicePrincipal

3) Used the generated AAD token to set ADO PAT using Git Credentials API via git credential API (doc here)

When I try to setup the Git Credential for the ServicePrincipal I need to insert the DevOps PAT which has a temporary validity.

If the DevOps PAT is not expired everything works fine but when the PAT expires I'm getting the error "Failed to checkout Git repository: PERMISSION_DENIED: Invalid Git provider credentials"

Is it possible to run the Job via ServicePrincipal without using any expiring token?

Thanks

[Kaniz · Accepted solution]

Hi @diego_poggioli, Unfortunately, there is no direct way to bypass the use of expiring tokens when accessing Azure DevOps. The Azure DevOps PAT is used as a security measure to ensure that only authorized users can access the resources, and it is designed to expire after a certain period for security reasons. 

However, you can automate refreshing the PAT before it expires.

This way, you can ensure that the Databricks job always has a valid PAT. Here are the steps you could take. Automate creating a new Azure DevOps PAT before the current one expires.

2. Use the Azure Active Directory (AAD) token for the Service Principal to set the newly generated Azure DevOps PAT using the Git Credentials API. Please note that you must handle the automation part programmatically, as there is no built-in feature in Azure DevOps to auto-refresh PATs.