Service Principal for remote repository in workflow/job expiring token

Administration & Architecture

Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.

I would like to create a databricks Job where the 'Run as' field is set to a ServicePrincipal. The Job points to notebooks stored in Azure DevOps.

The step I've already performed are:

I created the Service Principal and I'm now able to see it into the "Run as" dropdown menù of the Job

1) I created the ADO PAT (DevOps personal access token)

2) I created a AAD token for ServicePrincipal

3) Used the generated AAD token to set ADO PAT using Git Credentials API via git credential API (doc here)

When I try to setup the Git Credential for the ServicePrincipal I need to insert the DevOps PAT which has a temporary validity.

If the DevOps PAT is not expired everything works fine but when the PAT expires I'm getting the error "Failed to checkout Git repository: PERMISSION_DENIED: Invalid Git provider credentials"

Is it possible to run the Job via ServicePrincipal without using any expiring token?

Thanks