Re: Running jobs as service principal, while pulli...

saurabh18cs · ‎07-17-2025

can you try generating oauth databricks token for this sp and then pass this token to your databricks bundle deploy as env variables section instead of client id and secret.

Add following to your parameters or your preferred choice of deployment

- name: sp_app_id_dev

displayName: Service Principal App ID DEV (for oauth token)

type: string

default: ""

- name: sp_app_id_acc

displayName: Service Principal App ID ACC (for oauth token)

type: string

default: ""

- name: sp_app_id_prd

displayName: Service Principal App ID PRD (for oauth token)

type: string

default: ""

##############################################################

Add this job as first job:

######################

- job : oauth_bearer_token_sp

steps:

- script: |

wget https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux32 -O $(Build.Repository.LocalPath)/jq

chmod +x $(Build.Repository.LocalPath)/jq

displayName: Install jq

condition: succeeded()

- script: |

if [[ ${{ variables.env}} -eq 'dev' ]]

then

CLIENT_ID=${{ parameters.sp_app_id_dev}}

CLIENT_SECRET=$SP_SECRET_DEV

DATABRICKS_WORKSPACE_URL=${{ parameters.databricks_wrkspc_url_dev}}

elif [[ ${{ variables.env}} -eq 'acc' ]]

then

CLIENT_ID=${{ parameters.sp_app_id_acc}}

CLIENT_SECRET=$SP_SECRET_ACC

DATABRICKS_WORKSPACE_URL=${{ parameters.databricks_wrkspc_url_acc}}

else

CLIENT_ID=${{ parameters.sp_app_id_prd}}

CLIENT_SECRET=$SP_SECRET_PRD

DATABRICKS_WORKSPACE_URL=${{ parameters.databricks_wrkspc_url_prd}}

fi

DATABRICKS_URL="$DATABRICKS_WORKSPACE_URL/api/2.0/token/create"

access_token_val=$(curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \

https://login.microsoftonline.com/af73baa8-f594-4eb2-a39d-93e96cad61fc/oauth2/v2.0/token \

-d "client_id=$CLIENT_ID" \

-d 'grant_type=client_credentials'\

-d 'scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d%2F.default' \

-d "client_secret=$CLIENT_SECRET")

access_token=$(jq -r '.access_token' <<< "$access_token_val")

echo $access_token

api_response=$(curl -X POST $DATABRICKS_URL \

-H "Authorization: Bearer $access_token" \

-H "X-Databricks-Azure-SP-Management-Token:$access_token" \

-d '{"comment": "pipeline token"}')

echo "$api_response"

DATABRICKS_NEW_TOKEN=$(jq -r '.token_value' <<< "$api_response")

if [ -z "${DATABRICKS_NEW_TOKEN}" ]

then

echo "Token could not be created"

exit 1

else

echo "Successfully created a Databricks Token"

echo "##vso[task.setvariable variable=DATABRICKS_TOKEN;isOutput=true]$DATABRICKS_NEW_TOKEN"

echo "##vso[task.setvariable variable=ACCESS_TOKEN;isOutput=true]$access_token"

fi

displayName: 'Create oauth token'

name: oauth

condition: succeeded()

####################

pass this DATABRICKS_TOKEN to next stage or job as variables

variables:

DATABRICKS_TOKEN: $[ Dependencies.oauth_bearer_token_sp.outputs['oauth.DATABRICKS_TOKEN'] ]

###############

use this DATABRICKS_TOKEN as env for aset bundle deploy script