Oauth Token federation

[noorbasha534]

Dear all

Has anyone tried oauth token federation for authentication with Databricks REST APIs?

appreciate if there is a re-usable code snippet to achieve the same.

[noorbasha534]

Documentation says - '''An account federation policy enables all users and service principals in your Azure Databricks account to access Databricks APIs using tokens from your identity provider. '''' But, in the examples that are provided, I do not see how the subject claim should be for a service principal. For an interactive user, it seems it could be user@mycompany.com

issuer: "https://idp.mycompany.com/oidc"
audiences: ["2ff814a6-3304-4ab8-85cb-cd0e6f879c1d"]
subject_claim: "preferred_username"

[jack_zaldivar]

hi @noorbasha534 , can you pleas clarify your request a bit more? What exactly are you wanting to do/accomplish?

Happy to help, if I'm able to, or pull in other resources if it's beyond my personal skillset!

[saurabh18cs]

Hi @noorbasha534 

I used to generate oauth token for databricks for my service principal using azure devops pipeline in this manner, see if this token can help you (DATABRICKS_TOKEN) which you can then use in next stages. I generate this token on sp to authenticate towards databricks so when i deploy job it should use this sp identity and not the one running on agent:

 - stage : Create_oauth_token

    condition : succeeded()

    jobs :

      - job : oauth_bearer_token_sp

        steps:

          - script: |

              wget https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux32 -O $(Build.Repository.LocalPath)/jq

              chmod +x $(Build.Repository.LocalPath)/jq

            displayName: Install jq

            condition: succeeded()

          - script: |

              if [[ ${{ variables.env}} -eq 'dev' ]]

              then

                CLIENT_ID=${{ parameters.sp_app_id_dev}}

                CLIENT_SECRET=$SP_SECRET_DEV

                DATABRICKS_WORKSPACE_URL=${{ parameters.databricks_wrkspc_url_dev}}

              elif [[ ${{ variables.env}} -eq 'acc' ]]

              then

                CLIENT_ID=${{ parameters.sp_app_id_acc}}

                CLIENT_SECRET=$SP_SECRET_ACC

                DATABRICKS_WORKSPACE_URL=${{ parameters.databricks_wrkspc_url_acc}}

              else

                CLIENT_ID=${{ parameters.sp_app_id_prd}}

                CLIENT_SECRET=$SP_SECRET_PRD

                DATABRICKS_WORKSPACE_URL=${{ parameters.databricks_wrkspc_url_prd}}

              fi

              DATABRICKS_URL="$DATABRICKS_WORKSPACE_URL/api/2.0/token/create"

              access_token_val=$(curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \

                             https://login.microsoftonline.com/af73baa8-f5xxxxxxxxxxxxxxxxxxx/oauth2/v2.0/token \

                             -d "client_id=$CLIENT_ID" \

                             -d 'grant_type=client_credentials'\

                             -d 'scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d%2F.default' \

                             -d "client_secret=$CLIENT_SECRET")

              access_token=$(jq -r '.access_token' <<< "$access_token_val")

              echo $access_token

              api_response=$(curl -X POST $DATABRICKS_URL \

                            -H "Authorization: Bearer $access_token" \

                            -H "X-Databricks-Azure-SP-Management-Token:$access_token" \

                            -d '{"comment": "pipeline token"}')

              echo "$api_response"

              DATABRICKS_NEW_TOKEN=$(jq -r '.token_value' <<< "$api_response")

              if [ -z "${DATABRICKS_NEW_TOKEN}" ]

              then

                echo "Token could not be created"

                exit 1

              else

                echo "Successfully created a Databricks Token"

                echo "##vso[task.setvariable variable=DATABRICKS_TOKEN;isOutput=true]$DATABRICKS_NEW_TOKEN"

                echo "##vso[task.setvariable variable=ACCESS_TOKEN;isOutput=true]$access_token"

              fi

            displayName: 'Create oauth token'

            name: oauth

            condition: succeeded()

[nayan_wylde]

@noorbasha534  Here is a sample python code I use for getting oauth token from Azure Active Directory and then pass the token in databricks API. Prerequisite is the SPN needs to be a admin in the workspace.

import requests

# Azure AD credentials
tenant_id = 'your-tenant-id'
client_id = 'your-client-id'
client_secret = 'your-client-secret'

# Databricks workspace URL
databricks_instance = 'https://<your-databricks-instance>.azuredatabricks.net'

# Step 1: Get OAuth token from Azure AD
def get_aad_token(tenant_id, client_id, client_secret):
    url = f"https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token"
    payload = {
        'grant_type': 'client_credentials',
        'client_id': client_id,
        'client_secret': client_secret
    }
    response = requests.post(url, data=payload)
    response.raise_for_status()
    return response.json()['access_token']

# Step 2: Use token to call Databricks API
def call_databricks_api(token, endpoint='/api/2.0/clusters/list'):
    headers = {
        'Authorization': f'Bearer {token}'
    }
    url = f"{databricks_instance}{endpoint}"
    response = requests.get(url, headers=headers)
    response.raise_for_status()
    return response.json()

# Example usage
token = get_aad_token(tenant_id, client_id, client_secret)
result = call_databricks_api(token)
print(result)