Authenticating to Accounts Console Using Client ID and Secret via Terraform and Databricks CLI

jv_v
Databricks Partner

‎06-11-2024 07:35 AM

I am currently working on a project where I need to authenticate to the Databricks accounts console from Terraform using a client ID and client secret. Here is the relevant portion of my Terraform configuration:

// Provider for Databricks account
provider "databricks" {
alias = "azure_account"
host = "https://accounts.azuredatabricks.net"
account_id = var.account_id
azure_client_id = var.client_id
azure_client_secret = var.client_secret
}
I execute all the Terraform tasks from a shell script. Within this script, we use a Databricks CLI command to check whether the workspace is assigned to a metastore:

sh
databricks auth login --host https://accounts.azuredatabricks.net --account-id "$account_id"

 assignment=$(databricks account metastore-assignments get "$workspace_id" 2>/dev/null)

However, I am unable to authenticate since the command attempts to authenticate using a client ID and client secret.

Is there a way to authenticate using the client ID and client secret directly with the Databricks CLI? If not, can you suggest any alternative methods to achieve this authentication?

jv_v_0-1718116490905.png

 

0 Kudos

JianWu
Databricks Employee
Databricks Employee

‎06-11-2024 01:42 PM

Hi, 

You can follow instructions in the following doc to configure OAuth machine-to-machine authentication using Service Principal. 

https://docs.databricks.com/en/dev-tools/cli/authentication.html#oauth-machine-to-machine-m2m-authen... 

Once it is configured properly, you should be able to run Databricks CLI, then run following command to confirm metastore assignment for a given workspace:

databricks account metastore-assignments get <<WORKSPACE_ID>>

 

0 Kudos

jv_v
Databricks Partner
In response to JianWu

‎06-18-2024 06:33 AM

Hi

I am attempting to complete OAuth M2M (Machine-to-Machine) authentication as advised, but I keep encountering the following error:

 

{
    "error": "invalid_request",
    "error_id": "*****************",
    "error_description": "Invalid request"
}

 

Here is a summary of my current setup and the steps I have taken:

 

Script used to generate the tocken

 

export CLIENT_ID=<client id>
export CLIENT_SECRET=clent secret

curl --request POST \
         --url 'https://accounts.azuredatabricks.net/oidc/accounts/<accoutnt id>/v1/token \'
         --user "$CLIENT...

0 Kudos

JianWu
Databricks Employee
Databricks Employee

‎06-18-2024 09:15 AM

First, run following commands in shell, please replace placeholder according to your environment:

export CLIENT_ID=<client id>
export CLIENT_SECRET=<client secret>
export TOKEN_EP=https://accounts.cloud.databricks.com/oidc/accounts/<databricks account id>/v1/token

 then run this command to generate token:

curl --request POST --url $TOKEN_EP --user "$CLIENT_ID:$CLIENT_SECRET" --data 'grant_type=client_credentials&scope=all-apis'

 Please make sure you follow following instructions to create client id/secret for your service principal, it should be created at account level instead of workspace level.

https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html#step-3-create-an-oauth-secret-for-a-ser... 

0 Kudos