cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Authenticating to Accounts Console Using Client ID and Secret via Terraform and Databricks CLI

jv_v
New Contributor III

I am currently working on a project where I need to authenticate to the Databricks accounts console from Terraform using a client ID and client secret. Here is the relevant portion of my Terraform configuration:

// Provider for Databricks account
provider "databricks" {
alias = "azure_account"
host = "https://accounts.azuredatabricks.net"
account_id = var.account_id
azure_client_id = var.client_id
azure_client_secret = var.client_secret
}
I execute all the Terraform tasks from a shell script. Within this script, we use a Databricks CLI command to check whether the workspace is assigned to a metastore:

sh
databricks auth login --host https://accounts.azuredatabricks.net --account-id "$account_id"

 assignment=$(databricks account metastore-assignments get "$workspace_id" 2>/dev/null)

However, I am unable to authenticate since the command attempts to authenticate using a client ID and client secret.

Is there a way to authenticate using the client ID and client secret directly with the Databricks CLI? If not, can you suggest any alternative methods to achieve this authentication?

jv_v_0-1718116490905.png

 

3 REPLIES 3

jw-dbx
Databricks Employee
Databricks Employee

Hi, 

You can follow instructions in the following doc to configure OAuth machine-to-machine authentication using Service Principal. 

https://docs.databricks.com/en/dev-tools/cli/authentication.html#oauth-machine-to-machine-m2m-authen... 

Once it is configured properly, you should be able to run Databricks CLI, then run following command to confirm metastore assignment for a given workspace:

databricks account metastore-assignments get <<WORKSPACE_ID>>

 

jv_v
New Contributor III

Hi

I am attempting to complete OAuth M2M (Machine-to-Machine) authentication as advised, but I keep encountering the following error:

 

{
    "error": "invalid_request",
    "error_id": "*****************",
    "error_description": "Invalid request"
}

 

Here is a summary of my current setup and the steps I have taken:

 

Script used to generate the tocken

 

export CLIENT_ID=<client id>
export CLIENT_SECRET=clent secret

curl --request POST \
         --url 'https://accounts.azuredatabricks.net/oidc/accounts/<accoutnt id>/v1/token \'
         --user "$CLIENT...

jw-dbx
Databricks Employee
Databricks Employee

First, run following commands in shell, please replace placeholder according to your environment:

export CLIENT_ID=<client id>
export CLIENT_SECRET=<client secret>
export TOKEN_EP=https://accounts.cloud.databricks.com/oidc/accounts/<databricks account id>/v1/token

 then run this command to generate token:

curl --request POST --url $TOKEN_EP --user "$CLIENT_ID:$CLIENT_SECRET" --data 'grant_type=client_credentials&scope=all-apis'

 Please make sure you follow following instructions to create client id/secret for your service principal, it should be created at account level instead of workspace level.

https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html#step-3-create-an-oauth-secret-for-a-ser... 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group