Databricks Community

JacekJacek · ‎02-03-2025

I'm trying to deploy asset bundle from a CI/CD pipeline, I'd like to use the docker databricks CLI image for that, but I can't get it to authenticate. I'm using entra service principal for my deployments and we are using TeamCity as our CI/CD tool. The cli image does not come with az cli preinstalled, so that auth is not possible, my organization has disabled personal access tokens as well, so I'm trying to use oauth m2m, but can't get it to work. I keep getting an error:

08:22:45  08:22:45 INFO start pid=1 version=0.238.0 args="/app/databricks, bundle, validate, -t, prototype-dev, -p, DEFAULT, --log-level=debug"
08:22:45  08:22:45 DEBUG Found bundle root at /my-bundle (file /my-bundle/databricks.yml) pid=1
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load
08:22:45  08:22:45 INFO Phase: load pid=1 mutator=load
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=EntryPoint
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=scripts.preinit
08:22:45  08:22:45 DEBUG No script defined for preinit, skipping pid=1 mutator=load mutator=seq mutator=scripts.preinit
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=ProcessRootIncludes
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=ProcessRootIncludes mutator=seq
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=ProcessRootIncludes mutator=seq mutator=ProcessInclude(bundle/targets/prototype-dde.yml)
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=ProcessRootIncludes mutator=seq mutator=ProcessInclude(bundle/targets/prototype-uat.yml)
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=ProcessRootIncludes mutator=seq mutator=ProcessInclude(bundle/targets/prototype-prd.yml)
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=ProcessRootIncludes mutator=seq mutator=ProcessInclude(bundle/targets/prototype-dev.yml)
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=ProcessRootIncludes mutator=seq mutator=ProcessInclude(bundle/resources/source_types/file/pipelines/file_ingestion_pipeline.yml)
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=ProcessRootIncludes mutator=seq mutator=ProcessInclude(bundle/resources/source_types/file/jobs/configure_file_sources.yml)
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=ProcessRootIncludes mutator=seq mutator=ProcessInclude(bundle/resources/source_types/file/jobs/file_ingestion_job.yml)
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=ProcessRootIncludes mutator=seq mutator=ProcessInclude(bundle/resources/source_types/file/variables.common.yml)
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=VerifyCliVersion
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=EnvironmentsToTargets
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=ComputeIdToClusterId
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=InitializeVariables
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=DefineDefaultTarget(default)
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=PythonMutator(load)
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=validate:unique_resource_keys
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=load mutator=seq mutator=SelectTarget(prototype-dev)
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=<func>
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=<func>
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize
08:22:45  08:22:45 INFO Phase: initialize pid=1 mutator=initialize
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize mutator=seq
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize mutator=seq mutator=validate:AllResourcesHaveValues
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize mutator=seq mutator=RewriteSyncPaths
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize mutator=seq mutator=SyncDefaultPath
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize mutator=seq mutator=SyncInferRoot
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize mutator=seq mutator=MergeJobClusters
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize mutator=seq mutator=MergeJobParameters
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize mutator=seq mutator=MergeJobTasks
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize mutator=seq mutator=MergePipelineClusters
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize mutator=seq mutator=InitializeWorkspaceClient
08:22:45  08:22:45 DEBUG Loading DEFAULT profile from /root/.databrickscfg pid=1 sdk=true
08:22:45  08:22:45 DEBUG Apply pid=1 mutator=initialize mutator=seq mutator=PopulateCurrentUser
08:22:45  08:22:45 DEBUG Loading DEFAULT profile from /root/.databrickscfg pid=1 sdk=true
08:22:45  08:22:45 INFO Ignoring pat auth, because databricks-cli is preferred pid=1 sdk=true
08:22:45  08:22:45 INFO Ignoring basic auth, because databricks-cli is preferred pid=1 sdk=true
08:22:45  08:22:45 INFO Ignoring oauth-m2m auth, because databricks-cli is preferred pid=1 sdk=true
08:22:45  08:22:45 DEBUG Running command: /app/databricks auth token --host https://adb-2355869874698299.19.azuredatabricks.net pid=1 sdk=true
08:22:45  Error: failed during request visitor: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. Config: host=https://adb-xxx.azuredatabricks.net, profile=DEFAULT, azure_client_secret=***, azure_client_id=xxx, azure_tenant_id=xxx, client_id=xxx, client_secret=***, databricks_cli_path=/app/databricks. Env: ARM_CLIENT_SECRET, ARM_CLIENT_ID, ARM_TENANT_ID, DATABRICKS_CLI_PATH
08:22:45 
08:22:45  Name: xxx
08:22:45  Target: prototype-dev
08:22:45  Workspace:
08:22:45  Host: https://adb-xxx.azuredatabricks.net/
08:22:45 
08:22:45  Found 1 error

I'm invoking the cli this way:

docker run \
-v %teamcity.build.checkoutDir%:/my-bundle \
-v %teamcity.build.checkoutDir%/.databrickscfg:/root/.databrickscfg \
-w /my-bundle \
-e NO_PROXY=$NO_PROXY \
-e ARM_CLIENT_SECRET="%env.TF_VAR_client_secret%" \
-e ARM_CLIENT_ID="%env.TF_VAR_client_id%" \
-e ARM_TENANT_ID="%env.TF_VAR_tenant_id%" \
-e DATABRICKS_AUTH_TYPE="oauth-m2m" \
%docker.repository%/databricks/cli:0.238.0 bundle validate -t prototype-dev -p DEFAULT --log-level=debug

The .databrickscfg contents:

[DEFAULT]
host = https://adb-xxx.azuredatabricks.net/
client_id = %env.TF_VAR_client_id%
client_secret = %databricks.token%
auth_type = oauth-m2m

Tried numerous combinations of config/env but nothing seems to work.. do I need a custom image with both databricks cli and az cli?

JacekJacek · ‎02-04-2025

OK, tested and now everything is working - according to the docs bundle settings are of highest priority, https://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/authentication#auth-eval

No wonder none of my env vars or .databrickscfg settings worked. Oh well..

View solution in original post

JacekJacek · ‎02-03-2025

The above is the output of bundle validate command, so before we even run deployment, but that would fail the same way ofc.

aj_dbx_ · ‎02-03-2025

Hello @JacekJacek
Have you tried to follow dbx docs: https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html#step-4-use-oauth-m2m-authentication
?

BR,
aj

MariuszK · ‎02-03-2025

Hi,

I think you should use these environment variables:DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET, DATABRICKS_ACCOUNT_ID, DATABRICKS_HOST

JacekJacek · ‎02-03-2025

account_id is for authenticating to the accounts.azuredatabricks.net - I'm deploying a bundle to the workspace, but I just found that there was auth_type added in the bundle and that trumped all other methods / settings (strange), so now I've removed it and will give it another go...