sarahbhord
Databricks Employee
Databricks Employee

‎09-25-2025 10:36 AM

Hey PaulChen

Having your Databricks service principal (SP) work locally but fail in Bitbucket CI/CD usually means environment variables aren’t set up right, or the pipeline is falling back to an unexpected config.
 
Quick checklist:
  • Set all SP credentials (DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET) as Bitbucket repo/project variables—don’t hard-code.
  • Ensure the pipeline script actually exports those vars before running Databricks CLI/DAB.
  • Make sure your pipeline is referencing the right profile/environment and not a user config by mistake.
  • Confirm your service principal has all the permissions it needs in Databricks (deployments, workflow runs, etc.).
  • Match the Databricks CLI version in Bitbucket to what you use locally.
If you’re referencing a .databrickscfg file in CI, build it from env vars during your pipeline for security. Check error messages—missing or mistyped env var names are common causes. If you stay stuck, echo a redacted config dump in CI to compare with local.
 
Let me know if this was helpful! 
 
Sarah
 

 

View solution in original post