Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-08-2025 10:23 AM
The Databricks Asset Bundle is used for the Bitbucket CICD pipelines.
The service principle is used in both of the local Databricks configuration and Bitbucket CICD environment.
The service principle is only working in the local environment for deployment, but not in Bitbucket CICD environment.
The SP config similar to :
[DEFAULT]
host = <host>
account_id = <account_id>
client_id = <client_id>
client_secret = <client_secret>
If I use personal user configuration in local and Bitbucket CICD environment, the DAB worked fine in both envs without any issue.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-25-2025 10:36 AM
Hey PaulChen
Having your Databricks service principal (SP) work locally but fail in Bitbucket CI/CD usually means environment variables aren’t set up right, or the pipeline is falling back to an unexpected config.
Quick checklist:
- Set all SP credentials (
DATABRICKS_HOST,DATABRICKS_ACCOUNT_ID,DATABRICKS_CLIENT_ID,DATABRICKS_CLIENT_SECRET) as Bitbucket repo/project variables—don’t hard-code. - Ensure the pipeline script actually exports those vars before running Databricks CLI/DAB.
- Make sure your pipeline is referencing the right profile/environment and not a user config by mistake.
- Confirm your service principal has all the permissions it needs in Databricks (deployments, workflow runs, etc.).
- Match the Databricks CLI version in Bitbucket to what you use locally.
If you’re referencing a
.databrickscfg file in CI, build it from env vars during your pipeline for security. Check error messages—missing or mistyped env var names are common causes. If you stay stuck, echo a redacted config dump in CI to compare with local.Let me know if this was helpful!
Sarah
