Databricks Asset Bundles
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-06-2024 03:00 AM
Hi, I'm implementing Databricks Asset bundles, my scripts are in GitHub and my /resource has all the .yml of my Databricks workflow which are pointing to the main branch
git_source:
git_url: https://github.com/xxxx
git_provider: gitHubEnterprise
git_branch: main
Now when I work with DABs, I create a feature branch and when I deploy and run my bundle, I want my databricks workflows to point to my feature branch, is there a way to dynamically change my git_branch to my feature branch when I deploy my bundle and run it.
Also, can we create a script to /resource/xx.yml mapping, the reason is that I don't want all my jobs to run when I run my bundle for a script change that is part of a single job
@Retired_mod Appreciate your input here
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-17-2024 07:52 AM
Why not use Substitutions and Custom variables that can be specified on command line using --var="<key>=<value>"?
With all the features your databricks.yml would look as follows:
variables:
git_branch:
default: main
git_source:
git_url: https://github.com/xxxx
git_provider: gitHubEnterprise
git_branch: ${var.git_branch}
On command line you'd databricks deploy as follows:
databricks bundle deploy \
--var="git_branch=features/my_great_feature"
Think of Databricks Asset Bundles as a tool to prepopulate job/workflow definition templates (based on databricks.yml, command line, env vars) so once the assets are deployed, they are going to be executed / run with the values (unless overridden by asset-specific means, like Run now with different parameters in Jobs).

