Cannot deploy DAB with the Job branch using a feature branch in Workspace UI

[SangNguyen]

Hi, 

I tried to deploy DAB on Workspace UI with a feature branch (sf-trans-seq) targeted to Dev. After deploying successfully, the Job branch is, however, using the master branch (see the screenshot below).

Is there any option to force the Job branch to use the feature branch (e.g., sf-trans-seq) similar to the git-branch option when deploy using Databricks CLI in VS Code? 

databricks bundle deploy -var="git-branch=sf-trans-seq" -t dev

Thanks, 

 

[-werners-]

Is your job definition added to the feature branch too?
It seems like it is not the case.

[SangNguyen]

Hi Weerners, 

Yes, it is as you can see it there (job branch) when I deployed DAB using Databricks CLI command. 

Within the Workspace UI, I saw the Bundle branch is referred to the feature branch, but the Job branch is still referred to the master branch. Is there any option to select the feature branch for the Job branch in Workspace UI?

[-werners-]

It seems that the job was not defined within the bundle but separately (either via cli or UI).
In that case it makes sense.
When creating a bundle, I'd put everything related in that bundle (notebooks, py files, jobs) and check that into git.
Basically there should not be a separate job and bundle branch.
I don't know if that makes sense?

If you use the UI you can always show the generated yml like this (Open configuration):

[Coffee77]

Deploying with DAB is completely independent of branch you are using. I usually deploy in my DEV environment with features or DEV branches and everything is OK. I'd say you deployed previous job from master branch and then, you are not updating those jobs but creating new ones (or something similar). So, remember that you must assign same identifiers to deployed resources, by assigning only same names is not OK as new jobs will be created instead of "updating" those with same names. I hope this cloud help.

[SangNguyen]

Hi @Coffee77, 

Sorry as I uploaded the wrong screenshot. I used the same feature branch for Dev, so the correct screenshot is below: 

databricks bundle deploy -var="git-branch=sf-trans-seq" -t dev

      

@-werners- 

I have configured git-branch by default is master, in the dbx.vars.global.yml configuration file.

    

When deploy DAB with Databricks CLI, the job branch (master by default) can be overwritten by this option: -var="git-branch=sf-trans-seq"

However, when deploy DAB in DB Workspace UI, even though I selected the git-branch as the feature branch, but the job branch is still showing the master branch 

I am expecting the Workspace UI can also use the git branch for the Job branch as deploying in VS Code, instead of the master by default in my configuration.

Thanks,

[-werners-]

Ok i get it now.
I still find it strange to have a separate branch for the job.
To be honest I work in a complete different way with dab and git.

In my asset bundles I do not define any git branch.
I use git for version control and pull requests of the file objects themselves.
These PRs (and commits) launch devops pipelines which run the DAB deploy & validate.
Works flawless.

[SangNguyen · Accepted solution]

I found how to add the "git-branch" to override in Workspace UI. 

From Deploy --> Configure variable overrides --> enter the {"git-branch":"sf-trans-seq"} to the variable-overrides.json file.

After Deploy: 

I think it would be better if the Workspace UI can have the Job branch to automatically get the value from the Bundle branch without modifying the variable-overrides.json file.

[-werners-]

I agree.

Can you mark your (or someone else´s) answer as solved?  Because I think you won´t be the only one with this issue/feature.