cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks Asset Bundle Workflow Redeployment Issue

Sikki
New Contributor III

Hello All,

In my Databricks workflows, I have three tasks configured, with the final task set to run only if the condition "ALL_DONE" is met. During the first deployment, I observed that the dependency "ALL_DONE" was correctly assigned to the last task. However, upon subsequent redeployments, I noticed that "ALL_DONE" was also being assigned to other tasks. Upon further investigation, it appears that the naming of the task keys may be causing this discrepancy. This hypothesis was confirmed when I tested with different task key names, such as task-A, task-B, and task-C, and found that the workflow configuration was correctly maintained during redeployment as well.
Thanks,Sikki

8 REPLIES 8

Sikki
New Contributor III

Hi, I've attached two snapshots, one from the initial deployment and another from the second redeployment, where no changes were made to the code.

Yeshwanth
Honored Contributor
Honored Contributor

Hi @Sikki Good day!

There was an issue and it was fixed recently. Could you please confirm if you are still facing the issue?

Best regards,

Sikki
New Contributor III

The issue hasn't been resolved yet. Should I update my Databricks CLI?

Sikki
New Contributor III

The issue hasn't been resolved yet. Should I update my Databricks CLI to generate new bundle_config_schema.json.

 

Yeshwanth
Honored Contributor
Honored Contributor

@Sikki are you using the latest version of the CLI? v.28?

Sikki
New Contributor III

Hi @Yeshwanth I am using latest CLI version and using Azure DevOps to deploy the workflow.

Sikki
New Contributor III

After updating my CLI, I successfully deployed the job from Databricks CLI and it is functioning correctly. However, when attempting to deploy the same job using Azure DevOps, I encounter the same issue.

Yeshwanth
Honored Contributor
Honored Contributor

@Sikki thanks for confirming.

For Azure DevOps please check the version of Databricks CLI installed.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!