cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks Asset Bundle Workflow Redeployment Issue

Sikki
New Contributor III

Hello All,

In my Databricks workflows, I have three tasks configured, with the final task set to run only if the condition "ALL_DONE" is met. During the first deployment, I observed that the dependency "ALL_DONE" was correctly assigned to the last task. However, upon subsequent redeployments, I noticed that "ALL_DONE" was also being assigned to other tasks. Upon further investigation, it appears that the naming of the task keys may be causing this discrepancy. This hypothesis was confirmed when I tested with different task key names, such as task-A, task-B, and task-C, and found that the workflow configuration was correctly maintained during redeployment as well.
Thanks,Sikki

8 REPLIES 8

Sikki
New Contributor III

Hi, I've attached two snapshots, one from the initial deployment and another from the second redeployment, where no changes were made to the code.

Yeshwanth
Honored Contributor

Hi @Sikki Good day!

There was an issue and it was fixed recently. Could you please confirm if you are still facing the issue?

Best regards,

Sikki
New Contributor III

The issue hasn't been resolved yet. Should I update my Databricks CLI?

Sikki
New Contributor III

The issue hasn't been resolved yet. Should I update my Databricks CLI to generate new bundle_config_schema.json.

 

Yeshwanth
Honored Contributor

@Sikki are you using the latest version of the CLI? v.28?

Sikki
New Contributor III

Hi @Yeshwanth I am using latest CLI version and using Azure DevOps to deploy the workflow.

Sikki
New Contributor III

After updating my CLI, I successfully deployed the job from Databricks CLI and it is functioning correctly. However, when attempting to deploy the same job using Azure DevOps, I encounter the same issue.

Yeshwanth
Honored Contributor

@Sikki thanks for confirming.

For Azure DevOps please check the version of Databricks CLI installed.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group