cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks Workflow if else task type

suryateja405555
New Contributor II

Need help on databricks workflow if else task. 

Task-A : Notebook : executing set of the pyspark commands and loading the data into delta table.

I need to setup a custom status so that i can use status in the next if/else task.

Task-b: if/else: once i get the status of the previous task based on the status i need execute the set of notebook on True condition else stop the workflow.

need help on how to set and get the status of the previous notebook to current if/else task.

@databricksworkflow

#

1 ACCEPTED SOLUTION

Accepted Solutions

You have error in your code. To set a task value you should use dbutils.job.taskValues.set command. Instead, in  code you've provided you're using dbutils.job.taskValues.get.
Look at below picture. Here I have sample notebook where I'm setting status to "success"

szymon_dybczak_2-1727184822477.png

Then I created sample workflow:

szymon_dybczak_0-1727184733859.png

In if/else task I'm reffering to task value in following way:

szymon_dybczak_1-1727184765037.png

 

 

View solution in original post

3 REPLIES 3

szymon_dybczak
Contributor III

Hi @suryateja405555 ,

You can use task values in Task A to setup a custom status and then use that status in the if/else task to decide if you need to execute the of notebook on True condition or stop the workflow.

https://docs.databricks.com/en/jobs/share-task-context.html#using-task-values

but what is if/else expression we need to write to get the values from the previous task,

i tried with "tasks.previous_task_name.values.value_of_previous_task" =="SUCCESS" in the expression. throwing an error task value referenced by  tasks.<previous_task>.values.value was not found.

 

i tried to declare/set a custom status by using these approaches.

1) dbutils.notebook.exit({status:"success"})  2) dbutils.job.taskValues.get(key="status",values ="success")

Need your help to write if/else expression to get the value. 

Thanks !!!

You have error in your code. To set a task value you should use dbutils.job.taskValues.set command. Instead, in  code you've provided you're using dbutils.job.taskValues.get.
Look at below picture. Here I have sample notebook where I'm setting status to "success"

szymon_dybczak_2-1727184822477.png

Then I created sample workflow:

szymon_dybczak_0-1727184733859.png

In if/else task I'm reffering to task value in following way:

szymon_dybczak_1-1727184765037.png

 

 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group