โ09-24-2024 05:34 AM
Need help on databricks workflow if else task.
Task-A : Notebook : executing set of the pyspark commands and loading the data into delta table.
I need to setup a custom status so that i can use status in the next if/else task.
Task-b: if/else: once i get the status of the previous task based on the status i need execute the set of notebook on True condition else stop the workflow.
need help on how to set and get the status of the previous notebook to current if/else task.
@databricksworkflow
#
โ09-24-2024 06:35 AM - edited โ09-24-2024 06:50 AM
You have error in your code. To set a task value you should use dbutils.job.taskValues.set command. Instead, in code you've provided you're using dbutils.job.taskValues.get.
Look at below picture. Here I have sample notebook where I'm setting status to "success"
Then I created sample workflow:
In if/else task I'm reffering to task value in following way:
โ09-24-2024 05:53 AM
Hi @suryateja405555 ,
You can use task values in Task A to setup a custom status and then use that status in the if/else task to decide if you need to execute the of notebook on True condition or stop the workflow.
https://docs.databricks.com/en/jobs/share-task-context.html#using-task-values
โ09-24-2024 06:15 AM
but what is if/else expression we need to write to get the values from the previous task,
i tried with "tasks.previous_task_name.values.value_of_previous_task" =="SUCCESS" in the expression. throwing an error task value referenced by tasks.<previous_task>.values.value was not found.
i tried to declare/set a custom status by using these approaches.
1) dbutils.notebook.exit({status:"success"}) 2) dbutils.job.taskValues.get(key="status",values ="success")
Need your help to write if/else expression to get the value.
Thanks !!!
โ09-24-2024 06:35 AM - edited โ09-24-2024 06:50 AM
You have error in your code. To set a task value you should use dbutils.job.taskValues.set command. Instead, in code you've provided you're using dbutils.job.taskValues.get.
Look at below picture. Here I have sample notebook where I'm setting status to "success"
Then I created sample workflow:
In if/else task I'm reffering to task value in following way:
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group