I am trying to orchestrate my Databricks Workflows tasks using a parameter I would set in a notebook.
Given the workflow below I am trying to set a parameter in the Cinderella task which is a python notebook. Once set I would like to use this parameter in the true_false_selection block instead of this 1=1 condition. Something like:
if ${notebook_parameter} == "Rolling" --> execution continues to the isTrue task which is also a python notebook
elif ${notebook_parameter} == "Prophet" --> execution continues to the isNotTrue task which is another python notebook.
Precisely, I want to use parameter set inside notebook in this menu selection as the pseudo code shows:
I am aware of both
dbutils.jobs.taskValues.set(key='selection', value='Rolling')
&&
dbutils.widgets.text("selection", "Rolling")
but it appears neither can be used in the if/else condition task.
Here's how we can proceed, follow the instructions below:
In your previous task, depending on whether you're using Python or Scala, set the task value like this:
dbutils.jobs.taskValues.set("check_value", "2")
In your if-else task, you must reference the name of the previous task as follows:
{{tasks.['previous_task'].values.['check_value']}}
Please review the attached screenshot for further clarification
