DAB for_each_task - Passing task values

DarioB · ‎06-05-2025

I am trying to deploy a job with a for_each_task using DAB and Terraform and I am unable to properly pass the task value into the subsequent task.

These are my job tasks definition in the YAML:

tasks:

- task_key: FS_batching

job_cluster_key: job_cluster_fs

notebook_task:

notebook_path: ...

- task_key: FS_processing

depends_on:

- task_key: FS_batching

for_each_task:

inputs: {{tasks.FS_batching.values.var_name}}

task:

task_key: processing_iteration

job_cluster_key: job_cluster_fs

notebook_task:

notebook_path: ....

base_parameters:

input: {{input}}

For feeding the parameter into inputs and input fields I have tried different formats:

- {{tasks.FS_batching.values.var_name}}

- "{{tasks.FS_batching.values.var_name}}"

- ${tasks.FS_batching.values.var_name}

All of them failing with different errors. It seem that Terraform get crazy with those calls, any idea on how to solve it?

I am setting the parameter with: dbutils.jobs.taskValues.set(...) and retrieving it through: dbutils.widgets.get() because I am using scala in my notebooks and task values only seem to work with Python, so I use the widgets to retrieve them. But this should not mess with parameter setup into yaml.