For each: nested JAR task can’t reference upstream taskValues

javasquez — Fri, 22 Aug 2025 15:10:40 GMT

I’m creating a For each with a nested JAR task.
init_job (notebook) sets dbutils.jobs.taskValues.set("my_value", ...), and the For each container depends on init_job.

but the UI says: “Reference … can only be used in a task that depends on task 'init_job'.” This seems wrong because the nested task has a transitive dependency via the For each.

If I switch the JAR nested task to Notebook Task, it can read the value correctly. for example:

Is this a bug/known limitation for JAR tasks inside For each?

Re: For each: nested JAR task can’t reference upstream taskValues

WiliamRosa — Fri, 22 Aug 2025 18:01:06 GMT

Yep—this is a current Workflows limitation. Task value refs ({{tasks.<key>.values.<name>}}) only work if the consuming task directly depends on the producer. A For Each/group dependency doesn’t count—nested tasks don’t inherit those edges for interpolation. That’s why the UI complains even though the group depends on init_job.

topic For each: nested JAR task can’t reference upstream taskValues in Data Engineering

For each: nested JAR task can’t reference upstream taskValues

Re: For each: nested JAR task can’t reference upstream taskValues