Access Each Input Item of a For Each Task

rk2511 — Tue, 17 Dec 2024 17:38:18 GMT

I have two tasks. The first task (Sample_Notebook) returns a JSON array (Input_List).

Sample data in Input_List

['key1':value1, 'key2':value2, 'key3':value3]

The second task is a "For Each" task that executes a notebook for each entry in the Input_List.

The number of Iterations are as expected and even the key/value pairs are showing fine in the job window. However,

I am not sure how to access each item within the Notebook of the second task.

The below command does not work properly

dbutils.jobs.taskValues.get(taskKey="Sample_Notebook",key="Input_List")

It simply throws the complete input for each iteration.

How to access the corresponding item of the iteration?

Re: Access Each Input Item of a For Each Task

Louis_Frolio — Tue, 17 Dec 2024 20:50:03 GMT

To access each item of the iteration within the notebook of the second task in your Databricks workflow, you need to utilize the parameterization feature of the For Each task. Instead of trying to retrieve the entire list using dbutils.jobs.taskValues.get, you should configure the For Each task to pass individual items of the Input_List to each iteration as parameters.

topic Re: Access Each Input Item of a For Each Task in Warehousing & Analytics

Access Each Input Item of a For Each Task

Re: Access Each Input Item of a For Each Task