topic Re: Access specific input item of For Each Tasks in Warehousing & Analytics

Access specific input item of For Each Tasks

Hania__b — Tue, 04 Feb 2025 13:22:53 GMT

Hi,

I think I have a similar issue to the one in this post, but the answer isn't detailed enough for me.

I have a list defined in my first task, which contains the items I want to iterate through [1,2,3,4]. When I use it as Inputs to the For Each framing task, I can see it as Input, e.g. Input = 1, Input = 2 but can't access it in the notebook so that I could use it as a variable in the second task (not the entire list, but just the item in the list).

Can you explain your comment, with an example?

I've tried using a Job Parameter, but when it prints in the notebook, it seems to output the same parameter value for every iteration in the looper, but I might be applying it wrongly.

Re: Access specific input item of For Each Tasks

Walter_C — Tue, 04 Feb 2025 13:30:32 GMT

To access the individual items from your list [1, 2, 3, 4] in the notebook for each iteration of the For Each task, you can use the {{input}} reference. Here’s a step-by-step example to help you understand how to achieve this:

Define the For Each Task:
- Create a new For Each task in your job configuration.
- Set the inputs field to your list: "[1, 2, 3, 4]".
- Configure the nested task that will run for each item in the list.
Configure the Nested Task:
- In the nested task, use the {{input}} reference to access the current item in the iteration.
- For example, if you are using a notebook task, you can pass the {{input}} as a parameter to the notebook.
Access the Parameter in the Notebook:
- In your notebook, use dbutils.widgets.get to retrieve the parameter value.

Here’s an example configuration:

{ "name": "Process Items", "tasks": [ { "task_key": "process_items", "for_each_task": { "inputs": "[1, 2, 3, 4]", "task": { "task_key": "process_item_iteration", "notebook_task": { "notebook_path": "/path/to/your/notebook", "base_parameters": { "item": "{{input}}" } } } } } ] }

Re: Access specific input item of For Each Tasks

MariuszK — Tue, 04 Feb 2025 13:33:19 GMT

Hi,
You need a notebook to pass list:

dbutils.jobs.taskValues.set(key = "list", value = your_list)

In foreach you need a notebook with widgest to read passed parameters:

dbutils.widgets.text("param_1", "")
param_1 = dbutils.widgets.get("param_1")

Re: Access specific input item of For Each Tasks

Hania__b — Tue, 04 Feb 2025 13:51:00 GMT

Thank you both very much, I've nailed it 😁. I have accepted Walter_C's answer as solution because Step 2 is what I was missing. Thanks MariuszK as well for your contribution.