Access Each Input Item of a For Each Task
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-17-2024 09:38 AM
I have two tasks. The first task (Sample_Notebook) returns a JSON array (Input_List).
Sample data in Input_List
['key1':value1, 'key2':value2, 'key3':value3]
The second task is a "For Each" task that executes a notebook for each entry in the Input_List.
The number of Iterations are as expected and even the key/value pairs are showing fine in the job window. However,
I am not sure how to access each item within the Notebook of the second task.
The below command does not work properly
dbutils.jobs.taskValues.get(taskKey="Sample_Notebook",key="Input_List")
It simply throws the complete input for each iteration.
How to access the corresponding item of the iteration?
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-17-2024 12:50 PM
To access each item of the iteration within the notebook of the second task in your Databricks workflow, you need to utilize the parameterization feature of the For Each task. Instead of trying to retrieve the entire list using dbutils.jobs.taskValues.get, you should configure the For Each task to pass individual items of the Input_List to each iteration as parameters.
![](/skins/images/8C2A30E5B696B676846234E4B14F2C7B/responsive_peak/images/icon_anonymous_message.png)
![](/skins/images/8C2A30E5B696B676846234E4B14F2C7B/responsive_peak/images/icon_anonymous_message.png)