cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results for 
Search instead for 
Did you mean: 

Access Each Input Item of a For Each Task

rk2511
New Contributor
I have two tasks. The first task (Sample_Notebook) returns a JSON array (Input_List).
Sample data in Input_List
['key1':value1, 'key2':value2, 'key3':value3]
The second task is a "For Each" task that executes a notebook for each entry in the Input_List. 
The number of Iterations are as expected and even the key/value pairs are showing fine in the job window. However,
I am not sure how to access each item within the Notebook of the second task. 
The below command does not work properly
dbutils.jobs.taskValues.get(taskKey="Sample_Notebook",key="Input_List")
It simply throws the complete input for each iteration.
How to access the corresponding item of the iteration?
1 REPLY 1

BigRoux
Databricks Employee
Databricks Employee

To access each item of the iteration within the notebook of the second task in your Databricks workflow, you need to utilize the parameterization feature of the For Each task. Instead of trying to retrieve the entire list using dbutils.jobs.taskValues.get, you should configure the For Each task to pass individual items of the Input_List to each iteration as parameters.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group