Hi everyone,

I have a workflow involving two notebooks: Notebook A and Notebook B. At the end of Notebook A, we generate a variable number of files, let's call it N. I want to run Notebook B for each of these N files.

I know Databricks has a Foreach task that can iterate over a list of items.

Here's what I've tried so far

output_dir_paths = [<list of paths>]

dbutils.jobs.taskvalues.set(key="notebook_A_output_paths", value=output_dir_paths)

ForEach Loop:

The Task:

In Notebook B, I'm attempting to read each path like this:

path = dbutils.widgets.get("single_batch_file")

Could someone please help me correct the code to pass the list of paths from Notebook A, iterate over each path, and send it to Notebook B?