Access to For Each run ids from jobs rest API

N_M
Contributor

Hello Community

I'm using the for_each tasks in workflows, but I'm struggling to access to the job information through the job APIs.

In short, using the runs api (Get a single job run | Jobs API | REST API reference | Databricks on AWS), I'm able to access the single tasks run IDs of the specific run, when they are sub-jobs.

"tasks": [{'run_id': XXXXX,...},...]

 In case of for each tasks, I can access the "root" task (name, run ID, everything), I can access the input parameters used in the sub jobs steps, but I can't access the run IDs of each iteration (but this information is easily accessible from the databricks portal).

On the other hand, if I get via API the info about such steps using the specific iteration run ID (that I take from the portal), the job is of course there, and it's also connected to the root ID via

{job_run_id:RUN_ID}

  Am I missing something? How can I access the iteration run IDs knowing the run ID of the caller?

N_M
Contributor

Small correction:
Iteration jobs can be connected to root job IDs via the attribute

{parent_run_id:RUN_ID}

 

N_M
Contributor

I couldn't figure out anything to systematically access for_each tasks. I believe this is a quite important missing piece (unless of course I'm missing something).
Is there a way to ask to a developer or should I open a ticket through my company?

prabhatika
New Contributor II

This feature would be extremely helpful in monitoring each task in the `foreachtask` task.