cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks API - list job runs doesn't have 'task run id'

Gaurav19
New Contributor III

Hi all,

I am calling get job run list API to get all task ids and refer them in dbt-artifacts view created by dbt job run. The question is I can see 'task run id' on screen but it doesn't come back in api response? Is there a way to get it? I checked all the other api's for task_id in it but have no luck!

 

Gaurav19_0-1718197141799.png

 

1 ACCEPTED SOLUTION

Accepted Solutions

Gaurav19
New Contributor III

Never mind, I have found task_run_id present in getrun api
 https://docs.databricks.com/api/azure/workspace/jobs/getrun

I overlooked at first instance as it is buried under nested json structure
tasks[] > run_id.

This clarifies and solves my problem!

View solution in original post

3 REPLIES 3

daniel_sahal
Esteemed Contributor

@Gaurav19 
I can see in the documentation that /api/2.1/jobs/runs/list contains the run_idhttps://docs.databricks.com/api/workspace/jobs/listruns
It's hard to tell what is wrong based on screenshot. Could you share some more details?

@daniel_sahal - thanks for responding!

In the screenshot, you can see "Job ID", "Job run ID" and "Task run ID". I need the third one "Task run ID" - the list API does return run_id as you mentioned, but it is "Job run ID".

to give a context: I need the task id to find which dbt model and test got executed as part of the run - I am executing below query for that to co-relate a job run with model/test execution from dbt-artifacts:

select * from `hive_metastore`.`cfs_artifacts_src`.`invocations` where invocation_args like '%{task_run_id}%'.
 
In above table/view I can't find job_run_id anywhere. And it is a bit weird the API is providing one id but not the other - I also tried calling single job run api - https://docs.databricks.com/api/azure/workspace/jobs/getrun Unfortunately here also I don't see any field which contains 'Task run id'.

Gaurav19
New Contributor III

Never mind, I have found task_run_id present in getrun api
 https://docs.databricks.com/api/azure/workspace/jobs/getrun

I overlooked at first instance as it is buried under nested json structure
tasks[] > run_id.

This clarifies and solves my problem!

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!