cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Confusion about {{run_id}} and {{parent_run_id}} variables for Databricks jobs (Azure)

essentialDatabr
New Contributor II

In Databricks jobs on Azure you can use the {{run_id}} and {{parent_run_id}}

variables for a specific run: https://docs.databricks.com/workflows/jobs/jobs.html

For Databricks jobs with only two or more tasks, then {{run_id}} seems to correspond to 

task_run_id and {{parent_run_id}} seems to correspond to the job_run_id.

For Databricks jobs with only one task, then 

{{parent_run_id}} seems to correspond to the task_run_id, but what does the 

{{run_id}} correspond to? Is that the job_run_id?

1 REPLY 1

Anonymous
Not applicable

@Kasper H​ :

Yes, you are correct in your understanding that in Databricks jobs with multiple tasks, the {{run_id}} variable corresponds to the task_run_id and the {{parent_run_id}} variable corresponds to the job_run_id.

For Databricks jobs with only one task, the {{parent_run_id}} variable still corresponds to the task_run_id, which is the unique identifier for that particular run of the task. In this case, the {{run_id}} variable would correspond to the job_run_id, which is the unique identifier for the entire job run that includes that single task.

So, in summary:

  • For Databricks jobs with multiple tasks: {{run_id}} = task_run_id, {{parent_run_id}} = job_run_id
  • For Databricks jobs with only one task: {{run_id}} = job_run_id, {{parent_run_id}} = task_run_id

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.