cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Identify a job name in case notebook is triggered from another notebook/

Suman-Sourav
New Contributor II

I am running an another notebook from one of the notebook as below. The main notebook is scheuled with a job/workflow.

dbutils.notebook.run("./ABC/XYZ/another_notebook", 1000)
 
Usually to get the workflow/job name i use any of the below 2 options.
 
1.  desc history dbw_lakehouse.uat.table_name  (if  notebook loads to table)
 
or
 
2. Through API call
 
 
But in my scenario where i am calling notebook from extreanl notebook, i am not seeing the job name in any of the option output.
 
Auy suggesiton?
2 REPLIES 2

Alberto_Umana
Databricks Employee
Databricks Employee

When you run a notebook from another notebook using dbutils.notebook.run, the job name of the parent notebook (the one that is scheduled with a job/workflow) is not directly accessible in the child notebook (the one being called). This is why you are not seeing the job name in the output of your usual methods

 

To work around this, you can pass the job name as a parameter to the child notebook. Here’s how you can do it:

  1. Retrieve the job name in the parent notebook: You can use the Databricks REST API to get the job name. For example

 

import requests

import json

 

run_id = dbutils.notebook.entry_point.getDbutils().notebook().getContext().currentRunId().get()

response = requests.get(f"https://<databricks-instance>/api/2.0/jobs/runs/get?run_id={run_id}",

                        headers={"Authorization": f"Bearer {dbutils.notebook.entry_point.getDbutils().notebook().getContext().apiToken().get()}"})

job_name = response.json().get("job_name")

 

  1. Pass the job name to the child notebook: When calling the child notebook, pass the job name as an argument:

 

dbutils.notebook.run("./ABC/XYZ/another_notebook", 1000, {"job_name": job_name})

 

  1. Access the job name in the child notebook: In the child notebook, retrieve the job name from the arguments:

 

job_name = dbutils.widgets.get("job_name")

print(f"Job Name: {job_name}")

Suman-Sourav
New Contributor II

Thanks Alberto. I am aware of that solution but trying to get an option to get job name in the scenario i gave. 

So i assume there is no way to get the job name in that scenario. 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group