cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

I have a notebook in workspace, how to know in which job this particular notebook is referenced.

SS_RATH
New Contributor

I have a notebook in workspace, how to know in which job this particular notebook is referenced.

3 REPLIES 3

Ayushi_Suthar
Databricks Employee
Databricks Employee

Hi @SS_RATH, Good Day! 

I want to inform you that there isn't a direct way to search for a job by the notebook it references. You will have to manually check each job to see which one is using the specific notebook you are interested in. 

You can follow these steps to find out the job which refers a specific notebook: 

1. Navigate to the Databricks workspace.
2. In the sidebar, click on 'Workflows'.
3. In the 'Jobs' tab, you will see a list of all the jobs.
4. Click on the name of each job to view its details.
5. In the 'Job UI page', look for the 'Notebook' section under 'Task options'. This section will show the path of the notebook that is being used by the job. 

https://docs.databricks.com/en/workflows/jobs/create-run-jobs.html

Or you can use the Job API to list out all the Jobs and it will give you the complete details about the Jobs present in your workspace. Please refer to this document for the same: https://docs.databricks.com/api/workspace/jobs/list

Please let me know if this helps and leave a like if this information is useful, followups are appreciated.
Kudos
Ayushi

TamD
Contributor

Hi @Ayushi_Suthar I don't see "Task options" in the Job UI.  If I click on a Task, I see Task name, Type, Job, Depends on, Job parameters, Notifications, and Duration threshold.  @SS_RATH , did you find an answer?

Panda
Valued Contributor

@SS_RATH @TamD There are couple of ways

  1. Call Databricks REST API  - Use the /api/2.1/jobs/list API to list and search through all jobs. Example: - 

 

import requests

workspace_url = "https://<databricks-instance>"
databricks_token = "<your-databricks-token>"

url = f"{workspace_url}/api/2.1/jobs/list"
headers = {"Authorization": f"Bearer {databricks_token}"}

response = requests.get(url, headers=headers)
jobs = response.json().get('jobs', [])

notebook_path = "/Workspace/Users/xxx@domain.com/MyNotebook"

jobs_used_notebook = []
for job in jobs:
    job_name = job.get('settings', {}).get('name', 'Unnamed Job')  # Get job name
    for task in job.get('settings', {}).get('tasks', []):
        if task.get('notebook_task', {}).get('notebook_path') == notebook_path:
            jobs_used_notebook.append({'job_id': job['job_id'], 'job_name': job_name})

if jobs_used_notebook:
    for job_info in jobs_used_notebook:
        print(f"Job Name: {job_info['job_name']}, Job ID: {job_info['job_id']}")
else:
    print(f"No jobs found used the notebook path: {notebook_path}.")

 

2. Go to  Databricks Workspace UI - Job Search enter your full path of the notebook Then Click serach. After result populate make sure choose your type in your case it's "Job"

Panda_1-1728912669414.png

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group