topic Re: How obtain a list of workflows in Databricks? in Get Started Discussions

How obtain a list of workflows in Databricks?

VabethRamirez — Fri, 05 Apr 2024 16:16:09 GMT

I need to obtain a list of my Databricks workflows with their job IDs in a notebook Databricks

Re: How obtain a list of workflows in Databricks?

Lakshay — Fri, 05 Apr 2024 16:58:40 GMT

Can you check if this post answers you question: https://community.databricks.com/t5/community-discussions/reading-workflow-items/td-p/61277

Re: How obtain a list of workflows in Databricks?

shan_chandra — Fri, 05 Apr 2024 17:08:11 GMT

@VabethRamirez - Please find the below code snippet. Kindly modify according to your needs.

import requests, json # Set the Databricks API URL. <eg. dbi.databricks.cloud.com> api_url = 'https://<URL>/api/2.1' # Set the Databricks token token = '<API token>' # Get a list of all jobs jobs_endpoint = f"{api_url}/jobs/list" headers = { 'Authorization': f"Bearer {token}", 'Content-Type': 'application/json' } response = requests.get(jobs_endpoint, headers=headers) if response.status_code == 200: # Extract the list of jobs jobs = response.json()['jobs'] # Print the job names and IDs for job in jobs: print(f"Job ID: {job['job_id']}, Job Name: {job['settings']['name']}") else: print(f"Failed to retrieve jobs. Status Code: {response.status_code}")

Re: How obtain a list of workflows in Databricks?

VabethRamirez — Fri, 05 Apr 2024 17:29:42 GMT

I managed to solve it with this solution

# Importar la biblioteca de Databricks

from databricks_api import DatabricksAPI

# Configurar el token de acceso a la API de Databricks

token = "<token>"

host = "<url_your_databricks_host>"

# Crear una instancia de la API de Databricks

db = DatabricksAPI(host=host, token=token)

# Obtener una lista de todos los workflows en Databricks

jobs_list = jobs['jobs']

# Iterar sobre los trabajos y obtener información detallada, incluyendo Job ID

for job in jobs_list:

job_id = job['job_id']

job_name = job['settings']['name']

print(f"Job Name: {job_name}, Job ID: {job_id}")

Re: How obtain a list of workflows in Databricks?

Lakshay — Fri, 05 Apr 2024 17:33:14 GMT

Thank you for sharing!

Re: How obtain a list of workflows in Databricks?

artsheiko — Tue, 16 Apr 2024 14:17:25 GMT

Hi @VabethRamirez ,

Also, instead of using directly the API, you can use databricks Python sdk :

%pip install databricks-sdk --upgrade dbutils.library.restartPython()

from databricks.sdk import WorkspaceClient w = WorkspaceClient() job_list = w.jobs.list(expand_tasks=False)

Additional examples : https://github.com/databricks/databricks-sdk-py/tree/main/examples