Hi @dbuserng,

It is possible but it requires custom code setup based on your use-case. You can use job REST API: https://docs.databricks.com/api/workspace/jobs.

Create a Monitoring Job: Set up a job that will monitor the completion status of the three workflows. This job will periodically check if all three workflows have completed successfully.

Use the Jobs API: Utilize the Databricks Jobs API to check the status of the workflows. You can use the jobs/runs/list endpoint to get the status of the runs for each workflow.

Conditional Trigger: In the monitoring job, implement logic to check if all three workflows have completed successfully. If they have, trigger the desired job.

Here is a high-level example of how you can implement this:

import requests

# Define the job IDs for the three workflows

workflow_ids = [workflow1_id, workflow2_id, workflow3_id]

# Function to check the status of a workflow

def check_workflow_status(job_id):

    response = requests.get(f'https://<databricks-instance>/api/2.0/jobs/runs/list?job_id={job_id}')

    runs = response.json().get('runs', [])

    if runs and runs[0]['state']['life_cycle_state'] == 'TERMINATED' and runs[0]['state']['result_state'] == 'SUCCESS':

        return True

    return False

# Check the status of all workflows

all_completed = all(check_workflow_status(job_id) for job_id in workflow_ids)

# If all workflows are completed successfully, trigger the next job

if all_completed:

    trigger_response = requests.post('https://<databricks-instance>/api/2.0/jobs/run-now', json={"job_id": next_job_id})

    if trigger_response.status_code == 200:

        print("Next job triggered successfully")

    else:

        print("Failed to trigger the next job")