Invoking one job from another to execute a specific task

[shan-databricks]

I have multiple tasks, each working with different tables. Each table has dependencies across Bronze, Silver, and Gold layers.

I want to trigger and run a specific task independently, instead of running all tasks in the job. How can I do this?

Also, is there a way to trigger a specific task from another job?

[balajij8]

You can trigger a specific task from another job code.

• Triggering Task via Python
  You can invoke a specific task of a second job using below code in the first job. Ensure the calling Service Principal has Run permissions on the target job.

w = WorkspaceClient()

# Trigger the 'Task1' task within the specified Job ID
w.jobs.run_now(
job_id=615516,
only=["Task1"] # run Task1
)​

• You can also use Conditional Tasks (If/Else) within a job and use modular code in jobs to control it better

[emma_s · Accepted solution]

Hi,

There is no way that I'm aware of to just trigger one task in a pipeline. You can repair a run, but this will just trigger everything in the job from the point it failed out. If I've understood you correctly, the better approach may be to separate your table creations into individual jobs, then run an overall job to execute them for your day-to-day schedule. You will then be able to execute the individual components as well, as and when you need to.

I hope that helps.

Thanks,

Emma

[rohan22sri]

1. Go to job and left click on task you want to run .
2. Click on play button(highlighted in yellow in attachment )
3. This make sure that you run only 1 task at a time and not the whole job .