topic Repairing running workflow with few failed child jobs in Data Engineering

Repairing running workflow with few failed child jobs

holychs — Sun, 06 Apr 2025 18:07:00 GMT

I have a parent job that calls multiple child jobs in workflow, Out of 10 child jobs, one has failed and rest 9 are still running, I want to repair the failed child tasks. can I do that while the other child jobs are running?

Re: Repairing running workflow with few failed child jobs

Brahmareddy — Mon, 07 Apr 2025 02:39:22 GMT

Hi holychs,

How are you doing today?, As per my understanding, yes, in Databricks Workflows, if you're running a multi-task job (like your parent job triggering multiple child tasks), you can repair only the failed task without restarting the entire job. However, to do this, you need to wait until the rest of the tasks finish running, because Databricks doesn't currently allow you to repair individual tasks while others are still in progress. Once all the other child jobs have completed (successfully or failed), you can go to the job run in the UI and click “Repair run”, then select just the failed task to retry. It’s a helpful feature for large workflows, and it avoids re-running everything from scratch. Let me know if you want help setting up repair-friendly dependencies or alerts!

Regards,

Brahma

Re: Repairing running workflow with few failed child jobs

holychs — Wed, 23 Apr 2025 05:57:09 GMT

Yes, that is what I am currently doing. I wanted to know if there is any way to repair the failed ones while the parent job is running so that I don't need to wait which does not seem possible as of today.