topic Re: Pass the job even if specific task fails in Data Engineering

Pass the job even if specific task fails

Puru20 — Sun, 24 Aug 2025 12:50:31 GMT

Hi , I have multiple data pipelines and each has data quality check as a final task which runs on dbt. There are 1500 test cases altogether runs everyday which is being captured on dashboard. Is there a way to pass the job even if this particular talk fail due to test failure as we can check dashboard for any test failures and do not want the workflow to fail.

Re: Pass the job even if specific task fails

szymon_dybczak — Sun, 24 Aug 2025 13:56:33 GMT

Hi @Puru20 ,

Yes, there is 🙂 Databricks determines whether a job run was successful based on the outcome of the job's leaf tasks. A leaf task is a task that has no downstream dependencies.

So, just add another task that will have a dependency on your final task (so, now this task becomes your final one) and in that task you can define your success/failure logic.

https://docs.databricks.com/aws/en/jobs/monitor#how-does-databricks-determine-job-run-status

Re: Pass the job even if specific task fails

Puru20 — Tue, 26 Aug 2025 05:48:21 GMT

Hi @szymon_dybczak The solution works perfectly when I set leaf job to pass irrespective of dbt test task status. Thanks much!

Re: Pass the job even if specific task fails

szymon_dybczak — Tue, 26 Aug 2025 05:38:37 GMT

Hi @Puru20 , great that you managed to work it out. If the answer was helpful to you, please consider marking it as a best answer.