topic Re: run_if condition to handle prior task excluded? in Data Engineering

run_if condition to handle prior task excluded?

ChristianRRL — Tue, 05 May 2026 19:29:25 GMT

Hi there, I kind of know the answer here but want to check in case I'm missing anything (or else maybe vent slightly and hope for new functionality in the future).

Basically, I'm looking for a way to run a task if either (A) the prior step ran successfully, or (B) the prior step was excluded. However, based on available documentation, this doesn't seem to be possible because:

Configure task dependencies | run-if-condition-options
- "If all task dependencies are excluded, the task is also excluded, regardless of its Run if condition."

Here is an example highlighting this. As per the documentation, regardless of which run_if condition I use (All succeeded, At least one succeeded, None Failed, All done), because the prior step is excluded the following step is also excluded:

This leads me to the following configuration because existing functionality doesn't seem to support the simpler behavior I'm looking for:

Let me know if I'm missing anything, or is there really no way to a task if the prior task has been excluded? If so, can I request for this functionality to be added in the future?

Re: run_if condition to handle prior task excluded?

amirabedhiafi — Tue, 05 May 2026 20:51:18 GMT

Hello @ChristianRRL !

You are totally righy. With the current DBKS dependency semantics, a downstream task cannot run when all of its direct upstream dependencies are excluded regardless of the run_if option. If you check the doc it explicitly says (I am quoting here) "if all task dependencies are excluded, the task is also excluded, regardless of its run if condition." It also says excluded upstream tasks are treated as successful only when evaluating run_if but that rule does not help when all dependencies are excluded.

IMHO, your workaround is correct and you can add a second upstream dependency that is not excluded for example the condition task like Should_Run_Silver.

However, I would suggest using None failed instead of At least one succeeded for the curated tasks.

Why ? because if silver succeeds the curated runs and if it is excluded but Should_Run_Silver succeeded you will also have curated running and if silver fails the curated does not run.

With At least one succeeded, the curated task could still run even if the silver task failed because Should_Run_Silver succeeded.is common branching pattern.