Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-06-2024 12:52 AM
Extending to what @mhiltner has suggested, let's say you have 2 streaming tasks streamA and streamB. Create 2 separate tasks taskA and taskB. Each of these tasks should execute the same notebook which makes an API call to the CANCEL RUN or CANCEL ALL RUNS OF A JOB endpoint.
Task settings should be as follows :
- taskA be dependent on streamA with Run if dependencies as All failed. So taskA will only run when streamA fails.
- taskB to be dependent on streamB with Run if dependencies as All failed. So taskB will only run when streamB fails.
So now, if streamA fails, taskA will run and cancel the run. If streamB fails, taskB will run and cancel the run.