I have a basic 2 task job.
- The 1st notebook (task) checks whether the source file has changes and if so then refreshes a corresponding materialized view. In case we have no changes then I use dbutils.jobs.taskValues.set(key = "skip_job", value = 1) & dbutils.notebook.exit("SKIP").
- The 2nd notebook (task) checks the passed variable from the 1st notebook with skip_job = dbutils.jobs.taskValues.get(taskKey = "01_prepare_fms_rmf_data", key = "skip_job", default = 0, debugValue = 0) and skips execution accordingly if (skip_job == 1): dbutils.notebook.exit("SKIP")
This seems to work fine for my case but I have I minor issue. I want to receive email notification upon Success or failure but not upon skip. The job Status is Success even when I use the above skip approach.
Is there a solution to this? Maybe there is some other command I can use to set the job status as skipped?