How can I set the status of a databricks job as skipped via python?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-07-2023 06:35 AM
I have a basic 2 task job.
- The 1st notebook (task) checks whether the source file has changes and if so then refreshes a corresponding materialized view. In case we have no changes then I use dbutils.jobs.taskValues.set(key = "skip_job", value = 1) & dbutils.notebook.exit("SKIP").
- The 2nd notebook (task) checks the passed variable from the 1st notebook with skip_job = dbutils.jobs.taskValues.get(taskKey = "01_prepare_fms_rmf_data", key = "skip_job", default = 0, debugValue = 0) and skips execution accordingly if (skip_job == 1): dbutils.notebook.exit("SKIP")
This seems to work fine for my case but I have I minor issue. I want to receive email notification upon Success or failure but not upon skip. The job Status is Success even when I use the above skip approach.
Is there a solution to this? Maybe there is some other command I can use to set the job status as skipped?
Labels:
- Labels:
-
Databricks Job
-
Job
-
Task