cancel
Showing results for 
Search instead for 
Did you mean: 

How can I set the status of a databricks job as skipped via python?

Michael_Papadop
New Contributor II

I have a basic 2 task job.

  • The 1st notebook (task) checks whether the source file has changes and if so then refreshes a corresponding materialized view. In case we have no changes then I use dbutils.jobs.taskValues.set(key = "skip_job", value = 1) & dbutils.notebook.exit("SKIP").
  • The 2nd notebook (task) checks the passed variable from the 1st notebook with skip_job = dbutils.jobs.taskValues.get(taskKey = "01_prepare_fms_rmf_data", key = "skip_job", default = 0, debugValue = 0) and skips execution accordingly if (skip_job == 1): dbutils.notebook.exit("SKIP")

This seems to work fine for my case but I have I minor issue. I want to receive email notification upon Success or failure but not upon skip. The job Status is Success even when I use the above skip approach.

Is there a solution to this? Maybe there is some other command I can use to set the job status as skipped?

3 REPLIES 3

karthik_p
Esteemed Contributor

@Michael Papadopoulos​ please select below option in job notification settings , that should handle u r caseimage

Sorry, I forgot to mention that I had already tried that flag with no success. The job still sends notification even upon skip of both tasks.

Also the Statuses of both tasks and whole job is Succeeded.

image 

imageimage 

image

karthik_p
Esteemed Contributor

@Michael Papadopoulos​ usually that should not be the case i think, as for task level we have 3 level notifications ( success, failure,start), where as whole job level skip option is available to discard notification . will see if some one from community faced or respond

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.