topic Capturing notebook return codes in databricks jobs in Data Engineering

Capturing notebook return codes in databricks jobs

labromb — Fri, 28 Oct 2022 13:17:17 GMT

Hi,

I currently am running a number of notebook jobs from Azure Data Factory. A new requirement has come up where I need to capture a return code in ADF that has been generated from the note. I tried using

dbutils.notebook.exit(json.dumps({"return_value":"[some text or code]})) but this is not visible from the output of the job. I could however see this value if I triggered that notebook from a call notebook task in ADF.

Am I missing something, any assistance greatly appreciated

Re: Capturing notebook return codes in databricks jobs

JaviRuiz — Tue, 18 Mar 2025 14:09:13 GMT

I am looking for this too, as I want to run a job from an Azure DevOps pipeline authorized to access Databricks.

Re: Capturing notebook return codes in databricks jobs

JaviRuiz — Tue, 18 Mar 2025 15:22:11 GMT

I tried dbutils.notebook.exit(<value>) where value is 0 and 1. I also tried sys.exit(<value>).
None of this produces a difference in the output of the databricks CLI command

databricks jobs run-now --json '{"job_id": <JOB_ID>, "job_parameters": {"will_exit": "yes"}}'

Furthermore, if I use sys.exit(<value>) the API doesn't return a result in JSON, but fails:

⣷ Error: failed to reach TERMINATED or SKIPPED, got INTERNAL_ERROR: Task IntegrationTest failed with message: Workload failed, see run output for details. This caused all downstream tasks to get skipped

It's a pity...