Databricks Workflow Task Failure - Custom Error Messages

[ChristianRRL]

Hi there, kind of silly question but I'm hoping there's a simple way to address this.

I'm trying to make sure that when certain job failure conditions arise, that a workflow task step fails. I've tried this with both raise RuntimeError and sys.exit, but in both cases I'm seeing more in the output than I would like.

I would like to output the most minimal amount of text so that users who are troubleshooting can easily read the custom error message we are outputing.

Is there another/better approach to what I'm trying to do, or is this a Databricks limitation in how raised errors or system exit messages are output?

Also, would one be recommended over the other (raised exception vs system exit)?

Below are examples for both:

[K_Anudeep]

Hey @ChristianRRL !

When a Python wheel/notebook/script task fails, Jobs always does three things:

1. Render the Python traceback in task output (IPython / Databricks REPL showtraceback).
2. Extract a one-line summary (RuntimeError: <your message>), which is the yellow line at the bottom of your screenshot.
3. Set the task termination message to Workload failed, see run output for details. That string is a fixed platform copy for “user code failed.” You cannot replace it with your custom message.

Python wheel tasks are executed as notebook commands, so they receive the same wrapping. Jobs already strips some internal wheel frames, but your main() / raise frames stay.

There is no supported API today to fail a task with only a custom string and no traceback (dbutils.notebook.exit(...)). Also, it does not fail the task; it succeeds and returns a value.