topic Show full logs on job log in Warehousing & Analytics

Show full logs on job log

Kaz — Thu, 05 Oct 2023 10:14:26 GMT

Is it possible to show the full logs of a databricks job? Currently, the logs are skipped with:

*** WARNING: max output size exceeded, skipping output. ***

However, I don't believe our log files are more than 20 MB. I know you can press the logs button and view the full logs there. However, due to admin rights we would like to be able to read all logs in the job overview, without having access to the logs (and therefore other) button(s).

Re: Show full logs on job log

piotrsofts — Fri, 14 Mar 2025 07:17:12 GMT

Hey Kaz

Where did you get that ?

Re: Show full logs on job log

GalenSwint — Mon, 28 Apr 2025 21:38:00 GMT

I am having the same problem as Kaz. What's weird is I have another cluster where that doesn't happen. I didn't set up the other cluster but wondered if it had some sort of setting for this that I could change.

Re: Show full logs on job log

Isi — Wed, 07 May 2025 12:24:16 GMT

Hey @Kaz ,

Unfortunately, the output truncation limit in the Databricks job UI cannot be changed. Once that limit is exceeded, the rest of the logs are skipped, and the full logs become accessible only through the “Logs” button, which, as you mentioned, requires elevated permissions that might not be available in your case. You could ask for "Can View" permissions

A good workaround, especially when access to cluster logs is restricted, is to redirect important log output to a Unity Catalog volume during the job execution. This allows you to bypass the UI limitations and store the logs in a location that can be accessed later through standard permissions.

By writing logs to a volume, you:

Avoid losing output due to UI truncation
Persist logs in a governed and centralized location
Maintain separation from cluster internals — users can read the logs without needing access to the job or cluster itself

"Delivering logs to volumes is in Public Preview and is only supported on Unity-Catalog-enabled compute with Standard access mode or Dedicated access mode assigned to a user. This feature is not supported on compute with Dedicated access mode assigned to a group. If you select a volume as the path, ensure you have the READ VOLUME and WRITE VOLUME permissions on the volume"

https://learn.microsoft.com/en-us/azure/databricks/compute/configure#cluster-log-delivery

Hope this helps 🙂

Isi

Re: Show full logs on job log

jkb7 — Wed, 29 Oct 2025 08:05:45 GMT

Any news on this topic?
Have the limits on the notebook-cell-log output been resolved?