Hey @Kaz ,
Unfortunately, the output truncation limit in the Databricks job UI cannot be changed. Once that limit is exceeded, the rest of the logs are skipped, and the full logs become accessible only through the โLogsโ button, which, as you mentioned, requires elevated permissions that might not be available in your case. You could ask for "Can View" permissions
A good workaround, especially when access to cluster logs is restricted, is to redirect important log output to a Unity Catalog volume during the job execution. This allows you to bypass the UI limitations and store the logs in a location that can be accessed later through standard permissions.
By writing logs to a volume, you:
Avoid losing output due to UI truncation
Persist logs in a governed and centralized location
Maintain separation from cluster internals โ users can read the logs without needing access to the job or cluster itself
"Delivering logs to volumes is in Public Preview and is only supported on Unity-Catalog-enabled compute with Standard access mode or Dedicated access mode assigned to a user. This feature is not supported on compute with Dedicated access mode assigned to a group. If you select a volume as the path, ensure you have the READ VOLUME and WRITE VOLUME permissions on the volume"
https://learn.microsoft.com/en-us/azure/databricks/compute/configure#cluster-log-delivery
Hope this helps ๐
Isi
