We're currently using the python sdk, but the same problem is in the databricks cli. The documentation states that when using workspace.jobs.get_run_output().logs, the last 5 MB of these logs are returned. However, we notice that the logs are truncated even when the logs are much smaller (we know because we log the same logs to a file as well). I'm looking for the actual reason logs get truncated. Number of lines? Line length? Hope somebody knows!
