cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks CLI/SDKs not returning all logs even when less than 5 MB

Kaz
New Contributor II

We're currently using the python sdk, but the same problem is in the databricks cli. The documentation states that when using workspace.jobs.get_run_output().logs, the last 5 MB of these logs are returned. However, we notice that the logs are truncated even when the logs are much smaller (we know because we log the same logs to a file as well). I'm looking for the actual reason logs get truncated. Number of lines? Line length? Hope somebody knows!

1 REPLY 1

Kaniz
Community Manager
Community Manager

Hi @KazThe truncation of logs in Databricks occurs when the size of the JSON representation exceeds 100 KB. When this limit is reached, the values are truncated, and the string “truncated” is appended to the affected entries. In rare cases where a truncated map is still larger than 100 KB, a single TRUNCATED key with an empt...1. This behaviour applies to both the Python SDK and the Databricks CLI.

If you’re experiencing truncation even for smaller logs, it’s essential to check the overall size of the JSON representation of the logs. If the total size exceeds the specified limit, truncation will occur. You might want to investigate the specific content of the logs and their structure to identify any unusually large entries or patterns that contribute to the truncation.

Additionally, keep in mind that Databricks will only attempt to display the first 64,000 rows of data in dashboards. If the initial 64,000 rows are larger than 2,187 MB, it will fail to display anything2. While this is related to dashboard display, it highlights the importance of managing log sizes to avoid truncation.

If you need further assistance or have additional questions, feel free to ask! 😊

 
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!