cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

InternalError, error message: The output of the notebook is too large.

Chalki
New Contributor III

Hello guys. I am having the following annoying issue with DataBricks job executed from ADF. I have stopped all the logs, deleted all the print statements and display. And I am still getting this error. I don't understand what might cause it. I am attaching a screenshot from the driver log, which is showing that stdout log file is no more than 10 kb

8 REPLIES 8

Chalki
New Contributor III

Forgot to mention it is triggered only from ADF, in DataBricks the execution goes without any troubles

-werners-
Esteemed Contributor III

Do you happen to call other notebooks within that notebook?  Or have you entered a lot of markdown?

Chalki
New Contributor III

None of those

Debayan
Esteemed Contributor III

@Chalki Glad to hear it is working fine. Please let us know if any other help is needed. 

UmaMahesh1
Honored Contributor III

Hi @Chalki 

Is the output of the notebook being passed into any other ADF activity ? If yes, check that output.
Also, can you give further details like the configuration you are using in ADB and in ADF for compute ?

Uma Mahesh D

Chalki
New Contributor III

It is, but it is a short string, like a "success" message

Debayan
Esteemed Contributor III

Hi, Could you please upload the log4j logs here if thats okay? Also, please tag @Debayan with your next comment, which will notify me. 

Chalki
New Contributor III

Apologies for the late reply. I will try to upload the logs,cause it's been some time.

Interesting is how I solve it. I put all the code in a single cell and It is running since then without issues. Although it was just few cells with python code. I even increased the logging and I have no troubles for the time being

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group