InternalError, error message: The output of the notebook is too large.

[Chalki]

Hello guys. I am having the following annoying issue with DataBricks job executed from ADF. I have stopped all the logs, deleted all the print statements and display. And I am still getting this error. I don't understand what might cause it. I am attaching a screenshot from the driver log, which is showing that stdout log file is no more than 10 kb

[Chalki]

Forgot to mention it is triggered only from ADF, in DataBricks the execution goes without any troubles

[-werners-]

Do you happen to call other notebooks within that notebook?  Or have you entered a lot of markdown?

[Chalki]

None of those

[Debayan]

@Chalki Glad to hear it is working fine. Please let us know if any other help is needed. 

[UmaMahesh1]

Hi @Chalki 

Is the output of the notebook being passed into any other ADF activity ? If yes, check that output.
Also, can you give further details like the configuration you are using in ADB and in ADF for compute ?

[Chalki]

It is, but it is a short string, like a "success" message

[Debayan]

Hi, Could you please upload the log4j logs here if thats okay? Also, please tag @Debayan with your next comment, which will notify me. 

[Chalki]

Apologies for the late reply. I will try to upload the logs,cause it's been some time.

Interesting is how I solve it. I put all the code in a single cell and It is running since then without issues. Although it was just few cells with python code. I even increased the logging and I have no troubles for the time being