How to handle

marsjuli — Sun, 24 Oct 2021 17:39:07 GMT

Some libraries have intermediate IPython HTML-objects returned to the notebook cell output.

Since this happens during training a machine learning model the statements are typically buried within in the library so I cannot easily interfere. (e.g. in order overwrite the output statement to use the databricks function display() )

Is there - or will there be - an option to handle HTML-objects from IPython ?

Examplary versions:

DB-Runtime 10.0 ML (occurs in 8.X and 9.X as well)

ipython 7.22.0

ipython-genutils0.2.0

tqdm 4.59.0

mlflow 1.20.2

transformers 4.9.2

Example Output:

Re: How to handle

marsjuli — Wed, 17 Nov 2021 08:30:57 GMT

Hi @Kaniz Fatma ,

thanks for showing me the link. This helps if you are in control of the generated html-object. If the html-content comes from a library, that is where the problems start, because I cannot wrap displayHTML().

(I can of course look for that line and rebuild my own databricks-compatible version of the lib, but this would be improvisation, not solving the issue)

I am just worried that the number of python libraries will increase that use html-objects deep down. Surely there can be a way to capture this in the stack and handle it better than enforcing a call on another display method. I am sure this needs to be coordinated with the frontend developers of the databricks UI and the backend devs doing the Python-engine.

Is it possible to add such a configuration for a specific function of a library instead?

https://github.com/databrickslabs/jupyterlab-integration/blob/753882c37ead89013f302863d1548eef27e0baaf/databrickslabs_jupyterlab/connect.py#L480

topic Re: How to handle <IPython.core.display.HTML object> in Data Engineering

How to handle

Re: How to handle