Best way to make a scrollable DataFrame in a ipywidgets.Output?

yhyhy3 — Wed, 24 May 2023 01:28:30 GMT

I have a pretty complex Jupyter widgets UI in a databricks notebook that has a dataframe that

1. will be modified by some Jupyter widget callbacks

2. needs to be displayed to the user and updated as it is modified

3. is large and needs to support vertical scrolling

Since callbacks cannot directly access cell output, I have to use a widget (either ipywidgets.Output or ipywidgets.HTML) to display the dataframe.

Here are the things I tried (finally succeeding with the 4th attempt/workaround):

# Setup
import ipywidgets as widgets
import pandas as pd
pd.set_option("display.max_rows", 1000)
df = pd.DataFrame([[1, 2]] * 50)
 
# Attempt 1 - Use widgets.Output with databricks dataframe formatting
a1 = widgets.Output(layout={"scroll": "auto", "height": "300px"})
with a1:
    display(df)
display(a1)
##### RESULT ######
# <Databricks Output (not supported in output widgets)>
################
 
# Attempt 2 - Use widgets.Output with pandas dataframe formatting
from IPython.display import HTML
a2 = widgets.Output(layout={"scrollY": "auto", "height": "300px"})
with a2:
    display(HTML(df.to_html()))
display(a2)
##### RESULT ######
# Non-Databricks formatted pandas DataFrame but does not limit the height
# and there is no scrolling
################
 
# Attempt 3 - Use widgets.HTML with pandas dataframe formatting
from IPython.display import HTML
a3 = widgets.HTML(layout={"scroll": "auto", "height": "100px"})
a3.value = df.to_html()
display(a3)
##### RESULT ######
# Non-Databricks formatted pandas DataFrame and limit the height,
# but there is no scrolling, so the dataframe is truncated to a few rows.
################
 
# Attempt 4 - Use widgets.Output with pandas dataframe formatting and
#                       wrap it in a widgets.VBox
a4 = widgets.Output()
 
with a4:
    display(HTML(df.to_html()))
display(widgets.VBox(children=[a4], layout={"height": "300px", "scroll": "auto"}))
##### RESULT ######
# SUCCESS!!
# Non-Databricks formatted pandas DataFrame with scrolling enabled
################

Clearly, using widgets.VBox wrapping widgets.Output with a workaround to use the pandas to_html instead of Databrick's to_html function is not ideal. Is there a better way to make a scrollable DataFrame in a Databricks notebook? Or is this the best workaround until "<Databricks Output (not supported in output widgets)>" is fixed?

Re: Best way to make a scrollable DataFrame in a ipywidgets.Output?

Anonymous — Mon, 29 May 2023 00:40:01 GMT

Hi @Yushi Homma

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!

topic Best way to make a scrollable DataFrame in a ipywidgets.Output? in Data Engineering

Best way to make a scrollable DataFrame in a ipywidgets.Output?

Re: Best way to make a scrollable DataFrame in a ipywidgets.Output?