cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Best way to make a scrollable DataFrame in a ipywidgets.Output?

yhyhy3
New Contributor III

I have a pretty complex Jupyter widgets UI in a databricks notebook that has a dataframe that

1. will be modified by some Jupyter widget callbacks

2. needs to be displayed to the user and updated as it is modified

3. is large and needs to support vertical scrolling

Since callbacks cannot directly access cell output, I have to use a widget (either ipywidgets.Output or ipywidgets.HTML) to display the dataframe.

Here are the things I tried (finally succeeding with the 4th attempt/workaround):

# Setup
import ipywidgets as widgets
import pandas as pd
pd.set_option("display.max_rows", 1000)
df = pd.DataFrame([[1, 2]] * 50)
 
# Attempt 1 - Use widgets.Output with databricks dataframe formatting
a1 = widgets.Output(layout={"scroll": "auto", "height": "300px"})
with a1:
    display(df)
display(a1)
##### RESULT ######
# <Databricks Output (not supported in output widgets)>
################
 
# Attempt 2 - Use widgets.Output with pandas dataframe formatting
from IPython.display import HTML
a2 = widgets.Output(layout={"scrollY": "auto", "height": "300px"})
with a2:
    display(HTML(df.to_html()))
display(a2)
##### RESULT ######
# Non-Databricks formatted pandas DataFrame but does not limit the height
# and there is no scrolling
################
 
# Attempt 3 - Use widgets.HTML with pandas dataframe formatting
from IPython.display import HTML
a3 = widgets.HTML(layout={"scroll": "auto", "height": "100px"})
a3.value = df.to_html()
display(a3)
##### RESULT ######
# Non-Databricks formatted pandas DataFrame and limit the height,
# but there is no scrolling, so the dataframe is truncated to a few rows.
################
 
# Attempt 4 - Use widgets.Output with pandas dataframe formatting and
#                       wrap it in a widgets.VBox
a4 = widgets.Output()
 
with a4:
    display(HTML(df.to_html()))
display(widgets.VBox(children=[a4], layout={"height": "300px", "scroll": "auto"}))
##### RESULT ######
# SUCCESS!!
# Non-Databricks formatted pandas DataFrame with scrolling enabled
################

Clearly, using widgets.VBox wrapping widgets.Output with a workaround to use the pandas to_html instead of Databrick's to_html function is not ideal. Is there a better way to make a scrollable DataFrame in a Databricks notebook? Or is this the best workaround until "<Databricks Output (not supported in output widgets)>" is fixed?

1 REPLY 1

Anonymous
Not applicable

Hi @Yushi Homma​ 

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance! 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group