I am currently working in a databricks notebook and using an ipywidgets.Output to display a pandas Dataframe. Because spark.DataFrame cannot be displayed in an ipywidgets.Output widget, I have been using:
import pandas as pd
import numpy as np
import ipywidgets
from IPython.display import display
df = pd.DataFrame(
np.random.randint(0, 5, size=(4, 4))
)
output = ipywidgets.Output()
with output:
display(df)
output
This has been working until recently on the 13.2 ML runtime. However, recently, I moved to a different cluster still using the same 13.2 ML runtime and am now getting a bunch of commas prepended to my html output (see screenshot). The larger the dataframe, the longer the number of commas. How can I fix this issue?