How to display shap waterfall plot

[invalidargument]

Hi,

I have managed to display force plot for a single observation using the advice from this thread:

Solved: How to display SHAP plots? - Databricks - 28315

But is there anyway to display the newer "waterfall"-plot 

shap.plots.waterfall — SHAP latest documentation

When running

shap.plots.waterfall(shap_values[0])

 I get output like

Out[105]: <Figure size 576x468 with 3 Axes>

 I have tried calling:

• plt.show()
• display()
• display(plt.gcf())

But same result. Any advice? Is it supposed to work?

[Kaniz]

Hi @invalidargument , To display the SHAP waterfall plot in Databricks, you can use the display() function to show the plot in the Databricks Notebook.

Here's an example:

import shap
import pandas as pd

explainer = shap.Explainer(model)
shap_values = explainer(data)

# Create the waterfall plot and show it using display()
shap.plots.waterfall(shap_values[0])
display()

The display() function can display a wide range of data types including Pandas DataFrames, Matplotlib figures, and HTML.

Make sure that you are importing the display() function from Databricks and not the IPython.display() function so that it works correctly in Databricks.

Also, make sure that you are running your code in a Databricks Notebook with mlflow.shap.autolog() or mlflow.shap.log_explanation() before calling the waterfall() method. This ensures that SHAP values are available for the plot.

[invalidargument]

Thank you for the swift response. 

I made a minimal example and it does work as you said. However when I try with my own model it does not work, the only output is

<Figure size 576x468 with 3 Axes>

I tried to save the figure as a file and then I do get a plot, but it does look a little broken. 

p = shap.plots.waterfall(shap_values[0],show=False)

p.savefig("plot.png")

My guess is that the plot shap outputs for my shap_values is broken,too large or has some other problem so databricks fails to render it.