topic Re: How to display SHAP plots? in Data Engineering

How to display SHAP plots?

ArielHerrera — Wed, 13 Feb 2019 19:29:02 GMT

I am looking to display SHAP plots, here is the code:

import xgboost import shap

shap.initjs() # load JS visualization code to notebookX,y = shap.datasets.boston() # train XGBoost model

model = xgboost.train({"learning_rate": 0.01}, xgboost.DMatrix(X, label=y), 100)

explainer = shap.TreeExplainer(model) # explain the model's predictions using SHAP values

shap_values = explainer.shap_values(X)

shap_explain = shap.force_plot(explainer.expected_value, shap_values[0,:], X.iloc[0,:]) # visualize the first prediction's explanation

displayHTML(shap_explain.data) # display plot

However I am receiving the following error:

Any help greatly appreciated!

Re: How to display SHAP plots?

RimmaShafikova — Mon, 18 Feb 2019 06:26:51 GMT

I was finally able to plot them in DataBricks without js

shap_display = shap.force_plot(explainer.expected_value[1], shap_value[1], feat_x.iloc[0, :], matplotlib=True)

display(shap_display)

Re: How to display SHAP plots?

ArielHerrera — Thu, 21 Feb 2019 15:27:29 GMT

It also needs to be run on an earlier version of matplotlib

matplotlib: 1.5.3

Re: How to display SHAP plots?

sean_owen — Wed, 19 Jun 2019 01:21:59 GMT

For what it's worth, I got force plots to work in Databricks by copying the bundle.js Javascript from the SHAP package into a hidden cell in the notebook. See https://databricks.com/blog/2019/06/17/detecting-bias-with-shap.html and the notebook that accompanies it.

Re: How to display SHAP plots?

Vinhdqvinh87 — Tue, 07 Jan 2020 06:23:26 GMT

It is quite good but only works for a single explanation. If you want to apply to multiple samples, force_plot has not been supported yet as in Jan 2020.

Re: How to display SHAP plots?

lrnzcig — Wed, 05 Feb 2020 16:07:29 GMT

As @Vinh dqvinh87 noted, the accepted solution only works for

force_plot

. For other plots, the following trick works for me:

import matplotlib.pyplot as plt
p = shap.summary_plot(shap_values, test_df, show=False) 
display(p)