Databricks Notebook Rendering Issue: IPython.lib.display.IFrame

[SR_71]

Similar issue here: https://stackoverflow.com/questions/71336374/randomforestclassifier-explainer-dashboard-output-in-da...

Actual output – Databricks Notebook 

Expected Output – Jupyter Notebook 

Reproducible Code Example

#pip install explainerdashboard

from sklearn.ensemble import RandomForestClassifier

from explainerdashboard import ClassifierExplainer, ExplainerDashboard

from explainerdashboard.datasets import titanic_survive, feature_descriptions

X_train, y_train, X_test, y_test = titanic_survive()

model = RandomForestClassifier(n_estimators=50, max_depth=10).fit(X_train, y_train)

explainer = ClassifierExplainer(model, X_test, y_test,

                              cats=[''Deck', 'Embarked'],

                              descriptions=feature_descriptions,

                              labels=['Not survived', 'Survived'])

ExplainerDashboard(explainer, mode = 'inline',

                  importances=False,

                  model_summary=False,

                  contributions=True,

                  whatif=False,

                  shap_dependence=False,

                  shap_interaction=False,

                  decision_trees=False).run()

[Abishek]

Please do the following correction in your code then you will be able to get the ExplainerDashboard.

1) Set the Environment variables in Databricks Cluster

For example :

DASH_REQUEST_PATHNAME_PREFIX=/driver-proxy/o/4080082044610008/1004-091920-cxkidhkx/8888

Workspace ID : 4080082044610008 

Cluster-ID: 1004-091920-cxkidhkx 

Port number : 8888

2) Install the explainerdashboard library

%pip install explainerdashboard

3) Sample code for to validate the "dash" 

from sklearn.ensemble import RandomForestClassifier

from explainerdashboard import ClassifierExplainer, ExplainerDashboard

from explainerdashboard.datasets import titanic_survive, feature_descriptions

X_train, y_train, X_test, y_test = titanic_survive()

model = RandomForestClassifier(n_estimators=50, max_depth=10).fit(X_train, y_train)

explainer = ClassifierExplainer(model, X_test, y_test,

   cats=['Deck', 'Embarked'],

   descriptions=feature_descriptions,

   labels=['Not survived', 'Survived'])

ExplainerDashboard(explainer, mode = 'dash',

   importances=False,

   model_summary=False,

   contributions=True,

   whatif=False,

   shap_dependence=False,

   shap_interaction=False,

   decision_trees=False).run(8888)

4) Dashboard URL

https://xxxxxxxx.databricks.com/driver-proxy/o/4080082044610008/1004-091920-cxkidhkx/8888

[SophiaEvt]

Hi @Abishek Subramanian​, I experience the same issue and tried your solution step by step. It seems to work, but when I go to the dashboard URL it just says 'Loading...' indefinitely (I waited for an hour before killing it). I tried with both your example code and with my own dataset of only 360 records. Same issue... Any clues what could be wrong?

[Abishek]

Can you share the cluster URL / spark config / sample code 

[Abishek]

Environment variables:

Model Explainer:

 

[Abishek]

@Sophia Evtimova​  Once you add the Environment variables under the spark config you will get a dashboard

[ChanduBhujang]

Hi Abhishek, 

I followed your steps, I am having in identifying the dashboard link. How do I figure out the first two words dbc-dp- for my cluster?