cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks Notebook Rendering Issue: IPython.lib.display.IFrame

SR_71
New Contributor II

Similar issue here: https://stackoverflow.com/questions/71336374/randomforestclassifier-explainer-dashboard-output-in-da...

Actual output – Databricks Notebookimage 

Expected Output – Jupyter Notebookimage 

Reproducible Code Example

#pip install explainerdashboard

from sklearn.ensemble import RandomForestClassifier

from explainerdashboard import ClassifierExplainer, ExplainerDashboard

from explainerdashboard.datasets import titanic_survive, feature_descriptions

X_train, y_train, X_test, y_test = titanic_survive()

model = RandomForestClassifier(n_estimators=50, max_depth=10).fit(X_train, y_train)

explainer = ClassifierExplainer(model, X_test, y_test,

                              cats=[''Deck', 'Embarked'],

                              descriptions=feature_descriptions,

                              labels=['Not survived', 'Survived'])

ExplainerDashboard(explainer, mode = 'inline',

                  importances=False,

                  model_summary=False,

                  contributions=True,

                  whatif=False,

                  shap_dependence=False,

                  shap_interaction=False,

                  decision_trees=False).run()

6 REPLIES 6

Abishek
Valued Contributor
Valued Contributor

Please do the following correction in your code then you will be able to get the ExplainerDashboard.

1) Set the Environment variables in Databricks Cluster

For example :

DASH_REQUEST_PATHNAME_PREFIX=/driver-proxy/o/4080082044610008/1004-091920-cxkidhkx/8888

Workspace ID : 4080082044610008 

Cluster-ID: 1004-091920-cxkidhkx 

Port number : 8888

2) Install the explainerdashboard library

%pip install explainerdashboard

3) Sample code for to validate the "dash" 

from sklearn.ensemble import RandomForestClassifier

from explainerdashboard import ClassifierExplainer, ExplainerDashboard

from explainerdashboard.datasets import titanic_survive, feature_descriptions

X_train, y_train, X_test, y_test = titanic_survive()

model = RandomForestClassifier(n_estimators=50, max_depth=10).fit(X_train, y_train)

explainer = ClassifierExplainer(model, X_test, y_test,

   cats=['Deck', 'Embarked'],

   descriptions=feature_descriptions,

   labels=['Not survived', 'Survived'])

ExplainerDashboard(explainer, mode = 'dash',

   importances=False,

   model_summary=False,

   contributions=True,

   whatif=False,

   shap_dependence=False,

   shap_interaction=False,

   decision_trees=False).run(8888)

4) Dashboard URL

https://xxxxxxxx.databricks.com/driver-proxy/o/4080082044610008/1004-091920-cxkidhkx/8888

SophiaEvt
New Contributor II

Hi @Abishek Subramanian​, I experience the same issue and tried your solution step by step. It seems to work, but when I go to the dashboard URL it just says 'Loading...' indefinitely (I waited for an hour before killing it). I tried with both your example code and with my own dataset of only 360 records. Same issue... Any clues what could be wrong?

Abishek
Valued Contributor
Valued Contributor

Can you share the cluster URL / spark config / sample code 

Abishek
Valued Contributor
Valued Contributor

Environment variables:

Environment variablesModel Explainer:

Model Explainer 

Abishek
Valued Contributor
Valued Contributor

@Sophia Evtimova​  Once you add the Environment variables under the spark config you will get a dashboard

Environment variablesScreenshot 2023-06-22 at 8.44.11 PM

ChanduBhujang
New Contributor II

Hi Abhishek, 

I followed your steps, I am having in identifying the dashboard link. How do I figure out the first two words dbc-dp- for my cluster? 

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!