topic Re: Databricks Notebook Rendering Issue: IPython.lib.display.IFrame in Machine Learning

Databricks Notebook Rendering Issue: IPython.lib.display.IFrame

SR_71 — Fri, 05 Aug 2022 02:33:29 GMT

Actual output – Databricks Notebook

Expected Output – Jupyter Notebook

Reproducible Code Example

#pip install explainerdashboard

from sklearn.ensemble import RandomForestClassifier

from explainerdashboard import ClassifierExplainer, ExplainerDashboard

from explainerdashboard.datasets import titanic_survive, feature_descriptions

X_train, y_train, X_test, y_test = titanic_survive()

model = RandomForestClassifier(n_estimators=50, max_depth=10).fit(X_train, y_train)

explainer = ClassifierExplainer(model, X_test, y_test,

cats=[''Deck', 'Embarked'],

descriptions=feature_descriptions,

labels=['Not survived', 'Survived'])

ExplainerDashboard(explainer, mode = 'inline',

importances=False,

model_summary=False,

contributions=True,

whatif=False,

shap_dependence=False,

shap_interaction=False,

decision_trees=False).run()

Re: Databricks Notebook Rendering Issue: IPython.lib.display.IFrame

Abishek — Thu, 06 Oct 2022 07:37:28 GMT

Please do the following correction in your code then you will be able to get the ExplainerDashboard.

1) Set the Environment variables in Databricks Cluster

For example :

DASH_REQUEST_PATHNAME_PREFIX=/driver-proxy/o/4080082044610008/1004-091920-cxkidhkx/8888

Workspace ID : 4080082044610008

Cluster-ID: 1004-091920-cxkidhkx

Port number : 8888

2) Install the explainerdashboard library

%pip install explainerdashboard

3) Sample code for to validate the "dash"

from sklearn.ensemble import RandomForestClassifier

from explainerdashboard import ClassifierExplainer, ExplainerDashboard

from explainerdashboard.datasets import titanic_survive, feature_descriptions

X_train, y_train, X_test, y_test = titanic_survive()

model = RandomForestClassifier(n_estimators=50, max_depth=10).fit(X_train, y_train)

explainer = ClassifierExplainer(model, X_test, y_test,

cats=['Deck', 'Embarked'],

descriptions=feature_descriptions,

labels=['Not survived', 'Survived'])

ExplainerDashboard(explainer, mode = 'dash',

importances=False,

model_summary=False,

contributions=True,

whatif=False,

shap_dependence=False,

shap_interaction=False,

decision_trees=False).run(8888)

4) Dashboard URL

https://xxxxxxxx.databricks.com/driver-proxy/o/4080082044610008/1004-091920-cxkidhkx/8888

Re: Databricks Notebook Rendering Issue: IPython.lib.display.IFrame

Abishek — Thu, 06 Oct 2022 07:47:59 GMT

Environment variables:

Model Explainer:

Re: Databricks Notebook Rendering Issue: IPython.lib.display.IFrame

SophiaEvt — Tue, 31 Jan 2023 10:18:48 GMT

Hi @Abishek Subramanian, I experience the same issue and tried your solution step by step. It seems to work, but when I go to the dashboard URL it just says 'Loading...' indefinitely (I waited for an hour before killing it). I tried with both your example code and with my own dataset of only 360 records. Same issue... Any clues what could be wrong?

Re: Databricks Notebook Rendering Issue: IPython.lib.display.IFrame

Abishek — Thu, 22 Jun 2023 15:22:41 GMT

@Sophia Evtimova Once you add the Environment variables under the spark config you will get a dashboard

Re: Databricks Notebook Rendering Issue: IPython.lib.display.IFrame

ChanduBhujang — Fri, 14 Jul 2023 21:28:43 GMT

Hi Abhishek,

I followed your steps, I am having in identifying the dashboard link. How do I figure out the first two words dbc-dp- for my cluster?

Re: Databricks Notebook Rendering Issue: IPython.lib.display.IFrame

Abishek — Thu, 07 Mar 2024 15:18:31 GMT

Can you share the cluster URL / spark config / sample code