08-04-2022 07:33 PM
Similar issue here: https://stackoverflow.com/questions/71336374/randomforestclassifier-explainer-dashboard-output-in-da...
Actual output – Databricks Notebook
Expected Output – Jupyter Notebook
Reproducible Code Example
#pip install explainerdashboard
from sklearn.ensemble import RandomForestClassifier
from explainerdashboard import ClassifierExplainer, ExplainerDashboard
from explainerdashboard.datasets import titanic_survive, feature_descriptions
X_train, y_train, X_test, y_test = titanic_survive()
model = RandomForestClassifier(n_estimators=50, max_depth=10).fit(X_train, y_train)
explainer = ClassifierExplainer(model, X_test, y_test,
cats=[''Deck', 'Embarked'],
descriptions=feature_descriptions,
labels=['Not survived', 'Survived'])
ExplainerDashboard(explainer, mode = 'inline',
importances=False,
model_summary=False,
contributions=True,
whatif=False,
shap_dependence=False,
shap_interaction=False,
decision_trees=False).run()
10-06-2022 12:37 AM
Please do the following correction in your code then you will be able to get the ExplainerDashboard.
1) Set the Environment variables in Databricks Cluster
For example :
DASH_REQUEST_PATHNAME_PREFIX=/driver-proxy/o/4080082044610008/1004-091920-cxkidhkx/8888
Workspace ID : 4080082044610008
Cluster-ID: 1004-091920-cxkidhkx
Port number : 8888
2) Install the explainerdashboard library
%pip install explainerdashboard
3) Sample code for to validate the "dash"
from sklearn.ensemble import RandomForestClassifier
from explainerdashboard import ClassifierExplainer, ExplainerDashboard
from explainerdashboard.datasets import titanic_survive, feature_descriptions
X_train, y_train, X_test, y_test = titanic_survive()
model = RandomForestClassifier(n_estimators=50, max_depth=10).fit(X_train, y_train)
explainer = ClassifierExplainer(model, X_test, y_test,
cats=['Deck', 'Embarked'],
descriptions=feature_descriptions,
labels=['Not survived', 'Survived'])
ExplainerDashboard(explainer, mode = 'dash',
importances=False,
model_summary=False,
contributions=True,
whatif=False,
shap_dependence=False,
shap_interaction=False,
decision_trees=False).run(8888)
4) Dashboard URL
https://xxxxxxxx.databricks.com/driver-proxy/o/4080082044610008/1004-091920-cxkidhkx/8888
01-31-2023 02:18 AM
Hi @Abishek Subramanian, I experience the same issue and tried your solution step by step. It seems to work, but when I go to the dashboard URL it just says 'Loading...' indefinitely (I waited for an hour before killing it). I tried with both your example code and with my own dataset of only 360 records. Same issue... Any clues what could be wrong?
03-07-2024 07:18 AM
Can you share the cluster URL / spark config / sample code
10-06-2022 12:47 AM
06-22-2023 08:22 AM
07-14-2023 02:28 PM
Hi Abhishek,
I followed your steps, I am having in identifying the dashboard link. How do I figure out the first two words dbc-dp- for my cluster?
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group