Looking to learn how you can use responsible AI toolkits on Databricks? Interested in learning how you can incorporate open source tools like SHAP and Fairlearn with Databricks?
I would recommend checking out this blog: Mitigating Bias in Machine Learning With SHAP and Fairlearn from my colleague @Sean Owen.
SHAP is a explainability framework used to determine the relative importance of features used in an ML model to give better transparency, especially when used with more complex models. Fairlearn is a framework to quantify and minimize bias inherit to datasets used an in ML model.
In addition to leveraging these frameworks as discussed in the article, out of the box Databricks automatically logs SHAP explainability plots with most ml frameworks using mlflow autolog and SHAP plots are automatically generated as part of Databricks AutoML notebook output. You can learn more at our Explainable AI home page.
Let us know how you plan to add Responsible AI frameworks to your ML workflows in the chat!
