Hi Databricks support,
I'm training an ML model using mlflow on DBR 13.3 LTS ML, Spark 3.4.1 using databricks.automl_runtime 0.2.17 and databricks.automl 1.20.3, with shap 0.45.1. My training data has two float-type columns with three or fewer unique values, which automl flags for one-hot encoding. My training experiment finishes without error. When I examined the notebook of the best-performing model, I toggled `shap_enabled` to `True` to see the shap values. However, in the cell that produces shap values, the following error is produced: "TypeError: no supported conversion for types: (dtype('O'),)" (full traceback attached).
From my debugging, I believe the error occurs because the one-hot encoding of the two aforementioned columns fails, leading to object columns being passed to `scipy.sparse.csr_matrix` within the shap package. Indeed, when I go into the training notebook and try to fit the one-hot encoder to the two columns, I get the message "Warning: No categorical columns found. Calling 'transform' will only return input data."
Let me know if a full reprex is needed, and the best way to supply it.
Thanks in advance!
