I'm running a model using LGBM with Spark, within Pipeline, but when I log it to MlFlow using the log_model feature store function, I can't replicate the pipeline in prediction.
When I execute:
predict = fs.score_batch(logged_model, df_pred)
display(predict); I have this error:
