- 4392 Views
- 2 replies
- 2 kudos
MLFlow model loading taking long time and "model serving" failing during init
I am trying to load a simple Minmaxscaler model that was logged as a run through spark's ML Pipeline api for reuse. On average it takes 40+ seconds just to load the model with the following example: This is fine and the model transforms my data corre...
- 4392 Views
- 2 replies
- 2 kudos
- 2 kudos
Hello,Any solutions found for this issue?I'm serving up a large number of models at a time, but since we converted to PySpark (due to our data demands), the mlflow.spark.load_model() is taking hours.Part of the reason to switch to spark was to help w...
- 2 kudos
- 3503 Views
- 4 replies
- 1 kudos
Resolved! Tuning `CrossValidator` spark job performance
I am running a 3-fold cross validation of an ML pipeline that utilizes `GBTClassifier` as the final step. It takes 18 hours to run and I am looking for feedback into how to improve the performance as I expect this to go faster.For context here is the...
- 3503 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @Assaad Mrad​ , Just a friendly follow-up. Do you still need help, or @Chris Chalcraft​ 's response help you to find the solution? Please let us know.
- 1 kudos