Resolved! Limiting parallelism when external APIs are invoked (i.e. mlflow)
We are applying a groupby operation to a pyspark.sql.Dataframe and then on each group train a single model for mlflow. We see intermittent failures because the MLFlow server replies with a 429, because of too many requests/sWhat are the best practice...
- 5738 Views
- 7 replies
- 3 kudos
Latest Reply
To me it's already resolved through professional services. The question I do have is how useful is this community if people with the right background aren't here, and if it takes a month to get a no-answer.
- 3 kudos