Databricks Community

Edmondo · 01-17-2022

We are applying a groupby operation to a pyspark.sql.Dataframe and then on each group train a single model for mlflow. We see intermittent failures because the MLFlow server replies with a 429, because of too many requests/s What are the best pract...

Edmondo · 12-31-2021

As I am moving my first steps within the Databricks Machine Learning Workspace, I am getting confused by some features that by "documentation" seem to overlap. Does autolog for spark on mlflow provide different tracking than using a training set crea...

Edmondo · 02-24-2022

To me it's already resolved through professional services. The question I do have is how useful is this community if people with the right background aren't here, and if it takes a month to get a no-answer.

Edmondo · 02-24-2022

Yes. I confirm there is no sign of an answer to my question: "how to limit parallelism"

Edmondo · 02-24-2022

That's why I asked how to limit parallelism, not what a 429 error mean  That one I already know. We got an answer from our RA that we pay as professional customers, it looks like this community is pretty useless if the experts do not partecipate 

Edmondo · 01-17-2022

Thanks, the limits are the same by documentation for AWS (I had checked that). So there are three options: udf can be applied at maximum at 7 in parallel (how do I do it?)mlflow calls must be queued (again, how do I add a stateful queue across all c...

Databricks Community

User Stats

User Activity

Limiting parallelism when external APIs are invoked (i.e. mlflow)

MlFlow and Feature Store: mlflow.spark.autolog, using feature store on Databricks, FeatureStoreClient.log_model()?

Re: Limiting parallelism when external APIs are invoked (i.e. mlflow)

Re: Limiting parallelism when external APIs are invoked (i.e. mlflow)

Re: Limiting parallelism when external APIs are invoked (i.e. mlflow)

Re: Limiting parallelism when external APIs are invoked (i.e. mlflow)