Databricks Community

ianchenmu · 10-13-2022

I'm training a ML model (e.g., XGboost) and I have a large combination of 5 hyperparameters, say each parameter has 5 candidates, it will be 5^5 = 3,125 combos.Now I want to do parallelization for the grid search on all the hyperparameter combos for ...

ianchenmu · 10-13-2022

Thanks @Joseph Kambourakis ! It seems we could do the distributed XGBoost training using the num_workers regards to how many workers in the cluster. But can we also speed up by setting a parameter utilizing the number of cores in the cluster?

Databricks Community

User Stats

User Activity

Parallelization in training machine learning models using MLFlow

Re: Parallelization in training machine learning models using MLFlow