When doing hyperparameter tuning with Hyperopt, when should I use SparkTrials? Does it work with both single-machine ML (like sklearn) and distributed ML (like Apache Spark ML)?

Joseph_B
Databricks Employee
Databricks Employee

‎06-09-2021 05:51 PM

I want to know how to use Hyperopt in different situations:

  • Tuning a single-machine algorithm from scikit-learn or single-node TensorFlow
  • Tuning a distributed algorithm from Spark ML or distributed TensorFlow / Horovod
0 Kudos