topic what does this error in hyperopt mean, One error that users commonly encounter with Hyperopt is: There are no evaluation tasks, cannot return argmin of task losses.? in Data Engineering

what does this error in hyperopt mean, One error that users commonly encounter with Hyperopt is: There are no evaluation tasks, cannot return argmin of task losses.?

User16789201666 — Wed, 23 Jun 2021 14:46:54 GMT

This means that no trial completed successfully. This almost always means that there is a bug in the objective function, and every invocation is resulting in an error. See the error output in the logs for details. In Databricks, the underlying error is surfaced for easier debugging.

It can also arise if the model fitting process is not prepared to deal with missing / NaN values, and is always returning a NaN loss.

https://databricks.com/blog/2021/04/15/how-not-to-tune-your-model-with-hyperopt.html

Re: what does this error in hyperopt mean, One error that users commonly encounter with Hyperopt is: There are no evaluation tasks, cannot return argmin of task losses.?

tj-cycyota — Wed, 23 Jun 2021 15:03:18 GMT

The fmin function should be of the form:

def evaluate_hyperparams(params):
  """
  This method will be passed to `hyperopt.fmin()`.  It fits and evaluates the model using the given hyperparameters to get the validation loss.
  
  :param params: This dict of parameters specifies hyperparameter values to test.
  :return: dict with fields 'loss' (scalar loss) and 'status' (success/failure status of run)
  """
  
  # Train the model
  model, score = train(params)
        
  return {'loss': score, 'status': STATUS_OK}

The `train` function can be any arbitrary function that trains a model and returns both the fitted model class and the score metric of interest (RMSE, for example).