Hyperopt Error: There are no evaluation tasks, cannot return argmin of task losses.

[ChingizK]

The trials succeed when the cell in the notebook is executed manually:

However, the same process fails when executed as a Workflow:

The error simply says that there's an issue with the objective function. However how can that be the case if I'm able to successfully run the exact same code when I manually run the notebook cell? The notebook run fails when triggered through a Workflow run.

Unfortunately, changing compute cluster had no effect either.

Task fails with the following error: Exception: There are no evaluation tasks, cannot return argmin of task losses.

[Kaniz_Fatma · Accepted solution]

Hi @ChingizK ,  Error message: "Exception: There are no evaluation tasks, cannot return argmin of task losses".

 This error occurs when there are no successful evaluations of the objective function. 

Possible reasons for the error when running the code manually but not through a Workflow:

  - Differences in execution environments or contexts

Suggestions to investigate:
  1. Check the execution context and environment variables to ensure they are the same when running manually and through the Workflow
  2. Verify that the objective function can handle unexpected inputs and return a valid loss value in all cases
  3. Confirm that the MLflow tracking URI is set correctly
  4. If using SparkTrials for distributed hyperparameter tuning, be aware that MLflow runs may not be nested under the parent run.

5. Use MLflow to debug the issue by checking the MLflow Runs Table for failed runs and examining the logs, parameters, and metrics for anomalies