Databricks MlFlow Error: Timed out while evaluating the model.

[Shumi8]

Hi everyone,

I am using databricks and mlflow to create a model and then register it as a serving endpoint. Sometimes the models takes more than 2 minutes to run and after 2 minutes it gives a timeout error:

Timed out while evaluating the model. Verify that the model evaluates within the timeout.

These are the three parameters I have identified which I think would help solve the issue:

MLFLOW_SCORING_SERVER_REQUEST_TIMEOUT
MLFLOW_REQUIREMENTS_INFERENCE_TIMEOUT
MLFLOW_HTTP_REQUEST_TIMEOUT

But I can't seem to find out where to include these in the code. The MLFLow documentation doesn't seem to help.

I have tried to include the parameters in the conda env but that doesn't help.

[Kaniz]

Hi @Shumi8, When dealing with timeout issues in MLflow, it’s essential to configure the relevant parameters to ensure your server remains responsive.

Let’s address this step by step:

1. MLFLOW_SCORING_SERVER_REQUEST_TIMEOUT:

   • This parameter controls the timeout for MLflow’s Model Scoring server requests.
   • To set it, you can use an environment variable. For example, in your deployment environment, you can set:

     export MLFLOW_SCORING_SERVER_REQUEST_TIMEOUT=120  # Set to 2 minutes (in seconds)
     

   • MLflow will pick up this value during runtime.

2. MLFLOW_REQUIREMENTS_INFERENCE_TIMEOUT:

   • This parameter is related to inference requirements.
   • Similar to the previous one, you can set it as an environment variable:

     export MLFLOW_REQUIREMENTS_INFERENCE_TIMEOUT=120
     

3. MLFLOW_HTTP_REQUEST_TIMEOUT:

   • This parameter affects HTTP requests made by MLflow.
   • Set it using an environment variable:

     export MLFLOW_HTTP_REQUEST_TIMEOUT=120