Databricks MlFlow Error: Timed out while evaluating the model.

[Shumi8]

Hi everyone,

I am using databricks and mlflow to create a model and then register it as a serving endpoint. Sometimes the models takes more than 2 minutes to run and after 2 minutes it gives a timeout error:

Timed out while evaluating the model. Verify that the model evaluates within the timeout.

These are the three parameters I have identified which I think would help solve the issue:

MLFLOW_SCORING_SERVER_REQUEST_TIMEOUT
MLFLOW_REQUIREMENTS_INFERENCE_TIMEOUT
MLFLOW_HTTP_REQUEST_TIMEOUT

But I can't seem to find out where to include these in the code. The MLFLow documentation doesn't seem to help.

I have tried to include the parameters in the conda env but that doesn't help.

[case-k]

We also facing same issue. We set under the environment variable from databricks model serving endpoint when we create from databricks console. But it is not working and still get this error

MLFLOW_SCORING_SERVER_REQUEST_TIMEOUT=3600
MLFLOW_REQUIREMENTS_INFERENCE_TIMEOUT=3600
MLFLOW_HTTP_REQUEST_TIMEOUT=3600

Error: Timed out while evaluating the model. Verify that the model evaluates within the timeout.

[case-k]

I have checked ML flow code and error message is also different. So I think this is the limitation of databricks side.

except requests.exceptions.Timeout as to:
raise MlflowException(
f"API request to {url} failed with timeout exception {to}."
" To increase the timeout, set the environment variable "
f"{MLFLOW_HTTP_REQUEST_TIMEOUT!s} to a larger value."

https://github.com/mlflow/mlflow

DBX limitation
>Model execution duration Per request 120 seconds
https://docs.databricks.com/en/machine-learning/model-serving/model-serving-limits.html

[Aman-Patkar]

Hi, 
I want to know the maximum timeout for all of these env variables.

[Aman-Patkar]

Hi, Did you get the solution for this? This timeout issue.