Facing pydantic errors at runtime in MLFlow ChatAgent

[Rajat-TVSM]

Code works fine locally but deployment in serving endpoint gives me below error at runtime:

{"error_code": "BAD_REQUEST", "message": "Encountered an unexpected error while evaluating the model. Verify that the input is compatible with the model for inference. Error ''pydantic_core._pydantic_core.ValidationInfo' object has no attribute 'content''", "stack_trace": "Traceback (most recent call last):\n  File \"/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/mlflowserving/scoring_server/__init__.py\", line 834, in transformation\n    (raw_predictions, databricks_output, fs_metrics) = _score_model(\n  File \"/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/mlflowserving/scoring_server/__init__.py\", line 509, in _score_model\n    prediction, fs_metrics = score_model_maybe_with_fs_metrics(model, data, params)\n  File \"/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/mlflowserving/scoring_server/scoring_server_utils.py\", line 543, in score_model_maybe_with_fs_metrics\n    return (score_pyfunc_model(model, data, params), None)\n  File \"/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/mlflowserving/scoring_server/scoring_server_utils.py\", line 548, in score_pyfunc_model\n    return model.predict(data, params=params)\n  File \"/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/mlflow/pyfunc/__init__.py\", line 840, in predict\n    return self._predict(data, params)\n  File \"/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/mlflow/pyfunc/__init__.py\", line 890, in _predict\n    return self._predict_fn(data, params=params)\n  File \"/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/mlflow/pyfunc/loaders/chat_agent.py\", line 97, in predict\n    messages, context, custom_inputs = self._convert_input(model_input)\n  File \"/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/mlflow/pyfunc/loaders/chat_agent.py\", line 64, in _convert_input\n    messages = [ChatAgentMessage(**message) for message in dict_input.get(\"messages\", [])]\n  File \"/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/mlflow/pyfunc/loaders/chat_agent.py\", line 64, in <listcomp>\n    messages = [ChatAgentMessage(**message) for message in dict_input.get(\"messages\", [])]\n  File \"/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/pydantic/main.py\", line 250, in __init__\n    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)\n  File \"/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/mlflow/types/agent.py\", line 57, in check_content_and_tool_calls\n    content = values.content\nAttributeError: 'pydantic_core._pydantic_core.ValidationInfo' object has no attribute 'content'. Did you mean: 'context'?\n"}

[szymon_dybczak · Accepted solution]

Hi @Rajat-TVSM ,

Below someone had the exact same issue. The solution is to use newer version of mlflow >= 3.2
The problem is caused because there's been an update in pydantic. They changed a lot of things in pydantic v2. So not all APIs are compatibile between each other.

mlflow Dependency Error with pydantic - Databricks Community - 127017

[Rajat-TVSM]

Thanks, I upgraded to 3.4.0

[szymon_dybczak]

No problem @Rajat-TVSM . Super cool that it worked 🙂