Vectorsearch ConnectionResetError Max retries exceeded

RobinK — Wed, 24 Jul 2024 12:21:38 GMT

Hi,

we are serving a unity catalog langchain model with databricks model serving. When I run the predict() function on the model in a notebook, I get the expected output. But when I query the served model, errors occur in the service logs:

Error message:

2024-07-24T12:05:38.+0000 WARNING urllib3.connectionpool Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectionResetError(104, 'Connection reset by peer')': /api/2.0/<workspace-id>/vector-search/endpoints/my_vs_endpoint/indexes/my_vs_index/query

2024-07-24T12:05:44.+0000 WARNING mlflowserving.scoring_server LangChain invocation failed with MlflowException: 1 tasks failed. Errors: {0: 'error: ConnectionError(MaxRetryError("HTTPSConnectionPool(host=\'6cb7f5a6-3135-4280-987f-9758c9fa3753.vector-search.cloud.databricks.com\', port=443): Max retries exceeded with url:/api/2.0/<workspace-id>/vector-search/endpoints/my_vs_endpoint/indexes/my_vs_index/query (Caused by ProtocolError(\'Connection aborted.\', ConnectionResetError(104, \'Connection reset by peer\')))")) urllib3.exceptions.ProtocolError: (\'Connection aborted.\', ConnectionResetError(104, \'Connection reset by peer\'))\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/requests/adapters.py", line 667, in send\n resp = conn.urlopen(\n File "/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 873, in urlopen\n

Request:

{
"dataframe_split":
{
"columns":["payload"],
"data": ["my_data"]
}
}

Response:

{
"predictions": null
}

Working code in my notebook:

import mlflow

import pandas as pd

my_model = mlflow.pyfunc.load_model(model_info.model_uri)

my_model.predict(

pd.DataFrame(columns=["payload"], data=["my_data"]))

The served model consist of a query to a databricks vectorsearch index. I should be later extended by a llm.

req.txt:

ipykernel==6.29.4

numpy==1.26.4

databricks-connect==14.3.2

databricks-sdk==0.28.0

databricks-vectorsearch==0.40

langchain==0.2.10

langchain-community==0.2.9

mlflow==2.14.3

openpyxl==3.1.5

setuptools==71.1.0

DBR 15.2

Re: Vectorsearch ConnectionResetError Max retries exceeded

RobinK — Fri, 26 Jul 2024 04:44:13 GMT

downgrading langchain-community to version 0.2.4 solved my problem.

topic Vectorsearch ConnectionResetError Max retries exceeded in Machine Learning

Vectorsearch ConnectionResetError Max retries exceeded

Re: Vectorsearch ConnectionResetError Max retries exceeded