cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Vectorsearch ConnectionResetError Max retries exceeded

RobinK
Contributor

Hi,

we are serving a unity catalog langchain model with databricks model serving. When I run the predict() function on the model in a notebook, I get the expected output. But when I query the served model, errors occur in the service logs:

Error message:

2024-07-24T12:05:38.+0000 WARNING urllib3.connectionpool Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectionResetError(104, 'Connection reset by peer')': /api/2.0/<workspace-id>/vector-search/endpoints/my_vs_endpoint/indexes/my_vs_index/query

2024-07-24T12:05:44.+0000 WARNING mlflowserving.scoring_server LangChain invocation failed with MlflowException: 1 tasks failed. Errors: {0: 'error: ConnectionError(MaxRetryError("HTTPSConnectionPool(host=\'6cb7f5a6-3135-4280-987f-9758c9fa3753.vector-search.cloud.databricks.com\', port=443): Max retries exceeded with url:/api/2.0/<workspace-id>/vector-search/endpoints/my_vs_endpoint/indexes/my_vs_index/query (Caused by ProtocolError(\'Connection aborted.\', ConnectionResetError(104, \'Connection reset by peer\')))")) urllib3.exceptions.ProtocolError: (\'Connection aborted.\', ConnectionResetError(104, \'Connection reset by peer\'))\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/requests/adapters.py", line 667, in send\n resp = conn.urlopen(\n File "/opt/conda/envs/mlflow-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 873, in urlopen\n

Request:

{
"dataframe_split":
{
"columns":["payload"],
"data": ["my_data"]
}
}

Response:

{
"predictions": null
}

Working code in my notebook:

import mlflow
import pandas as pd
my_model = mlflow.pyfunc.load_model(model_info.model_uri)

my_model.predict(
pd.DataFrame(columns=["payload"], data=["my_data"]))
 
The served model consist of a query to a databricks vectorsearch index. I should be later extended by a llm.
 
req.txt:
ipykernel==6.29.4
numpy==1.26.4
databricks-connect==14.3.2
databricks-sdk==0.28.0
databricks-vectorsearch==0.40
langchain==0.2.10
langchain-community==0.2.9
mlflow==2.14.3
openpyxl==3.1.5
setuptools==71.1.0
 
DBR 15.2
1 ACCEPTED SOLUTION

Accepted Solutions

RobinK
Contributor

downgrading langchain-community to version 0.2.4 solved my problem.

View solution in original post

1 REPLY 1

RobinK
Contributor

downgrading langchain-community to version 0.2.4 solved my problem.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group