How to Resolve ConnectTimeoutError When Registering Models with MLflow
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-06-2024 10:34 AM
Hello everyone,
I'm trying to register a model with MLflow in Databricks, but encountering an error with the following command:
model_version = mlflow.register_model(f"runs:/{run_id}/random_forest_model", model_name)
The error message is as follows:
ConnectTimeoutError: Connect timeout on endpoint URL: "https://s3.amazonaws.com/***(unitycatalog-s3bucket-name)?location"
File /databricks/python/lib/python3.10/site-packages/urllib3/connection.py:174, in HTTPConnection._new_conn(self)
173 try:
--> 174 conn = connection.create_connection(
175 (self._dns_host, self.port), self.timeout, **extra_kw
176 )
178 except SocketTimeout:
File /databricks/python/lib/python3.10/site-packages/botocore/httpsession.py:490, in URLLib3Session.send(self, request)
486 raise ProxyConnectionError(
487 proxy_url=mask_proxy_url(proxy_url), error=e
488 )
489 except URLLib3ConnectTimeoutError as e:
--> 490 raise ConnectTimeoutError(endpoint_url=request.url, error=e)
491 except URLLib3ReadTimeoutError as e:
492 raise ReadTimeoutError(endpoint_url=request.url, error=e)
This command is referenced from this site's notebook: Databricks Scikit-learn Notebook.
It's worth mentioning that there are no issues with the DeltaTable registration process.
What could be causing this error? And how can it be resolved?
Thank you in advance for your help.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-15-2024 12:35 AM
@otara_geni if you are still struggling, try this - set the environmental variable in your code just before logging the model with the URL of the regional S3 endpoint (from the error it looks like MLFlow is attempting to use global one, which may not work in some cases, e.g. if you are using a backend private link). Just the domain name, not the full path.
Make sure to use your Databricks/bucket regional endpoint, e.g. `https://s3.eu-west-1.amazonaws.com` is for `eu-west-1`
import os
os.environ['MLFLOW_S3_ENDPOINT_URL'] = 'https://s3.<region>.amazonaws.com'HTH, Niko