Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- Databricks Community
- Machine Learning
- Error in creating a serving endpoint: registered m...
Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Error in creating a serving endpoint: registered model not found
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
yesterday
I have registered a custom model which loads another model in the load_context method. Everything works fine when I load (with mlflow.pyfunc.load_model) and use the model in a notebook. When I try to create a serving endpoint for it I keep becoming the following error:
mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
Here is a code snippet of the custom model:
class BldgModel(mlflow.pyfunc.PythonModel😞
def __init__(self, model😞
self.model = model
def load_context(self, context😞
self.bldg_normalizer = mlflow.sklearn.load_model("models:/dbw_dev_we_data_hub.ml2.bldg_normalizer/6")
...
As steted above mlflow.sklearn.load_model("models:/dbw_dev_we_data_hub.ml2.bldg_normalizer/6") in a notebook works fine.
Service logs din't help mutch either:
[kn2wj] [2024-11-28 08:50:40 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 253, in _terminate
[kn2wj] [2024-11-28 08:50:40 +0000] sys.exit(1)
[kn2wj] [2024-11-28 08:50:40 +0000] SystemExit
[kn2wj] [2024-11-28 08:50:40 +0000] :
[kn2wj] [2024-11-28 08:50:40 +0000] 1
[kn2wj] [2024-11-28 08:50:40 +0000] [440] [INFO] Booting worker with pid: 440
[kn2wj] [2024-11-28 08:50:40 +0000] [442] [INFO] Booting worker with pid: 442
[kn2wj] [2024-11-28 08:50:43 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:43 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:43 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:43 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:43 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:43 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:43 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:43 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:43 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:43 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:43 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:43 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:43 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:43 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:43 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:43 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:43 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:43 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:43 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:43 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:43 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:43 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:43 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:43 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:43 +0000] [425] [INFO] Worker exiting (pid: 425)
[kn2wj] [2024-11-28 08:50:44 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:44 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:44 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:44 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:44 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:44 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:44 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:44 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:44 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:44 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:44 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:44 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:44 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:44 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:44 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:44 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:44 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:44 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:44 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:44 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:44 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:44 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:44 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:44 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:44 +0000] [427] [INFO] Worker exiting (pid: 427)
[kn2wj] [2024-11-28 08:50:46 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:46 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:46 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:46 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:46 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:46 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:46 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:46 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:46 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:46 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:46 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:46 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:46 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:46 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:46 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:46 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:46 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:46 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:46 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:46 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:46 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:46 +0000] [440] [INFO] Worker exiting (pid: 440)
[kn2wj] [2024-11-28 08:50:46 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:46 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:46 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:46 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:46 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:46 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:46 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:46 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:46 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:46 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:46 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:46 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:46 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:46 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:46 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:46 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:46 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:46 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:46 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:46 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:46 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:46 +0000] [442] [INFO] Worker exiting (pid: 442)
[kn2wj] [2024-11-28 08:50:48 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:48 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:48 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:48 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:48 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:48 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:48 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:48 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:48 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:48 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:48 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:48 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:48 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:48 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:48 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:48 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:48 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:48 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:48 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:48 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:48 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:48 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:48 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:48 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:48 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:48 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:48 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:48 +0000] Exception ignored in:
[kn2wj] [2024-11-28 08:50:48 +0000] <module 'threading' from '/opt/conda/envs/mlflow-env/lib/python3.11/threading.py'>
[kn2wj] [2024-11-28 08:50:48 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1553, in _shutdown
[kn2wj] [2024-11-28 08:50:48 +0000] atexit_call()
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 31, in _python_exit
[kn2wj] [2024-11-28 08:50:48 +0000] t.join()
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1112, in join
[kn2wj] [2024-11-28 08:50:48 +0000] self._wait_for_tstate_lock()
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1132, in _wait_for_tstate_lock
[kn2wj] [2024-11-28 08:50:48 +0000] if lock.acquire(block, timeout):
...
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 253, in _terminate
[kn2wj] [2024-11-28 08:50:48 +0000] sys.exit(1)
[kn2wj] [2024-11-28 08:50:48 +0000] SystemExit
[kn2wj] [2024-11-28 08:50:48 +0000] :
[kn2wj] [2024-11-28 08:50:48 +0000] 1
[kn2wj] [2024-11-28 08:50:49 +0000] [462] [INFO] Booting worker with pid: 462
[kn2wj] [2024-11-28 08:50:49 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:49 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:49 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:49 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:49 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:49 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:49 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:49 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:49 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:49 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:49 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:49 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:49 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:49 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:49 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:49 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:49 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:49 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:49 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:49 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:49 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:49 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:49 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:49 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:49 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:49 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:49 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:49 +0000] Exception ignored in:
[kn2wj] [2024-11-28 08:50:49 +0000] <module 'threading' from '/opt/conda/envs/mlflow-env/lib/python3.11/threading.py'>
[kn2wj] [2024-11-28 08:50:49 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1553, in _shutdown
[kn2wj] [2024-11-28 08:50:49 +0000] atexit_call()
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 31, in _python_exit
[kn2wj] [2024-11-28 08:50:49 +0000] t.join()
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1112, in join
[kn2wj] [2024-11-28 08:50:49 +0000] self._wait_for_tstate_lock()
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1132, in _wait_for_tstate_lock
[kn2wj] [2024-11-28 08:50:49 +0000] if lock.acquire(block, timeout):
...
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 253, in _terminate
[kn2wj] [2024-11-28 08:50:49 +0000] sys.exit(1)
[kn2wj] [2024-11-28 08:50:49 +0000] SystemExit
[kn2wj] [2024-11-28 08:50:49 +0000] :
[kn2wj] [2024-11-28 08:50:49 +0000] 1
[kn2wj] [2024-11-28 08:50:49 +0000] [471] [INFO] Booting worker with pid: 471
[kn2wj] line 632, in load_model
Labels:
- Labels:
-
Model Serving
0 REPLIES 0
Connect with Databricks Users in Your Area
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group
Announcements
Related Content
- Serving Endpoint: Container Image Creation Fails in Machine Learning
- Serving Endpoint Deployment in Machine Learning
- Error to create an endpoint of databricks with 2 primary keys online table in Machine Learning
- ModuleNotFoundError: No module named 'model_train' when using mlflow.sklearn.load_model in Machine Learning