Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- Databricks Community
- Machine Learning
- Error in creating a serving endpoint: registered m...
Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-28-2024 01:09 AM
I have registered a custom model which loads another model in the load_context method. Everything works fine when I load (with mlflow.pyfunc.load_model) and use the model in a notebook. When I try to create a serving endpoint for it I keep becoming the following error:
mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
Here is a code snippet of the custom model:
class BldgModel(mlflow.pyfunc.PythonModel😞
def __init__(self, model😞
self.model = model
def load_context(self, context😞
self.bldg_normalizer = mlflow.sklearn.load_model("models:/dbw_dev_we_data_hub.ml2.bldg_normalizer/6")
...
As steted above mlflow.sklearn.load_model("models:/dbw_dev_we_data_hub.ml2.bldg_normalizer/6") in a notebook works fine.
Service logs din't help mutch either:
[kn2wj] [2024-11-28 08:50:40 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 253, in _terminate
[kn2wj] [2024-11-28 08:50:40 +0000] sys.exit(1)
[kn2wj] [2024-11-28 08:50:40 +0000] SystemExit
[kn2wj] [2024-11-28 08:50:40 +0000] :
[kn2wj] [2024-11-28 08:50:40 +0000] 1
[kn2wj] [2024-11-28 08:50:40 +0000] [440] [INFO] Booting worker with pid: 440
[kn2wj] [2024-11-28 08:50:40 +0000] [442] [INFO] Booting worker with pid: 442
[kn2wj] [2024-11-28 08:50:43 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:43 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:43 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:43 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:43 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:43 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:43 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:43 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:43 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:43 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:43 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:43 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:43 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:43 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:43 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:43 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:43 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:43 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:43 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:43 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:43 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:43 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:43 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:43 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:43 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:43 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:43 +0000] [425] [INFO] Worker exiting (pid: 425)
[kn2wj] [2024-11-28 08:50:44 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:44 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:44 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:44 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:44 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:44 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:44 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:44 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:44 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:44 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:44 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:44 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:44 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:44 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:44 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:44 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:44 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:44 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:44 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:44 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:44 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:44 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:44 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:44 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:44 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:44 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:44 +0000] [427] [INFO] Worker exiting (pid: 427)
[kn2wj] [2024-11-28 08:50:46 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:46 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:46 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:46 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:46 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:46 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:46 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:46 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:46 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:46 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:46 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:46 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:46 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:46 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:46 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:46 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:46 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:46 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:46 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:46 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:46 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:46 +0000] [440] [INFO] Worker exiting (pid: 440)
[kn2wj] [2024-11-28 08:50:46 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:46 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:46 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:46 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:46 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:46 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:46 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:46 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:46 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:46 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:46 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:46 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:46 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:46 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:46 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:46 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:46 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:46 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:46 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:46 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:46 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:46 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:46 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:46 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:46 +0000] [442] [INFO] Worker exiting (pid: 442)
[kn2wj] [2024-11-28 08:50:48 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:48 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:48 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:48 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:48 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:48 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:48 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:48 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:48 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:48 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:48 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:48 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:48 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:48 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:48 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:48 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:48 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:48 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:48 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:48 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:48 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:48 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:48 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:48 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:48 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:48 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:48 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:48 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:48 +0000] Exception ignored in:
[kn2wj] [2024-11-28 08:50:48 +0000] <module 'threading' from '/opt/conda/envs/mlflow-env/lib/python3.11/threading.py'>
[kn2wj] [2024-11-28 08:50:48 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1553, in _shutdown
[kn2wj] [2024-11-28 08:50:48 +0000] atexit_call()
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 31, in _python_exit
[kn2wj] [2024-11-28 08:50:48 +0000] t.join()
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1112, in join
[kn2wj] [2024-11-28 08:50:48 +0000] self._wait_for_tstate_lock()
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1132, in _wait_for_tstate_lock
[kn2wj] [2024-11-28 08:50:48 +0000] if lock.acquire(block, timeout):
...
[kn2wj] [2024-11-28 08:50:48 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 253, in _terminate
[kn2wj] [2024-11-28 08:50:48 +0000] sys.exit(1)
[kn2wj] [2024-11-28 08:50:48 +0000] SystemExit
[kn2wj] [2024-11-28 08:50:48 +0000] :
[kn2wj] [2024-11-28 08:50:48 +0000] 1
[kn2wj] [2024-11-28 08:50:49 +0000] [462] [INFO] Booting worker with pid: 462
[kn2wj] [2024-11-28 08:50:49 +0000] An error occurred while loading the model: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:49 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:49 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:49 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:49 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[kn2wj] [2024-11-28 08:50:49 +0000] self.model = self.model_future.result()
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[kn2wj] [2024-11-28 08:50:49 +0000] return self.__get_result()
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[kn2wj] [2024-11-28 08:50:49 +0000] raise self._exception
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[kn2wj] [2024-11-28 08:50:49 +0000] result = self.fn(*self.args, **self.kwargs)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[kn2wj] [2024-11-28 08:50:49 +0000] model = load_model_fn(path)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
[kn2wj] [2024-11-28 08:50:49 +0000] is_func_called, result = True, f(*args, **kwargs)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1027, in load_model
[kn2wj] [2024-11-28 08:50:49 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 550, in _load_pyfunc
[kn2wj] [2024-11-28 08:50:49 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 544, in _load_context_model_and_signature
[kn2wj] [2024-11-28 08:50:49 +0000] python_model.load_context(context=context)
[kn2wj] [2024-11-28 08:50:49 +0000] File "/root/.ipykernel/14823/command-63560780221756-4259897093", line 9, in load_context
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/sklearn/__init__.py", line 632, in load_model
[kn2wj] [2024-11-28 08:50:49 +0000] local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/artifact_utils.py", line 108, in _download_artifact_from_uri
[kn2wj] [2024-11-28 08:50:49 +0000] repo = get_artifact_repository(artifact_uri=root_uri)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 133, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:49 +0000] return _artifact_repository_registry.get_artifact_repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 78, in get_artifact_repository
[kn2wj] [2024-11-28 08:50:49 +0000] return repository(artifact_uri)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 59, in __init__
[kn2wj] [2024-11-28 08:50:49 +0000] ) = ModelsArtifactRepository._get_model_uri_infos(artifact_uri)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 94, in _get_model_uri_infos
[kn2wj] [2024-11-28 08:50:49 +0000] download_uri = client.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/client.py", line 4144, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:49 +0000] return self._get_registry_client().get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 316, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:49 +0000] return self.store.get_model_version_download_uri(name, version)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 823, in get_model_version_download_uri
[kn2wj] [2024-11-28 08:50:49 +0000] model_version = self._fetch_file_model_version_if_exists(name, version)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 783, in _fetch_file_model_version_if_exists
[kn2wj] [2024-11-28 08:50:49 +0000] registered_model_version_dir = self._get_model_version_dir(name, version)
[kn2wj] [2024-11-28 08:50:49 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/store/model_registry/file_store.py", line 561, in _get_model_version_dir
[kn2wj] [2024-11-28 08:50:49 +0000] raise MlflowException(
[kn2wj] [2024-11-28 08:50:49 +0000] mlflow.exceptions.MlflowException: Registered Model with name=dbw_dev_we_data_hub.ml2.bldg_normalizer not found
[kn2wj] [2024-11-28 08:50:49 +0000] Exception ignored in:
[kn2wj] [2024-11-28 08:50:49 +0000] <module 'threading' from '/opt/conda/envs/mlflow-env/lib/python3.11/threading.py'>
[kn2wj] [2024-11-28 08:50:49 +0000] Traceback (most recent call last):
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1553, in _shutdown
[kn2wj] [2024-11-28 08:50:49 +0000] atexit_call()
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 31, in _python_exit
[kn2wj] [2024-11-28 08:50:49 +0000] t.join()
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1112, in join
[kn2wj] [2024-11-28 08:50:49 +0000] self._wait_for_tstate_lock()
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/threading.py", line 1132, in _wait_for_tstate_lock
[kn2wj] [2024-11-28 08:50:49 +0000] if lock.acquire(block, timeout):
...
[kn2wj] [2024-11-28 08:50:49 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 253, in _terminate
[kn2wj] [2024-11-28 08:50:49 +0000] sys.exit(1)
[kn2wj] [2024-11-28 08:50:49 +0000] SystemExit
[kn2wj] [2024-11-28 08:50:49 +0000] :
[kn2wj] [2024-11-28 08:50:49 +0000] 1
[kn2wj] [2024-11-28 08:50:49 +0000] [471] [INFO] Booting worker with pid: 471
[kn2wj] line 632, in load_model
Labels:
- Labels:
-
Model Serving
1 ACCEPTED SOLUTION
Accepted Solutions
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
a month ago
It is registered in the Unity Catalog. I have found a complete other solution now. With the help of TransformedTargetRegressor I don't need a separate normalisation step anymore and therefore don't load a model in load_context anymore.
2 REPLIES 2
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
a month ago
Hey @robertol
Is your "dbw_dev_we_data_hub.ml2.bldg_normalizer" model registered in the Unity Catalog or in the Workspace Model Registry? If it's registered in the Unity Catalog, please update the model_uri in your load_context method to match the format <catalog>.<schema>.<model-name>. After registering a new version of your PyFunc model and attempting to serve it, please let me know if this resolves the issue for you.
Best regards,
Jéssica Santos
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
a month ago
It is registered in the Unity Catalog. I have found a complete other solution now. With the help of TransformedTargetRegressor I don't need a separate normalisation step anymore and therefore don't load a model in load_context anymore.
Connect with Databricks Users in Your Area
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group
Announcements
Related Content