โ12-27-2024 05:15 AM - edited โ12-27-2024 05:20 AM
RestException: NOT_FOUND: The directory being accessed is not found.
when calling
โ12-27-2024 05:43 AM
The error message "RestException: NOT_FOUND: The directory being accessed is not found" typically indicates that the model or its version cannot be located in the specified directory. Here are a few steps you can take to troubleshoot and resolve this issue:
Here is an example of how to verify the model URI and check the model registration
import mlflow
# Set the registry URI to Databricks
mlflow.set_registry_uri("databricks-uc")
# Define the model URI
model_uri = f"models:/{model_name}/{model_version}"
# Check if the model exists
try:
model_info = mlflow.models.get_model_info(model_uri)
print(f"Model {model_name} version {model_version} exists.")
except mlflow.exceptions.MlflowException as e:
print(f"Error: {e}")
โ12-27-2024 07:54 AM
When running this, getting this which looks like it's going to the default directory, this is not where the model sits I think:
Error: RESOURCE_DOES_NOT_EXIST: Routine or Model 'viable_test.default.egg_chat' does not exist.
โ12-27-2024 12:17 PM
Check the code or configuration files where the model is being referenced or loaded. The path or directory of the model is often specified in these files.
โ12-30-2024 05:10 AM
Checked the config.yml file and it looks like this:
โ12-30-2024 09:01 AM - edited โ12-30-2024 09:08 AM
Thanks for sharing the details so far! To help construct the correct resource path and resolve the issue, could you provide a bit more information?
Model Details:
Cluster/Endpoint Setup:
llm_endpoint
(databricks-meta-llama-3-3-70b-instruct
) accessible and correctly set up for your workspace?Error Message Context:
agents.deploy
or a related step)?Config File Alignment:
config.yml
align with the actual model location in the registry? For example, is main.egg_shop.*
referencing the correct schema for viable_test.default.egg_chat
?Ultimately, what is the version of this test code with values if it can be shared, to help us diagnose this problem. If the uc_functions parameter (main.egg_shop.*) relates to Unity Catalog, ensure the model's schema (viable_test.default) matches the catalog/schema setup. Also, check you have the necessary permissions to access the model (GRANT USE on Catalog / GRANT SELECT ON Model).
โ12-31-2024 01:33 AM
Just wanted to add some clarity on this issue that might shine some light regarding the issue, i've tried setting up multiple models including the Databricks 'dbdemos', 'llm-tag-chatbot' examples and it's resulted in the same directory issues around agents.deploy()
Model Details:
Cluster/Endpoint Setup:
Error Message Context:
Config File Alignment:
โ12-31-2024 04:34 AM
- viable_test.default.egg_chat does not exist, this is the issue I believe
I see. Let's ignore the "rag_chatbot.basic_rag_demo" error for now, and focus on the egg_chat.
So, you stated in your details: "The model is registered in the Unity Catalog 'My organization' under the 'main' folder, I've tried switching the workspace default catalog to main but the problem still persists.". This indicates that the main catalog is where the model resides. Your config.yml includes: uc_functions: "main.egg_shop.*". This suggests that egg_shop is the schema intended for use with this model. The model name egg_chat is consistent throughout your responses, therefore assuming itโs the correct name, the fully qualified name (FQN) should logically be main.egg_shop.egg_chat (please correct me if I'm wrong), instead of viable_test. Viable_test may be getting prepended as default catalog perhaps? Not really sure, but it does sounds like we have to manually set it to be "main" instead.
Can you please run this code below once, from a notebook? It should check both UC and WMR, and ideally find and extract where it is registered if so along with metadata related to it, hope it helps:
from mlflow.tracking.client import MlflowClient
import mlflow
# Initialize MLflow client
client = MlflowClient()
# Specify the model's logical name or partial name to search for
search_model = "egg_chat"
# Function to search for models in a specific registry
def search_registry(uri, model_name):
mlflow.set_registry_uri(uri)
client = MlflowClient()
models = client.search_registered_models()
for model in models:
if model_name in model.name:
return model
return None
# Check Unity Catalog first
mlflow.set_registry_uri("databricks-uc")
uc_model = search_registry("databricks-uc", search_model)
# If not found in UC, check Workspace Model Registry
if not uc_model:
print("Model not found in Unity Catalog. Searching Workspace Model Registry...")
workspace_model = search_registry(None, search_model)
else:
workspace_model = None
# Function to print detailed model metadata
def print_model_details(model, registry_type):
print(f"Model found in {registry_type}: {model.name}")
parts = model.name.split(".")
if len(parts) == 3:
catalog, schema, model_name = parts
print(f"Catalog: {catalog}, Schema: {schema}, Model: {model_name}")
else:
print(f"Model Name: {model.name}")
print(f"Tags: {model.tags}")
if model.latest_versions:
for version in model.latest_versions:
print(f" - Version: {version.version}")
print(f" Source: {version.source}")
print(f" Stage: {version.current_stage}")
print(f" Run ID: {version.run_id}")
else:
print("No versions available for this model.")
# Print results for both registries
if uc_model:
print_model_details(uc_model, "Unity Catalog")
if workspace_model:
print_model_details(workspace_model, "Workspace Model Registry")
if not uc_model and not workspace_model:
print("Model not found in either Unity Catalog or Workspace Model Registry.")
Then update both the config.yml and agents.deploy as required.
Going back to the original code, what's the output you're getting from:
import mlflow
mlflow.set_registry_uri("databricks-uc")
model_uri = f"models:/main.egg_shop.egg_chat/1"
try:
model_info = mlflow.models.get_model_info(model_uri)
print(f"Model {model_name} version {model_version} exists.")
except mlflow.exceptions.MlflowException as e:
print(f"Error: {e}")โ
* Note: you may need to later on check on the model grants, or version, these other side issues should be straight forward to resolve.
โ12-31-2024 05:36 AM
When running the first section of code this is the reponse:
Model found in Unity Catalog: main.egg_shop.egg_chat Catalog: main, Schema: egg_shop, Model: egg_chat Tags: <bound method RegisteredModelSearch.tags of <RegisteredModelSearch: >> No versions available for this model.
When running second section
Error: NOT_FOUND: The directory being accessed is not found.
โ01-09-2025 08:00 AM
Hi @VZLA @Alberto_Umana , I am currently facing the same issue as @stevomcnevo007 . I get exactly the same outputs in the scripts that you have provided above and still unable to deploy the model. Can you please help?
โ01-09-2025 08:42 AM
Hi @ezermoysis / @stevomcnevo007,
Can you confirm that there are versions of the model registered yet?
This can happen if the model has been registered but no versions have been created or logged yet
โ01-09-2025 08:52 AM
Thanks @Alberto_Umana. I think so, yes.
I am using the following code to register the models and when I print the
and when I print
I get this, so I believe there are versions registered
<ModelVersion: aliases=[], creation_timestamp=1736436606155, current_stage=None, description='', last_updated_timestamp=1736436613763, name='emoysis_catalog.cookbook.sop_chatbot', run_id='', run_link=None, source='dbfs:/databricks/mlflow-tracking/3498969875739103/0ee0f2039ef54180b474e285380fabca/artifacts/agent', status='READY', status_message='', tags={}, user_id='e.moysis@instadeep.com', version='10'>
โ01-09-2025 08:56 AM
I note that I am using free version and trying to follow this guide: https://ai-cookbook.io/nbs/5-hands-on-build-poc.html
โ01-10-2025 05:38 AM
@Alberto_UmanaBelow I add all the log:
File <command-3498969875739223>, line 12
10 print(UC_MODEL_NAME)
11 print(uc_registered_model_info.version)
---> 12 deployment_info = agents.deploy("emoysis_catalog.cookbook.sop_chatbot", 13)
14 # Wait for the Review App to be ready
15 print("\nWaiting for endpoint to deploy. This can take 15 - 20 minutes.", end="")
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/databricks/agents/deployments.py:479, in deploy(model_name, model_version, scale_to_zero, environment_vars, instance_profile_arn, tags, workload_size, endpoint_name, **kwargs)
451 def deploy(
452 model_name: str,
453 model_version: int,
(...)
460 **kwargs,
461 ) -> Deployment:
462 """
463 Deploy new version of the agent.
464
(...)
477 Chain deployment metadata.
478 """
--> 479 _check_model_is_rag_compatible(model_name, model_version)
480 _check_model_name(model_name)
481 if endpoint_name is None:
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/databricks/agents/utils/mlflow_utils.py:96, in _check_model_is_rag_compatible(model_name, version)
95 def _check_model_is_rag_compatible(model_name: str, version: int):
---> 96 input_schema, output_schema = _load_model_schema(model_name, version)
97 return _check_model_is_rag_compatible_helper(input_schema, output_schema)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/databricks/agents/utils/mlflow_utils.py:90, in _load_model_schema(model_name, version)
88 mlflow.set_registry_uri(get_databricks_uc_registry_uri())
89 model_uri = f"models:/{model_name}/{str(version)}"
---> 90 model_info = mlflow.models.get_model_info(model_uri)
91 signature = model_info.signature
92 return signature.inputs, signature.outputs
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/mlflow/models/model.py:984, in get_model_info(model_uri)
981 from mlflow.pyfunc import _download_artifact_from_uri
983 meta_file_uri = model_uri.rstrip("/") + "/" + MLMODEL_FILE_NAME
--> 984 meta_local_path = _download_artifact_from_uri(artifact_uri=meta_file_uri)
985 model_meta = Model.load(meta_local_path)
986 return ModelInfo(
987 artifact_path=model_meta.artifact_path,
988 flavors=model_meta.flavors,
(...)
997 metadata=model_meta.metadata,
998 )
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/mlflow/tracking/artifact_utils.py:111, in _download_artifact_from_uri(artifact_uri, output_path, lineage_header_info)
108 repo = get_artifact_repository(artifact_uri=root_uri)
110 if isinstance(repo, ModelsArtifactRepository):
--> 111 return repo.download_artifacts(
112 artifact_path=artifact_path,
113 dst_path=output_path,
114 lineage_header_info=lineage_header_info,
115 )
116 return repo.download_artifacts(artifact_path=artifact_path, dst_path=output_path)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/mlflow/store/artifact/models_artifact_repo.py:201, in ModelsArtifactRepository.download_artifacts(self, artifact_path, dst_path, lineage_header_info)
199 # Pass lineage header info if model is registered in UC
200 if isinstance(self.repo, UnityCatalogModelsArtifactRepository):
--> 201 model_path = self.repo.download_artifacts(
202 artifact_path, dst_path, lineage_header_info=lineage_header_info
203 )
204 else:
205 model_path = self.repo.download_artifacts(artifact_path, dst_path)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/mlflow/store/artifact/unity_catalog_models_artifact_repo.py:157, in UnityCatalogModelsArtifactRepository.download_artifacts(self, artifact_path, dst_path, lineage_header_info)
156 def download_artifacts(self, artifact_path, dst_path=None, lineage_header_info=None):
--> 157 return self._get_artifact_repo(lineage_header_info=lineage_header_info).download_artifacts(
158 artifact_path, dst_path
159 )
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/mlflow/store/artifact/artifact_repo.py:268, in ArtifactRepository.download_artifacts(self, artifact_path, dst_path)
266 # Submit download tasks
267 futures = {}
--> 268 if self._is_directory(artifact_path):
269 for file_info in self._iter_artifacts_recursive(artifact_path):
270 if file_info.is_dir: # Empty directory
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/mlflow/store/artifact/artifact_repo.py:167, in ArtifactRepository._is_directory(self, artifact_path)
166 def _is_directory(self, artifact_path):
--> 167 listing = self.list_artifacts(artifact_path)
168 return len(listing) > 0
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/mlflow/store/artifact/presigned_url_artifact_repo.py:104, in PresignedUrlArtifactRepository.list_artifacts(self, path)
101 req_body = json.dumps({"page_token": page_token}) if page_token else None
103 response_proto = ListDirectoryResponse()
--> 104 resp = call_endpoint(
105 host_creds=self.db_creds,
106 endpoint=endpoint,
107 method="GET",
108 json_body=req_body,
109 response_proto=response_proto,
110 )
111 for dir_entry in resp.contents:
112 rel_path = posixpath.relpath(dir_entry.path, self.artifact_uri)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/mlflow/utils/rest_utils.py:370, in call_endpoint(host_creds, endpoint, method, json_body, response_proto, extra_headers)
367 call_kwargs["json"] = json_body
368 response = http_request(**call_kwargs)
--> 370 response = verify_rest_response(response, endpoint)
371 response_to_parse = response.text
372 js_dict = json.loads(response_to_parse)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0ea80acf-754f-4b74-9414-f078ba584e3e/lib/python3.10/site-packages/mlflow/utils/rest_utils.py:240, in verify_rest_response(response, endpoint)
238 if response.status_code != 200:
239 if _can_parse_as_json_object(response.text):
--> 240 raise RestException(json.loads(response.text))
241 else:
242 base_msg = (
243 f"API request to endpoint {endpoint} "
244 f"failed with error code {response.status_code} != 200"
245 )
โ01-13-2025 08:47 AM
Any fix here?
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group