topic Error accessing file from dbfs inside mlflow serve endpoint in Get Started Discussions

Error accessing file from dbfs inside mlflow serve endpoint

sanjay — Sun, 07 Jan 2024 13:29:56 GMT

Hi,

I have mlflow model served using serverless GPU which takes audio file name as input and then file will be passed as parameter to huggiung face model inside predict method. But I am getting following error

HFValidationError(\nhuggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/dbfs/tmp'. Use `repo_type` argument if needed.\n"}

Appreciate any help.

Regards,

Sanjay

Re: Error accessing file from dbfs inside mlflow serve endpoint

sanjay — Tue, 09 Jan 2024 06:56:00 GMT

Thank you @Retired_mod for prompt response. I am able to load the model from Huggingface correctly. The issue is with loading input audio file which is stored in local directory in databrick dbfs. This error is coming only after creating serverless endpoint. Without serverless endpoint, I am able to load model from registry and read incoming audio file from same location and process it.

Re: Error accessing file from dbfs inside mlflow serve endpoint

txti — Fri, 17 Jan 2025 17:37:09 GMT

I have the same issue.
I have a large file that I cannot access from an MLFlow service.
Things I have tried (none of these work):

Read-only from DBFS
- `dbfs:/myfolder/myfile.chroma` does not work
- `/dbfs/myfolder/myfile.chroma` does not work
Read-only from Unity Catalog Volume
- `/Volumes/mycatalog/mydb/myfolder/myfile.chroma` does not work
Read-only from S3 storage
- `s3://mybucket/mydb/myfolder/myfile.chroma` does not work

So far, the only thing that works is parking the huge file as MLFlow artifact and accessing it locally in the service (awful).

The errors are all similar to:

`ValueError: Dataset at path Volumes/non_prod/metrics/files/recommender/embed-db.chroma was not found`

Notice that the path strips the leading `/` from `/Volumes/` and does the same with `s3://` protocol, etc.

I can't use MLFlow endpoints without this very basic functionality.