cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Error accessing file from dbfs inside mlflow serve endpoint

sanjay
Valued Contributor II

Hi,

I have mlflow model served using serverless GPU which takes audio file name as input and then file will be passed as parameter to huggiung face model inside predict method. But I am getting following error

HFValidationError(\nhuggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/dbfs/tmp'. Use `repo_type` argument if needed.\n"}

Appreciate any help.

Regards,

Sanjay

 

2 REPLIES 2

sanjay
Valued Contributor II

Thank you @Retired_mod for prompt response. I am able to load the model from Huggingface correctly. The issue is with loading input audio file which is stored in local directory in databrick dbfs. This error is coming only after creating serverless endpoint. Without serverless endpoint, I am able to load model from registry and read incoming audio file from same location and process it.

txti
New Contributor III

I have the same issue.
I have a large file that I cannot access from an MLFlow service.
Things I have tried (none of these work):

  • Read-only from DBFS
    • `dbfs:/myfolder/myfile.chroma` does not work
    • `/dbfs/myfolder/myfile.chroma` does not work
  • Read-only from Unity Catalog Volume
    • `/Volumes/mycatalog/mydb/myfolder/myfile.chroma` does not work
  • Read-only from S3 storage
    • `s3://mybucket/mydb/myfolder/myfile.chroma` does not work

So far, the only thing that works is parking the huge file as MLFlow artifact and accessing it locally in the service (awful).

The errors are all similar to:

`ValueError: Dataset at path Volumes/non_prod/metrics/files/recommender/embed-db.chroma was not found`

Notice that the path strips the leading `/` from `/Volumes/` and does the same with `s3://` protocol, etc.

I can't use MLFlow endpoints without this very basic functionality.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group