Can't load model from UC due to DBFS issue
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-19-2024 11:14 AM
I want to load a model I have registered in Unity Catalog using a Shared cluster, but it seems to be trying to use dbfs under the hood and it gives me an error.
I am using DBR 13.3 LTS and mlflow-skinny[databricks]==2.14.3
My code
import mlflow
mlflow.set_registry_uri("databricks-uc")
mlflow.spark.load_model("models:/my_uc_catalog.my_schema.my_model@my_alias")Throws:
OSError: [Errno 95] Operation not supported: '/dbfs/tmp'If I run it on a Single User cluster it works fine, but I want it to work on Shared clusters