- 4327 Views
- 1 replies
- 1 kudos
MlflowException: Unable to download model artifacts in Databricks while registering model with MLflo
I am attempting to log, register, and deploy a finetuned GPT2 model in Databricks. While I have been able to get my logging code to run, when I try to run my registration code, I get an MlflowException error.Here is my model logging code.mlflow.set_r...
- 4327 Views
- 1 replies
- 1 kudos
- 1 kudos
I've experience the same error. The issue is that the model uri is not correct.The model is logged with:mlflow.transformers.log_model( ... , artifact_path="gpt2", ...)The artifact_path is the last part of the model uri. If you don't specify it, it's ...
- 1 kudos
- 431 Views
- 0 replies
- 0 kudos
Learn Databricks AI medium article series for fellow learners.
When it comes to machine learning, the platform plays a pivotal role in successful implementation. Databricks offers a best-in-class machine learning platform with cutting-edge features such as MLflow, Model Registry, Feature Store, and MLOps, which ...
- 431 Views
- 0 replies
- 0 kudos
- 1325 Views
- 13 replies
- 2 kudos
MLflow: Connect python with Community Edition
Hello,I am new to databricks and want to work with MLFlow in the Databricks Community Edition. In python i am using mlflow.login(). This requests me to enter a password. But i do not have any password due to the fact that databricks login only requir...
- 1325 Views
- 13 replies
- 2 kudos
- 2 kudos
I am currently looking with our internal teams if this will be provided in the near future, still waiting for confirmation.
- 2 kudos
- 575 Views
- 2 replies
- 2 kudos
Resolved! XGBoost Feature Weighting
We are trying to train a predictive ML model using the XGBoost Classifier. Part of the requirements we have gotten from our business team is to implement feature weighting as they have defined certain features mattering more than others. We have 69 f...
- 575 Views
- 2 replies
- 2 kudos
- 2 kudos
Hello @sjohnston2 here is some information i found internally: Possible Causes Memory Access Issue: The segmentation fault suggests that the program is trying to access memory that it's not allowed to, which could be caused by an internal bug in XGBo...
- 2 kudos
- 525 Views
- 2 replies
- 1 kudos
Resolved! Online Feature Table : Storage
Databricks Online Feature Table is in public preview , And we have some questions on this 1) What storage is being used for Online Feature table's Data. Our offline feature table is stored in Unity Catalog managed S3 bucket (Customer AWS ). Does onli...
- 525 Views
- 2 replies
- 1 kudos
- 388 Views
- 1 replies
- 1 kudos
Resolved! No Spark Session Available Within Model Serving Environment
Hi,Is it possible to have a Spark session, that can be to used query the Unity Catalog etc, available within a Model Serving?I have an MLFlow Pyfunc model that needs to get data from a Feature Table as part of its `.predict()` method. See my earlier ...
- 388 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @mharrison Creating a Spark session within a Model Serving environment is not directly supported, which is why you are encountering the Exception: No SparkSession Available! error. This limitation arises because the serving environment does not a...
- 1 kudos
- 21740 Views
- 3 replies
- 0 kudos
dbfs not found
Hi, I've saved a custom pyfunc and now I'm trying to load it in a pandas_udf. It works on small samples or if I repartition everything to 1 partition, but when I try to run it on a larger sample and distribute it across my cluster it fails repeatably...
- 21740 Views
- 3 replies
- 0 kudos
- 0 kudos
This problem can often be attributed to the model artifacts not being available on all the executors, especially in a distributed environment. Can you try using the dbutils.fs.refreshMounts() in your code? If the model is small enough, broadcast it t...
- 0 kudos
- 455 Views
- 2 replies
- 0 kudos
Feature Lookup Help
Hi,ContextI'm looking for help trying to get Unity Catalog Feature Lookup to work with my model how I need it to.I have a trained darts time series model that takes as input to its `.predict()` method both the history of the variable in question, and...
- 455 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks for your response. It sounds like the 2nd approach is best for me, modifying the `predict()` method to perform the required history lookup.Is it possible to do this via the Feature Engineering client within that method, or should I simply quer...
- 0 kudos
- 545 Views
- 4 replies
- 1 kudos
How to serve a Unity Catalog ML model to external usage
Hello everyone I am following this notebook tutorial https://docs.databricks.com/en/machine-learning/manage-model-lifecycle/index.html#example-notebook Now I can register a machine learning model in Unity Catalog, but the tutorial only shows how to u...
- 545 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @johndoe99012 If the answer resolved your question, please consider marking it as the solution. It helps others in the community find answers more easily.
- 1 kudos
- 1168 Views
- 5 replies
- 1 kudos
Hosting R apps and models
Hello!Has anyone tried to host an R Shiny app using Databricks apps? It is very useful for Python apps, but it would be nice to know if anyone have any workaround to make it work with R Shiny Apps.We also wonder about Databricks model hosting for R m...
- 1168 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @AntonDBUser, Did you get a chance to test this? Let me know if the above solution can be marked as accepted. Thanks
- 1 kudos
- 3194 Views
- 3 replies
- 0 kudos
Maximum wait time Databricks Model Serving
hi, hope you are fineI deployed a model 3 or 2 months ago using Databricks Serving and MLFlow. The model worked good using GPU from model serving.I stopped using it for some months and when I tried again deploying it, it has some errors.1. [FIXED] A ...
- 3194 Views
- 3 replies
- 0 kudos
- 645 Views
- 2 replies
- 0 kudos
Resolved! Error in creating a serving endpoint: registered model not found
I have registered a custom model which loads another model in the load_context method. Everything works fine when I load (with mlflow.pyfunc.load_model) and use the model in a notebook. When I try to create a serving endpoint for it I keep becoming t...
- 645 Views
- 2 replies
- 0 kudos
- 0 kudos
It is registered in the Unity Catalog. I have found a complete other solution now. With the help of TransformedTargetRegressor I don't need a separate normalisation step anymore and therefore don't load a model in load_context anymore.
- 0 kudos
- 4721 Views
- 3 replies
- 1 kudos
Resolved! Tuning `CrossValidator` spark job performance
I am running a 3-fold cross validation of an ML pipeline that utilizes `GBTClassifier` as the final step. It takes 18 hours to run and I am looking for feedback into how to improve the performance as I expect this to go faster.For context here is the...
- 4721 Views
- 3 replies
- 1 kudos
- 1 kudos
Hello @Assaad Mrad​ , So this looks like trying to decide between putting the pipeline in the cross validator or the cross validator in the pipeline. Since you are doing the polynomial expansion as part of the pipeline you might want to consider putt...
- 1 kudos
- 1868 Views
- 4 replies
- 0 kudos
Training Job Failure (Driver Error)
We have a new model training job that was running fine for a few days and then started failing. I have attached images for more details.I am wondering if 'can't reach driver cluster' is a red herring. It says the driver is healthy right before execut...
- 1868 Views
- 4 replies
- 0 kudos
- 0 kudos
In our case, we needed to correct our dependent libraries. We had an incorrect path referenced.
- 0 kudos
- 326 Views
- 0 replies
- 0 kudos
Rolling predictions with FeatureEngineeringClient
I am performing a time series analysis, using a XGBoostRegressor with rolling predictions. I am doing so using the FeatureEngineeringClient (in combination with Unity Catalog), where I create and load in my features during training and inference, as ...
- 326 Views
- 0 replies
- 0 kudos
