- 2072 Views
- 3 replies
- 5 kudos
Deploy a ML model, trained and registered in Databricks to AKS
Hi,I can train, registered a ML Model in my Datbricks Workspace.Then, to deploy it on AKS, I need to register the model in Azure ML, and then, deploy to AKS.Is it possible to skip the Azure ML step?I would like to deploy directly into my AKS instance...
- 2072 Views
- 3 replies
- 5 kudos
- 5 kudos
Is it still the case, can't we serve the model in Databricks. I am new to this, so I am just wondering the capabilities.
- 5 kudos
- 3218 Views
- 3 replies
- 2 kudos
How to proper use Databricks MLFlow Managed tracker/register with Databricks Workflow
Hey.I'm building a DevOps/MLOps pipeline to train/register simple scikit learn model.I created a simple Databricks Workflow to execute training and register task on specific .git branch. (Workflow is setup with Databricks Repo on specifc branch, with...
- 3218 Views
- 3 replies
- 2 kudos
- 2 kudos
I had same issue while trying to call notebook from workflow. I was able to do what you did. But it needs new experiment name for each run, so I had to do this:# Set the experimentexperiment_name = f"/Workspace/MLOps/{env}/experiment/{experiment}_{ti...
- 2 kudos
- 2099 Views
- 1 replies
- 2 kudos
Run mlflow project from a Job.
Hey Guys, I'm trying to make automated process to run ML training sessions using mlflow and databricks jobs.While developing the model on my local machine using IDE, When finished I have a template notebook that get as parameters the mlflow project p...
- 2099 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi,Were you able to figure out this one? I have same issue trying to call training notebook from workflow. Each run needs a new experiment name which I can do but then it creates a new experiment ID/name for each workflow run. Where as when you run f...
- 2 kudos
- 3063 Views
- 2 replies
- 1 kudos
What is the best practice for applying MLFlow to clustering algorithms?
What is the best practice for applying MLFlow to clustering algorithms? What are the kinds of metrics customers track?
- 3063 Views
- 2 replies
- 1 kudos
- 1 kudos
Good question! I'll divide my suggestions into 2 parts:(1) In terms of MLflow Tracking, clustering is pretty similar to other ML workflows, so not much changes.(2) In terms of specific parameters, metrics, etc. to track, clustering is very different...
- 1 kudos
- 4931 Views
- 6 replies
- 7 kudos
How to save model produce by distributed training?
I am trying to save model after distributed training via the following codeimport sys from spark_tensorflow_distributor import MirroredStrategyRunner import mlflow.keras mlflow.keras.autolog() mlflow.log_param("learning_rate", 0.001) import...
- 4931 Views
- 6 replies
- 7 kudos
- 7 kudos
I think I finally worked this out.Here is the extra code to save out the model only once and from the 1st node:context = pyspark.BarrierTaskContext.get() if context.partitionId() == 0: mlflow.keras.log_model(model, "mymodel")
- 7 kudos
- 2174 Views
- 2 replies
- 5 kudos
How can I view the storage space taken by a registered model using MLFlow?
The information viewed about the registered models on the Models tab is very minimal. Just showing the tags we pass in and version information. How can I get more details about the model such as the size on disk?
- 2174 Views
- 2 replies
- 5 kudos
- 5 kudos
Hi,I have used the MLFlow client, but I am not sure where to find the size of the model image.The response to client.search_registered_models() I am getting is the following:<RegisteredModel: aliases={}, creation_timestamp=17061..., description='', l...
- 5 kudos
- 7362 Views
- 4 replies
- 1 kudos
Resolved! Shap Values for predictions from registered model
I have saved a model in the model registry using MLFlow. How can I find the shap values for this model once I have generated predictions in batch mode? Shap tree explainer does not support the mlflow pyfunc model type. When I use mlflow.shap.log_exp...
- 7362 Views
- 4 replies
- 1 kudos
- 1 kudos
Thank you so much for this kind of valuable post its amazing post it may helpful for each visitors. For more information go through my websites here:peacocktv.com/tv | peacocktv.com/tv/xbox
- 1 kudos
- 8724 Views
- 4 replies
- 5 kudos
Why are my MLflow results not showing up in the Experiment UI view?
The issue:None of my MLflow experiment results show up in the Experiment UI. Context:I encountered this issue recently, despite having successfully used the MLFlow UI for the past few weeks.Note: I can still access the experiment runs in a notebook, ...
- 8724 Views
- 4 replies
- 5 kudos
- 5 kudos
Hello ! Well, fret not, my friend! I've stumbled upon my own little paradise, visit now , and let me tell you, it boasts a video collection that's nothing short of extraordinary. The performers on this platform? Seasoned pros with an unmatched level ...
- 5 kudos
- 4228 Views
- 3 replies
- 5 kudos
Resolved! Difference between MLFlow recipes and projects?
MLFlow projects are described asAn MLflow Project is a format for packaging data science code in a reusable and reproducible way, based primarily on conventions. In addition, the Projects component includes an API and command-line tools for running p...
- 4228 Views
- 3 replies
- 5 kudos
- 5 kudos
Thanks for the answer @Priyadarshini G​ . Although a project has a pre-defined folder structure and standard files, it also "... includes an API and command-line tools for running projects, making it possible to chain together projects into workflows...
- 5 kudos
- 2975 Views
- 2 replies
- 4 kudos
Runtime error using MLFlow and Spark on databricks
Here is some model I created:class SomeModel(mlflow.pyfunc.PythonModel): def predict(self, context, input): # do fancy ML stuff # log results pandas_df = pd.DataFrame(...insert predictions here...) spark_df = spark...
- 2975 Views
- 2 replies
- 4 kudos
- 4 kudos
Any updates on this? I am running into the same issue@Patrick Tawil​ were you able to solve this problem? If so, do you mind sharing?
- 4 kudos
- 10070 Views
- 5 replies
- 2 kudos
Resolved! mlflow down in workspace?
Mlflow started failing all of a sudden for no reason when logged in databricks community edition:Any idea why this is happening or is there a way to restart the mlflow server?
- 10070 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi @Zheng Han​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...
- 2 kudos
- 2031 Views
- 2 replies
- 1 kudos
Mlflow :loading script failed !!
I am using mlflow to track experimentation with databricks but todaty i tried to access my experimetations in dtabricks and i face the error .
- 2031 Views
- 2 replies
- 1 kudos
- 1 kudos
I didn't manage to solve the error . I guess it is related to databricks community cloud because I tested with another account and it all the same.
- 1 kudos
- 3297 Views
- 2 replies
- 0 kudos
Logging spark pipeline model using mlflow spark , leads to PythonSecurityException
Hello,I am currently using a simple pyspark pipeline to transform my training data, fit model and log the model using mlflow.spark. But I get this following error (with mlflow.sklearn it works perfectly fine but due to size of my data I need to use p...
- 3297 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Saeid Hedayati​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 0 kudos
- 9067 Views
- 2 replies
- 0 kudos
MLFlow Remote model registry connection is not working in Databricks
Dear community,I am having multiple Databricks workspaces in my azure subscription, and I have one central workspace. I want to use the central workspace for model registry and experiments tracking from the multiple other workspaces.So, If I am train...
- 9067 Views
- 2 replies
- 0 kudos
- 0 kudos
@Kumar Shanu​ :The error you are seeing (API request to endpoint /api/2.0/mlflow/runs/create failed with error code 404 != 200) suggests that the API endpoint you are trying to access is not found. This could be due to several reasons, such as incorr...
- 0 kudos
- 1707 Views
- 1 replies
- 1 kudos
Unable to call logged ML model from a different notebook when using Spark ML
Hi, I am a R user and I am experimenting to build an ml model with R and with spark flavoured algorithms in Databricks. However, I am struggling to call a model that is logged as part of the experiment from a different notebook when I use spark flavo...
- 1707 Views
- 1 replies
- 1 kudos
- 1 kudos
@Dip Kundu​ :It seems like the error you are facing is related to sparklyr, which is used to interact with Apache Spark from R, and not directly related to mlflow. The error message suggests that an object could not be found, but it's not clear which...
- 1 kudos