topic Re: Best practice for model promotion so that models are not removed from previous stage in Machine Learning

Best practice for model promotion so that models are not removed from previous stage

thibault — Fri, 02 Dec 2022 13:56:11 GMT

Hi,

Using Model Registry to promote models is great. However, I am facing an issue, where multiple Databricks workspaces (SIT / UAT / Prod) use a model at various stages (Staging for SIT and UAT, Production for Prod workspace).

We have a workflow running in all environments, everything is equal except input and output data, and the model staging state. This means that the workflow fails in SIT & UAT as soon as the model is promoted to Production state, since it no longer exists in Staging state.

Is there a way to promote a model, but still keeping a copy of it with the "None" or "Staging" state? Otherwise, what would be a good practice to keep the testing environments running with the same model?

Thanks a lot any MLFlow / Databricks experts!

Re: Best practice for model promotion so that models are not removed from previous stage

User16502773013 — Mon, 30 Jan 2023 20:09:28 GMT

Hello Thibault,

For reusing already built model there are multiple options:

Register the model from dev to QA model registry as described here OR
- In this scenario only the registered model will be copied
- Lineage to runs is not possible
You can copy dev's registered model and the MLflow runs that are linked to its versions to QA workspace (details here and here)
- This approach will keep the lineage
- This approach has some limitations (listed here)

If the ask is changing the stage of the registered model version in the current registry:

You can only have one stage per version. However, you can register the same run's MLflow model with more than one registered model (version). which is available using API (example here)

I am happy to help with any further information needed.

Regards

Re: Best practice for model promotion so that models are not removed from previous stage

thibault — Tue, 31 Jan 2023 20:10:11 GMT

Thanks for your reply!

I see, so as I see it, a clean way to do it is to

have a model registry per workspace
use the mlflow import/export tool to copy a model from dev -> sit -> qa -> prod workspaces based on cicd and mlops
not use the model stage functionality and use only one stage

Benefits :

simplicity as there is no dependency on the model staging transitions
easily integrates with mlops and cicd
keeps the lineage

Cons :

need to create a model registry per workspace

Does this sound reasonable? I'll give it a try and see how much I can automate from model building in dev to prod through my cicd pipeline.

Re: Best practice for model promotion so that models are not removed from previous stage

GregoryNation — Wed, 29 May 2024 09:23:34 GMT

Thanks for interesting information.

Re: Best practice for model promotion so that models are not removed from previous stage

KarenGalvez — Mon, 03 Jun 2024 08:35:34 GMT

Thats what I need