topic Re: Difference between MLFlow recipes and projects? in Machine Learning

Difference between MLFlow recipes and projects?

smedegaard — Sat, 10 Jun 2023 07:28:38 GMT

An MLflow Project is a format for packaging data science code in a reusable and reproducible way, based primarily on conventions. In addition, the Projects component includes an API and command-line tools for running projects, making it possible to chain together projects into workflows.

MLFlow Rcipes (previously "pipelines") are described as

MLFlow Recipes is a framework that enables data scientists to quickly develop high-quality models and deploy them to production. Compared to ad-hoc ML workflows, MLflow Recipes offers several major benefits...

They see to serve the same purpose.

What's the difference?
When does it make sense to use one over the other?

Re: Difference between MLFlow recipes and projects?

Priyag1 — Sun, 11 Jun 2023 17:19:36 GMT

@Anders Smedegaard Pedersen Each project is simply a directory of files, or a Git repository, containing your code whereas recipe is an ordered composition of Steps used to solve an ML problem or perform an MLOps task, such as developing a regression model or performing batch model scoring on production data. MLflow Recipes provides APIs and a CLI for running recipes and inspecting their results.

Here a Step represents an individual ML operation, such as ingesting data, fitting an estimator, evaluating a model against test data, or deploying a model for real-time scoring. Each Step accepts a collection of well-defined inputs and produce well-defined outputs according to user-defined configurations and code.

Hope this helps you.

Re: Difference between MLFlow recipes and projects?

Anonymous — Wed, 14 Jun 2023 05:51:15 GMT

Hi @Anders Smedegaard Pedersen

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!

Re: Difference between MLFlow recipes and projects?

smedegaard — Thu, 15 Jun 2023 11:37:40 GMT

Thanks for the answer @Priyadarshini G . Although a project has a pre-defined folder structure and standard files, it also "... includes an API and command-line tools for running projects, making it possible to chain together projects into workflows." and we can "run multi-step workflows" with projects, either with the cli or mlflow.projects.run()

"Recipe Templates are git repositories with a standardized, modular layout.". In a recipe we define the flow in recipe.yaml, define our steps in python files in the ./steps folder, and profiles and tests in the corresponding folders

they might have slightly different interfaces, but they still seem to cover 95% of the same needs?

Please enlighten me if there is something I am not seeing.