topic How to isolate environments for different projects in a single mlflow server? in Machine Learning

How to isolate environments for different projects in a single mlflow server?

naveen_marthala — Sat, 30 Apr 2022 17:05:33 GMT

I am planning to deploy MLFlow server deployed in AWS ECS as a centralised repositories for my machine learning experiments and runs and to strore events and artifacts. I would like to use MLflow Tracking Server enabled with proxied artifact storage access to eliminate the need to have authentication to the underlying storage(S3 bucket) at my side(client). And I would like to have different environments or isolated environments as i keep creating projects? for example, for project titanic, i might have 10 experiments, which i would not want to see from project iris and this project will have different experiments. How do I achieve this?

Thanks.

Re: How to isolate environments for different projects in a single mlflow server?

Anonymous — Sun, 01 May 2022 23:12:27 GMT

You would create a new experiment for each dataset yo just change the name. https://www.mlflow.org/docs/latest/python_api/mlflow.html#mlflow.create_experiment

For a new environment,

https://mlflow.org/docs/latest/cli.html#cmdoption-mlflow-models-predict-env-manager

There is a slackworkspace that is very helpful too:

https://join.slack.com/t/mlflow-users/shared_invite/zt-g6qwro5u-odM7pRnZxNX_w56mcsHp8g

Re: How to isolate environments for different projects in a single mlflow server?

naveen_marthala — Mon, 02 May 2022 04:46:49 GMT

I understand an experiment in the context of mlflow. i will be creating multiple experiments for a single project. and i want to not see those experiments when i start a new project but not delete them. may be some switch to show and let me usa a different page for experiments and runs, while using the same server.

by environment' i didn't mean a python or conda virtual environment. i shouldn't have used the word 'environment'. i meant to convey different (or may be a new) page or space for experiments for every new project i start, while sticking to the same mlflow server deployed in the same place(this is going to be AWS ECS)

Re: How to isolate environments for different projects in a single mlflow server?

naveen_marthala — Wed, 15 Jun 2022 06:52:45 GMT

yes i still need help. @josephk’s reply hasn’t helped me. I still need to know how I can isolate or have different sets of experiments in order to use a single server for different projects.