How to isolate environments for different projects in a single mlflow server?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-30-2022 10:05 AM
I am planning to deploy MLFlow server deployed in AWS ECS as a centralised repositories for my machine learning experiments and runs and to strore events and artifacts. I would like to use MLflow Tracking Server enabled with proxied artifact storage access to eliminate the need to have authentication to the underlying storage(S3 bucket) at my side(client). And I would like to have different environments or isolated environments as i keep creating projects? for example, for project titanic, i might have 10 experiments, which i would not want to see from project iris and this project will have different experiments. How do I achieve this?
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-01-2022 04:12 PM
You would create a new experiment for each dataset yo just change the name. https://www.mlflow.org/docs/latest/python_api/mlflow.html#mlflow.create_experiment
For a new environment,
https://mlflow.org/docs/latest/cli.html#cmdoption-mlflow-models-predict-env-manager
There is a slackworkspace that is very helpful too:
https://join.slack.com/t/mlflow-users/shared_invite/zt-g6qwro5u-odM7pRnZxNX_w56mcsHp8g
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-01-2022 09:46 PM
I understand an experiment in the context of mlflow. i will be creating multiple experiments for a single project. and i want to not see those experiments when i start a new project but not delete them. may be some switch to show and let me usa a different page for experiments and runs, while using the same server.
by environment' i didn't mean a python or conda virtual environment. i shouldn't have used the word 'environment'. i meant to convey different (or may be a new) page or space for experiments for every new project i start, while sticking to the same mlflow server deployed in the same place(this is going to be AWS ECS)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-14-2022 11:52 PM
yes i still need help. @josephk’s reply hasn’t helped me. I still need to know how I can isolate or have different sets of experiments in order to use a single server for different projects.