cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

What added advantage would using an Sql Alchemy database have, while I have an artifact destination for my MlFlow server?

naveen_marthala
Contributor

I am setting up mlflow server with Postgres and S3 on AWS ECS(or AWS EC2) for personal usage. I would like to know if using Postgres would actually give me any benefit?

as shown in scenario 5 in docs, I would like to set up server with proxied artifact storage, but without postgres in AWS RDS and also not in `--artifacts-only` and `--artifacts-only` modes as shown in scenario 6 in docs. So, if i launch a mlflow tracking server like this, will all the metadata that usually gets saved to an SQLAlchemy database(postgres in my case) be saved to the location of attribute `--artifacts-destination` as files?

I would like to do this to save costs from AWS RDS that runs Postgres.

If you aren't familiar with AWS ECS, it is a fully managed service (a simpler version of kubernetes) that runs container base applications. and EC2 is just user managed machines or servers.

2 REPLIES 2

Kaniz
Community Manager
Community Manager

Hi @Naveen Marthala​, Here is a step-by-step guide to set up MLflow with a Postgres DB for storing metadata and a systemd unit to keep it running. Please have a look and let us know if that helps.

Thanks for trying to help me @Kaniz Fatma​ .

My question was not to learn how to set up mlflow with postgres. But, to learn what additional advantage would using an sql-alchemy database like postgres would have, instead of storing all the metadata to files. that is, what advantage would I have, if I use an sql-alchemy for backend and not use one. please note that, the mlflow server will not be in '--artifacts-only' mode?

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!