Two or more different ml model on one cluster.

[TomasP]

Hi, have you already dealt with the situation that you would like to have two different ml models in one cluster? i.e: I have a project which contains two or more different models with more different pursposes. The goals is to have three different real-time ml endpoints but these models should be deployed only on one cluster.

Tomas

[Debayan]

Hi, @Tomas Peterek​  you can go through the MLflow guide: https://docs.databricks.com/mlflow/index.html

If I have understood it correctly, if you are running mlflow models in jobs runs in that case , each one of the jobs/job-runs gets a dedicated cluster that turns off right after the job finishes. It’s possible running a lot of clusters in parallel in order to execute many independent jobs. In a job cluster a single job run deploys a single cluster which cannot be shared. Please correct me if I am wrong.

[TomasP]

Hi @Debayan Mukherjee​  Thanks for answer. Yes, I am familiar with the classical approach. I'm more interested if there is any work around. For two model Im able to transfer one model to production stage and second model to staging. Both of them have their own containers and have their own endpoints.  it does not matter if one is designed in tensorflow and second one in pytorch. But I would like to find way how to deploy more models on one cluster. I know it goes against mlflow concept but the aim is to save costs. 

[Anonymous]

Hi @Tomas Peterek​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!