Migration from self-managed MLflow to Databricks managed MLflow.

[Maverick1]

Would it be possible to somehow save the data, metrics of all experiments captured by self-managed mlflow using A/mazon RDS, S3 as backend and then load it to databricks managed mlflow and make it available in the UI? This is required as a part of migration activity.

[User16783853898]

You can migrate your MLflow experiment and runs using the MLflow API with this toolkit: https://github.com/amesar/mlflow-export-import

[Maverick1]

@Andre Mesarovic​ : Is this solution, Databricks approved for enterprise usage?

I believe this is not an approved solution and also I can see many issues in it while doing import and export like nested metrics are not imported correctly etc.

[User16783853898]

mlflow-export-import is an open source tool used by both OSS and Databricks users. It is in the process of being a recommended solution for Databricks MLflow migrations. Create a ticket for any issues at: https://github.com/amesar/mlflow-export-import/issues

[Maverick1 · Accepted solution]

@Kaniz Fatma​ : Hi Fatima,

mlflow-export-import is still not a recommended solution and has no support from Databricks. This is clarified by Databricks SA's themselves.

I have developed an another approach where I am capturing the statistics of a particular run which is needed to be migrated to higher env. (like pre-prod or prod) and then creating a mlflow run with those statistics instead of re-running the entire huge model. This is working perfectly and is within the logical support of Databricks.