Ok I see,

do not have a clear answer for that.

But there was a session at spark summit about this (well, more CI/CD related but it might give ideas):

https://databricks.com/session_na20/productionalizing-models-through-ci-cd-design-with-mlflow

The whole ci/cd shebang might be too much for your needs but maybe you can pick some parts which are useful?