Hi all,
We are constructing our CI/CD pipelines with the Repos feature following this guide:
https://databricks.com/blog/2021/09/20/part-1-implementing-ci-cd-on-databricks-using-databricks-note...
I'm trying to implement my pipes for models that haven't been trained with the MLFlow paradigm. The code promotion with repos is the same as explained in the link.
However, the model artifact, that is, the serialized file that contains the trained model
on MLFlow travels from its own infrastructure (the MLFlow artifact version control). It points either to the DB FileStore or any other storage previously set.
WITHOUT the MLFlow scheme: what are good practices for promoting the Model Artifact ? Here are my thoughts:
- Save it on the data lake, and read the same direction throughout environments. Obstacle: How to add version control here??
- When the release pipeline triggers, include it on the Repository artifact.
- Any other?
I'm really confused about the solution.