cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

What is the best practice for applying MLFlow to clustering algorithms?

User16826993440
Databricks Employee
Databricks Employee

What is the best practice for applying MLFlow to clustering algorithms? What are the kinds of metrics customers track?

2 REPLIES 2

Joseph_B
Databricks Employee
Databricks Employee

Good question! I'll divide my suggestions into 2 parts:

(1) In terms of MLflow Tracking, clustering is pretty similar to other ML workflows, so not much changes.

(2) In terms of specific parameters, metrics, etc. to track, clustering is very different, so being aware of common and useful things to track is helpful.

For (1), the generic pieces of an ML workflow should be tracked in the same way as for classification, regression, and other problems:

  • Params, especially whatever hyperparameters you changed from defaults
  • Metrics (see below)
  • Data source and version
  • Code / notebook
  • etc.

For (2), I'll list some recommendations I have for important params, metrics, etc., but I'll be interested to hear from others, especially if you have links to more detailed resources.

The "right" metrics to use can be very problem-dependent and model-dependent. At a high level, I'd make sure to log:

Hope this helps!

wallco26
New Contributor III

Does it make sense to register a Kmeans clustering model once the experiment has been tracked and you are satisfied with the outcome? If so, how do you do it?

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group