cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

How to input initial centroids to K-Means or GMM Clustering in SparkML ?

HariK1
New Contributor II

Hi, I want to use KMeans Model or Gaussian Mixture Model algorithm for clustering using the SparkML library, in which I want to specify the initial centroids. The option of giving initial centroids is there in the Spark MLlib (RDD based APIs) however not available in the Pyspark DataFrame based APIs (SparkML). Since Spark MLlib is in the maintenance mode, I do not want to use it and instead use SaprkML library. Is anyone knows a workaround? Thank You.

2 REPLIES 2

HariK1
New Contributor II

Hi @Kaniz Fatma​ , thanks for your response. Since MLlib is currently in the maintenance mode we don't want to use it in production and hence want to use SparkML (data frame based APIs). Here is the problem. In SparkML libraries there is no mention of how to input user specified centroids for K-Means or GMM. That was my question.

HariK1
New Contributor II

@Kaniz Fatma​ I still haven't got an answer to my question!!!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group