https://community.databricks.com/t5/machine-learning/in-spark-mllib-what-is-the-difference-between-an-estimator-and-a/m-p/23204#M1313 Thu, 17 Jun 2021 15:05:21 GMT User16826992666 2021-06-17T15:05:21Z https://community.databricks.com/t5/machine-learning/in-spark-mllib-what-is-the-difference-between-an-estimator-and-a/m-p/23204#M1313 Thu, 17 Jun 2021 15:05:21 GMT https://community.databricks.com/t5/machine-learning/in-spark-mllib-what-is-the-difference-between-an-estimator-and-a/m-p/23204#M1313 User16826992666 2021-06-17T15:05:21Z https://community.databricks.com/t5/machine-learning/in-spark-mllib-what-is-the-difference-between-an-estimator-and-a/m-p/23205#M1314 <P>These terms are borrowed from scikit-learn, and the idea is the same. A transformer is just a component of a pipeline that transforms the data in some way. An estimator is also a transfomer, but one that additionally needs to be 'fit' on data before it knows how to transform.</P><P></P><P>For example, a StringTokenizer is just a transformer, because it does not need to see any data to know what to do, to tokenize strings. A machine learning model like LogisticRegression is also a transformer, because it transforms data by adding a prediction. However it has to be fit on data first before it can do so. So it is (also) an estimator.</P> Thu, 17 Jun 2021 18:21:49 GMT https://community.databricks.com/t5/machine-learning/in-spark-mllib-what-is-the-difference-between-an-estimator-and-a/m-p/23205#M1314 sean_owen 2021-06-17T18:21:49Z