cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Pyspark models iterative/augmented training capability

ChanduBhujang
New Contributor II

Does Pyspark tree based models have iterative or augmented training capabilities ? Similar to sklearn package can be used to train models using model artifact and use that model to train using additional data? 

 

#ML_Models_Pyspark

1 REPLY 1

Kumaran
Databricks Employee
Databricks Employee

Hi @ChanduBhujang,

Thank you for contacting Databricks community.

PySpark tree-based models do not have built-in iterative or augmented training capabilities like Scikit-learn's partial_fit method. While there are workarounds to update the model with new data, they may not be as efficient or effective as native support for incremental training.

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now