cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Pyspark models iterative/augmented training capability

ChanduBhujang
New Contributor II

Does Pyspark tree based models have iterative or augmented training capabilities ? Similar to sklearn package can be used to train models using model artifact and use that model to train using additional data? 

 

#ML_Models_Pyspark

1 REPLY 1

Kumaran
Valued Contributor III
Valued Contributor III

Hi @ChanduBhujang,

Thank you for contacting Databricks community.

PySpark tree-based models do not have built-in iterative or augmented training capabilities like Scikit-learn's partial_fit method. While there are workarounds to update the model with new data, they may not be as efficient or effective as native support for incremental training.
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!