• Databricks Community
  • Machine Learning
  • Re: Parallelization in training machine learning m...

Anonymous
Not applicable
Options
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Subscribe to RSS Feed
  • Permalink
  • Print
  • Report Inappropriate Content

‎10-13-2022 08:15 AM

This blog should be very helpful:

https://www.databricks.com/blog/2021/04/15/how-not-to-tune-your-model-with-hyperopt.html

Here are the docs on xgboost

https://docs.databricks.com/machine-learning/train-model/xgboost.html

A simple rule is never use sc.parallelize.

2 Kudos
Reply
Powered by Khoros