Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-28-2026 02:45 AM
@tkfm_s
Yes, using SynapseML's LightGBMClassifier / LightGBMRegressor lets you train directly on a Spark DataFrame, no pandas conversion required and also ensure partitions match executor cores so LightGBM uses them all. And if you have wide range of columns it is advised to decrease them to avoid OOM.
Attaching the document for lightgbm distributed training:
https://lightgbm.readthedocs.io/en/latest/Parallel-Learning-Guide.html
Jahnavi N