Hi @Suheb , 

For large datasets or distributed training, use Apache Spark MLlib RandomForest on Databricks; trees are trained in parallel and scale with cluster size.
Ref Doc -  https://www.databricks.com/blog/2015/01/21/random-forests-and-boosting-in-mllib.html

Index categorical features (StringIndexer) and avoid one-hot encoding; ensure maxBins is at least the highest categorical cardinality so splits are meaningful.
Cache the prepared training data before fitting—Spark tree algorithms benefit from caching due to iterative passes over data.
Use Optuna with MLflow 3 on Databricks ML Runtime 17.0+ for parallel, distributed searches (MlflowSparkStudy + MlflowStorage), and autolog runs to MLflow.
Tune key RF params: numTrees, maxDepth, featureSubsetStrategy (“auto”), minInstancesPerNode, and maxBins. Increasing trees usually improves test error but increases training time; keep trees reasonably deep to avoid overfitting and long runtimes.
Use CrossValidator or TrainValidationSplit with MLlib, Put your data prep stages outside the cross-validator (wrap the CV inside the Pipeline), so prep isn’t re-run for every hyperparam trial; this saves time at scale.

If Random Forest plateaus, try XGBoost via xgboost.spark for distributed gradient boosting within Spark ML Pipelines; set num_workers to sc.defaultParallelism for full-cluster training.