Hi Community,
I am currently evaluating various gradient boosting options on Databricks using production-level data, including the CatBoost Spark integration (ai.catboost:catboost-spark).
I would love to hear from others who have successfully used this specific integration for production workloads. How have you found its stability and resource requirements, particularly concerning the driver, compared to alternatives like XGBoost Spark or LightGBM (via SynapseML)?
Are there any other preferred libraries or approaches for robust gradient-boosting training within the Databricks environment?
Thank you for sharing your insights!
