error: not found: type XGBoostEstimator

[amal15]

error: not found: type XGBoostEstimator 

Spark & Scala 

[Kaniz_Fatma]

Hi @amal15, The error message you’re encountering, “XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark,” indicates that the XGBoostEstimator class is not being recognized within the specified package. 

1. Check Dependencies:

   • Ensure that you have the necessary dependencies correctly set up in your project.
   • Verify that you have the XGBoost library (both the XGBoost4J and XGBoost4J-Spark components) properly installed.
   • If you’re using Spark, make sure you have the appropriate version of XGBoost4J-Spark compatible with your Spark version.

2. Classpath Configuration:

   • When running your Spark application, make sure the XGBoost JAR is included in the classpath.
   • You can specify the JAR using the --conf spark.jars=/path/to/xgboost4j.jar option when launching Spark. For example:

     ./bin/spark-shell --conf spark.jars=/path/to/xgboost4j.jar
     

3. Import Statements:

   • Double-check your import statements in your Scala code.
   • Make sure you’re importing the XGBoostEstimator class from the correct package. The correct import should look like this:

     import ml.dmlc.xgboost4j.scala.spark.XGBoostEstimator
     

4. Partition Count (Advanced):

   • In some cases, the error might be related to the expected number of partitions in your training data.
   • Refer to the XGBoost4J-Spark source code for insights into how the partition count is handled.

If you continue to face issues, feel free to provide additional context or ask for further assistance! 😊

For more details, you can refer to the Databricks community forum.¹²³

To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback not only helps us assist you better but also benefits other community members who may have similar questions in the future.

If you found the answer helpful, consider giving it a kudo. If the response fully addresses your question, please mark it as the accepted solution. This will help us close the thread and ensure your question is resolved.

We appreciate your participation and are here to assist you further if you need it!

[shan_chandra]

@amal15 - can you please include the below to the import statement and see if it works.

ml.dmlc.xgboost4j.scala.spark.XGBoostEstimator