AutoML Runs Failing

[miahopman]

After the Data Exploration notebook runs successfully, all AutoML trials fail without providing a source notebook. I have ensured that the training data labels have no null values or any labels with 16 or less occurrences associated with them. I cannot find a failure message anywhere, nor look through the trial notebook since it is not attached. How do I fix this error.

[Kaniz]

Hi @miahopman , 

It appears that the issue you're facing is related to AutoML trials failing in Databricks.

Here's a set of steps to troubleshoot and resolve the problem:

1. Check AutoML Run Settings:

   • Ensure that your AutoML run configuration settings are correct. This includes specifying the target column, training data, validation data, and other relevant parameters.
   • Refer to the Databricks AutoML documentation to validate that your run configuration settings follow the correct format.

2. Check Training Data and Labels:

   • Verify that your training data does not contain null or missing values, as this can affect the training process.
   • Make sure the label column has a sufficient number of distinct values, ideally no less than 16, to allow the AutoML model to accurately learn and extract features.

3. Check for Class Imbalance:

   • Examine the distribution of your label data to identify any class imbalance issues. If class imbalances are present, consider employing techniques like oversampling or undersampling to balance the classes.

4. Check Model Log:

   • Review the Model Log to search for error messages related to the failed trials. The Model Log should provide insights into the failed trials and their respective metrics, which can be valuable for debugging.

5. Attach and Run the Trial Notebook:

   • If no error messages or logs are available, consider attaching the trial notebook to the AutoML experiment and running a single trial. This can help you identify the specific error or exception preventing the trials from succeeding. Utilize the trial notebook for debugging and root cause analysis.

6. Check Cluster Configuration:

   • Ensure that the cluster configuration is suitable for your AutoML experiment. Verify that the cluster has adequate resources, including memory and cores, to support the AutoML process.

By diligently following these steps, you should be able to diagnose and address the issue that's causing your AutoML trials to fail in Databricks. If you encounter specific error messages or need further assistance, please provide more details for more targeted guidance.

[Annapurna_Hiriy]

@miahopman We understand that you are looking for a better way of troubleshooting in AutoML. We have an internal feature request raised to address precisely the issues you have discussed here.