Error in automl.regress

[staskh]

Hi,

I'm running example notebook from https://docs.databricks.com/aws/en/machine-learning/automl/regression-train-api on a node with ML cluster 17.0 (includes Apache Spark 4.0.0, Scala 2.13) and getting error at

from databricks import automl

summary = automl.regress(dataset=train_pdf, target_col="MedHouseVal", timeout_minutes=30)

TypeError: Retry.__init__() got an unexpected keyword argument 'method_whitelist'

File <command-910054366217001>, line 2 1 from databricks import automl ----> 2 summary = automl.regress(dataset=train_pdf, target_col="MedHouseVal", timeout_minutes=30)

Any ideas/recommendations why and how to fix it?

Regards,

Stas

[ilir_nuredini]

Hello @staskh 

From the error, it looks related to the urllib3 library (used by requests), which Databricks AutoML relies on under the hood. The method_whitelist parameter was deprecated and removed starting from urllib3==1.26.0. As far as I know, databricks-automl is pulling in a later version, which causes the issue.

Can you try manually installing an earlier version of urllib3, for example:

%pip install "urllib3==1.25.11"
dbutils.library.restartPython()

Then re-run the code.

If that doesn’t work, try installing:

%pip install "requests==2.26.0" "urllib3==1.26.2"
dbutils.library.restartPython()

and see if it resolves the problem.

Hope that helps. Let me know how it goes, we can try something else if needed.

Best,
Ilir

[staskh · Accepted solution]

Ilir, greetings!

Thank you for a prompt response. Unfortunately, none of the suggested solutions works. I checked with Genie:
"The error occurs because databricks-automl is not available for Databricks Runtime 17.0.x. Databricks AutoML is not supported on this runtime version. You cannot install or use databricks-automl on 17.0.x clusters."

It seems correct, as I got AutoML working as expected once I switched to the 16.4 ML runtime.

Thank you

Stas

[ilir_nuredini]

Hello @staskh ,

Awesome, great you managed to solve it!

Best, Ilir