ML Practioner | ml 09 - automl notebook | error on importing databricks.automl

[Slalom_Tobias]

executing the following code...

from databricks import automl

summary = automl.regress(train_df, target_col="price", primary_metric="rmse", timeout_minutes=5, max_trials=10)

generates the error...

ImportError: cannot import name 'automl' from 'databricks' (/local_disk0/.ephemeral_nfs/envs/pythonEnv-7a7f3e61-af65-4995-9ebc-bb12f01b5c60/lib/python3.8/site-packages/databricks/__init__.py)

---------------------------------------------------------------------------

ImportError Traceback (most recent call last)

<command-1508431422070671> in <module>

----> 1 from databricks import automl

2

3 summary = automl.regress(train_df, target_col="price", primary_metric="rmse", timeout_minutes=5, max_trials=10)

ImportError: cannot import name 'automl' from 'databricks' (/local_disk0/.ephemeral_nfs/envs/pythonEnv-7a7f3e61-af65-4995-9ebc-bb12f01b5c60/lib/python3.8/site-packages/databricks/__init__.py)

Command took 0.12 seconds -- by tobiasc@slalom.com at 7/30/2022, 1:44:21 PM on SandboxML 10.3

prior to attempting to execute the above code, the required packages were successfully installed with the following statement...

pip install mlflow hyperopt databricks xgboost

i've spent a few hours attempting to troubleshoot this issue so really hoping somebody can help me out with a solution.

i'm using the "Machine Learning" persona.

[Prabakar · Accepted solution]

@Tobias Cortese​  I could see you are using the standard DBR version and it is expected. Please use the ML DBR version and it should resolve your issue.

[Slalom_Tobias]

Thanks Prabakar!

The root issue is that my company's sandbox didn't have an ML cluster available so I was trying to import the required libraries, which don't seem to be publicly available. I'm now guessing that's done purposefully to protect the code. I've tested this within community edition and am asking the admins for my company sandbox to stand-up an ML cluster.

[artsheiko]

@Tobias Cortese​, the value added to use ML runtime is in a fact that the Databricks Runtime ML includes a variety of popular ML libraries that are updated with each release to include new features and fixes.

In addition, Databricks has a subset of the supported libraries as top-tier libraries. For these libraries, Databricks provides a faster update cadence, updating to the latest package releases with each runtime release (barring dependency conflicts).

In addition to the pre-installed libraries, Databricks Runtime ML differs from Databricks Runtime in the cluster configuration and in how you manage Python packages.

Finally, ML runtime includes tools to automate the model development process and help you efficiently find the best performing model : AutoML, Managed MLFlow, Hyperopt.

Hope that you'll get ML runtime at your disposal

[Krueger156]

I'm happy to see a particularly subject.