topic Re: autoML' is not found when using databricks.automl with runtime 112.ML (and runtime 10.4 LTS ML). in Data Engineering

autoML' is not found when using databricks.automl with runtime 112.ML (and runtime 10.4 LTS ML).

Christine — Fri, 09 Sep 2022 11:20:43 GMT

I have tried to set up a autoML experiment with runtime 11.2ML and data from a delta table. However I receive the error "ModuleNotFoundError: No module named 'databricks.automl'" and "AutoML not available: Use Databricks Runtime 8.3 ML or above." though I am using a runtime newer than 8.3. Does anyone know how to get the module so I can proceed with autoML?

Re: autoML' is not found when using databricks.automl with runtime 112.ML (and runtime 10.4 LTS ML).

Anonymous — Fri, 09 Sep 2022 21:44:12 GMT

It should be

from databricks import automl

The table doesn't need to be delta, but does need to be a table in the metastore.

Re: autoML' is not found when using databricks.automl with runtime 112.ML (and runtime 10.4 LTS ML).

Christine — Mon, 12 Sep 2022 06:15:08 GMT

Hi @Joseph Kambourakis

When I run that code I get the message on the figure below. It seems that my "databricks" library doesn't include automl.

(I am using runtime 11.2 ML though it states that autoML is not available for my cluster. I have tried earlier versions too.)

Re: autoML' is not found when using databricks.automl with runtime 112.ML (and runtime 10.4 LTS ML).

Anonymous — Mon, 12 Sep 2022 16:29:37 GMT

You do not need to install this library. It is automatically installed within the newest (10.0 or above) Databricks ML runtimes. Use 10.4 ML LTS for the more stable runtime and use the newest runtimes for the latest features.

Re: autoML' is not found when using databricks.automl with runtime 112.ML (and runtime 10.4 LTS ML).

Christine — Tue, 13 Sep 2022 06:18:20 GMT

hi @Salma Mayorquin

In the original post I was using runtime 11.2ML and I have now tried with runtime 10.4ML, but I still get the same error.

Re: autoML' is not found when using databricks.automl with runtime 112.ML (and runtime 10.4 LTS ML).

Christine — Tue, 13 Sep 2022 10:54:58 GMT

I deleted the cluster and created a new with runtime 9.1 LTS ML which solved the problem.

Re: autoML' is not found when using databricks.automl with runtime 112.ML (and runtime 10.4 LTS ML).

sean_owen — Mon, 19 Sep 2022 22:20:35 GMT

Just to confirm, this is correct, and I do not see this problem on 11.2 ML. The software is installed, and your import is correct. I am not sure why you observe this; are init scripts modifying the ML runtime when the cluster starts, maybe?

Re: autoML' is not found when using databricks.automl with runtime 112.ML (and runtime 10.4 LTS ML).

Christine — Tue, 20 Sep 2022 06:03:48 GMT

I used the script provided by databricks as I use the autoML incorporated in databricks, so I would not assume that it modifies the ML runtime. I think there must have been something wrong with the cluster configuration, because I have created a new cluster and now it seems to work.