cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
colinsorensen
by • New Contributor III
  • 2629 Views
  • 3 replies
  • 1 kudos

"All trials either failed or did not return results to hyperopt." AutoML is not working on a fairly simple classification problem.

First the exploratory notebook fails, though when I run it manually it works just fine.After that, the AutoML notebook eventually fails without completing any trials. I get this: Tried to attach usage logger `pyspark.databricks.pandas.usage_logger`, ...

Data Engineering
Reply
  • 2629 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
colinsorensen
New Contributor III
  • 1 kudos

Ultimately this problem magically resolved itself. I think I updated the cluster or something.

  • 1 kudos
2 More Replies
Qwetroman
by • New Contributor
  • 1616 Views
  • 1 replies
  • 0 kudos

AutoML runs fail after 5 seconds

Hi everyoneI am exploring automl, and I met a strange problem - after I launch a classification experiment on my personal newly created cluster (screenshot attached) it successfully performs data exploration, but after that, all runs fail after appro...

Data Engineering
Reply
  • 1616 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
swethaNandan
Databricks Employee
  • 0 kudos

Hi Qwetroman,we can see the following error message in the notebook - ExecutionTimeoutError: Execution timed out before any trials could be successfully run. Please increase the timeout for AutoML to run some trials.What's the size of the dataset? St...

  • 0 kudos
anonturtle
by • New Contributor
  • 1504 Views
  • 1 replies
  • 0 kudos

How does automl classify which feature is numeric or categorical?

When running automl on its UI, it classifies a feature "local_convenience_store" as both a numeric and categorical column. This affects the result as for numeric columns a scaler is used while in a categorical column it is one hot encoded. For contex...

Data Engineering
Reply
  • 1504 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@hr then​ :The approach taken by AutoML to classify features as numeric or categorical depends on the specific AutoML framework or library being used, as different implementations may use different methods or heuristics to make this determination.In ...

  • 0 kudos
RRO
by • Contributor
  • 1653 Views
  • 1 replies
  • 3 kudos

AutoML forecasting with monthly data?

ARIMA and FBProphet have the capability to forecast monthly data. When using AutoML (via the API or the UI) it seems like it is not possible to have a monthly freq (e.g. 'MS').Is there a way / workaround to make it work with monthly data or is it pla...

Data Engineering
Reply
  • 1653 Views
  • 1 replies
  • 3 kudos
View Replies
Latest Reply
MateuszLomanski
New Contributor II
  • 3 kudos

It is possible to use AutoML to forecast monthly data, but it may require some additional steps or adjustments.One approach is to resample the monthly data to a lower frequency such as weekly or daily, and then use AutoML to forecast at that lower fr...

  • 3 kudos
Christine
by • Contributor II
  • 8808 Views
  • 7 replies
  • 5 kudos

Resolved! autoML' is not found when using databricks.automl with runtime 112.ML (and runtime 10.4 LTS ML).

I have tried to set up a autoML experiment with runtime 11.2ML and data from a delta table. However I receive the error "ModuleNotFoundError: No module named 'databricks.automl'" and "AutoML not available: Use Databricks Runtime 8.3 ML or above." tho...

image
Data Engineering
Reply
  • 8808 Views
  • 7 replies
  • 5 kudos
View Replies
Latest Reply
Christine
Contributor II
  • 5 kudos

I deleted the cluster and created a new with runtime 9.1 LTS ML which solved the problem.

  • 5 kudos
6 More Replies
Hayley
by • Databricks Employee
  • 3466 Views
  • 1 replies
  • 2 kudos

What is the best way to do EDA in Databricks?

Are there example notebooks to quickstart the exploratory data analysis?

Data Engineering
Reply
  • 3466 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Hayley
Databricks Employee
  • 2 kudos

A quick way to start exploratory data analysis is to use the EDA notebook that is created when you use Databricks AutoML. Then you can use the notebook generated as is, or as a starting point for modeling. You’ll need a cluster with Databricks Runtim...

  • 2 kudos
Labels