cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Custom AutoML pipeline: Beyond StandardScaler().

sharpbetty
New Contributor II

The automated notebook pipeline in an AutoML experiment applies StandardScaler to all numerical features in the training dataset as part of the PreProcessor. See below.

sharpbetty_0-1728884608851.png

But I want a more nuanced and varied treatment of my numeric values (e.g. I have log transforms, winsorization, sigmoid transforms etc etc)

I want to either:
a) Remove all feature engineering / scaling from the automated Preprocessor and have the AutoML notebook run my features as presented to the AutoML experiment.
or
b) Edit the AutoML default pipeline to include a custom PreProcessor script to do more than simply scale every numeric value.

How can i achieve this? I can't seem to find any option to customize this in the UI AutoML Experiment setup and I've got no idea where to find code for the default pipeline that's invoked on every experiment.

0 REPLIES 0

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group