Hi @Nhat Hoangโ ,
While not Databricks-specific, here's a good answer:
"If you perform the encoding before the split, it will lead to data leakage (train-test contamination). In this sense, you will introduce new data (integers of Label Encoders) and use it for your models thus it will affect the end predictions results (good validation scores but poor in deployment).
After the train and validation data category is already matched up, you can perform fit_transform on the train data, then only transform for the validation data - based on the encoding maps from the train data.
Almost all feature engineering like standardization, Normalisation etc should be done after the train test split. "
Additionally, if you were to run an AutoML experiment and look at the underlying notebook you should see that the data is split before encoding is applied.
