💡 ML Training Tip Of The Week #4: Speed up your ML workload with one toggle

lin-yuan — Tue, 11 Mar 2025 06:16:50 GMT

Machine learning workload can be time-consuming due to training data processing such as ETL and feature engineering and iterative model training steps.

Recent ML Runtime releases have made both faster by 2X with a single toggle during the cluster creation:

Photon (available in MLR 15.2 or above), which is the C++ query execution engine, can speed up Spark SQL and Spark DataFrame that are commonly used in ETL and feature engineering by 1.5~2X
Graviton instances (available in MLR 15.4 LTS or above) powered by ARM-based CPUs designed by AWS can speed up XGBoost, LightGBM, etc. algorithms by up to 1.5X.

To enable Photon, select “Use Photon Acceleration” when creating a cluster as shown below:

To use ARM-based Graviton instance on AWS, search for “7g” in the instance types as shown below:

Read more about the use case of Photon and Graviton in the Databricks blog posts:

article 💡 ML Training Tip Of The Week #4: Speed up your ML workload with one toggle in Technical Blog

💡 ML Training Tip Of The Week #4: Speed up your ML workload with one toggle