Hi @lizou1 ,

Databricks serverless compute jobs automatically scale based on workload, so there’s no need to manually configure clusters, Databricks manages that for you.

However, autoscaling doesn’t always resolve issues caused by data skew or shared resource limits. Try disabling Photon if many rows share the same key, and consider breaking large joins into smaller steps or optimizing your data layout.

Also, ensure your data is UTF-8 encoded before loading it into Databricks for the best results.