11-18-2024 02:49 AM
@genevive_mdonça You have calculate the correct number of shuffle partitions for your case considering the cluster configurations.
Please follow this doc to calculate it: https://www.databricks.com/discover/pages/optimize-data-workloads-guide
