Hi @dyusuf ,

It could be because AQE (Adaptive Query Execution) is enabled.

...AQE, dynamically handles skew...

Please refer below link for more details:

https://docs.databricks.com/aws/en/optimizations/aqe

Can you please disable AQE and check if this works?

spark.conf.set("spark.sql.adaptive.enabled", "false")

HTH