Hello @szymon_dybczak 

Let me rephrase the problem... so, I did set following parameters on the pipeline advanced configuration in order to alter the shuffle partitions however they are not taking effect and this is what the actual issue is:

spark.sql.shuffle.partitions = 20

spark.databricks.adaptive.autoOptimizeShuffle.enabled = true

During wide transformation, still see the partitions as 200