Soma
Valued Contributor

df1=df.withColumn("NewColumn",functions.sum("Magnitude").over(Window.partitionBy("Category"))

/functions.sum("Magnitude").over(Window.partitionBy(functions.lit("1"))))