Hi, This is a super old question but answering in case anyone else comes across it.

This isn't working because the add months expects an integer rather than a column name you can get round this by using expr() inside withColumn:

from pyspark.sql.functions import expr

df = df.withColumn(
"history_effective_quarter",
expr("add_months(history_effective_month, -(month(history_effective_month) % 3) + 1)")
)

Thanks,

Emma