pvignesh92
Honored Contributor

Hi Thushar,

You have not mentioned the return type of the getPartitionColumns method. You have to return the partition columns as collection Ex list ['Team', 'Speciality']

Then the below method should work.

df.write.option("header", True) \

        .partitionBy(*partitionColumn) \

        .mode("overwrite") \

        .csv(targetPath)

Kindly try.