Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-09-2023 01:54 AM
Hi Thushar,
You have not mentioned the return type of the getPartitionColumns method. You have to return the partition columns as collection Ex list ['Team', 'Speciality']
Then the below method should work.
df.write.option("header", True) \
.partitionBy(*partitionColumn) \
.mode("overwrite") \
.csv(targetPath)
Kindly try.