AmanSehgal
Honored Contributor III

But by using coalesce(1) in you pyspark df, you're doing the same thing. It'll be processed on one node.