-werners-
Esteemed Contributor III

do you actually run the code on a distributed environment (meaning a driver and multiple workers)?

If not, there is no use in using pyspark as all code will be executed locally.