Python running far slower than locally, even with large cluster and multiple workers

[JoeAckerman]

I have a notebook that is running extremely slowly even when I try to do pretty basic python functions. It is running far slower than locally no matter what I try, this is in spite of using a 32gb 4 core cluster with 4-8 workers. For context, my data is around 7m rows with about 20 columns, so not huge! Any help would be much appreciated.

[JoeAckerman]

Just noticed it says Python, in title - it should say Spark.

[cgrant]

Please share more information, for example:

• Type of data source
• Type of operations being executed (sharing code if possible)
• Timings of local runs and Databricks runs