Hi,

I have spark job which is processing large data set, its taking too long to process the data. In Spark UI, I can see its running 1 tasks out of 9 tasks. Not sure how to run this in parellel. I have already mentioned auto scaling and providing upto 8 instances.

Attached image of spark UI.

Please suggest how to debug this and fix the performance issue.