I have a job with multiple tasks running asynchronously and I don't think its leveraging all the nodes on the cluster based on runtime.

dave_hiltbrand
New Contributor II

I have a job with multiple tasks running asynchronously and I don't think its leveraging all the nodes on the cluster based on runtime. I open the Spark UI for the cluster and checkout the executors and don't see any tasks for my worker nodes. How can I monitor the cluster to ensure my tasks are running in parallel and taking advantage of my multiple node cluster?