I have a job with multiple tasks running asynchronously and I don't think its leveraging all the nodes on the cluster based on runtime.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-22-2023 07:47 PM
I have a job with multiple tasks running asynchronously and I don't think its leveraging all the nodes on the cluster based on runtime. I open the Spark UI for the cluster and checkout the executors and don't see any tasks for my worker nodes. How can I monitor the cluster to ensure my tasks are running in parallel and taking advantage of my multiple node cluster?
Labels:
- Labels:
-
Job
-
Multiple Tasks
-
Spark ui