Jobs running forever in Spark UI

[brickster_2018]

On the Spark UI, Jobs are running forever. But my notebook already completed the operations. Why the resources are wasted

[brickster_2018 · Accepted solution]

This happens if the Spark driver is missing events. The jobs/task are not running. The Spark UI is reporting incorrect stats. This can be treated as a harmless UI issue. If you continue to see the issue consistently, then it might be good to review what is causing the Spark driver to be memory intensive and lose the Spark events.