https://community.databricks.com/t5/data-engineering/jobs-running-forever-in-spark-ui/m-p/19032#M12707 <P>This happens if the Spark driver is missing events. The jobs/task are not running. The Spark UI is reporting incorrect stats. This can be treated as a harmless UI issue.&nbsp;If you continue to see the issue consistently, then it might be good to review what is causing the Spark driver to be memory intensive and lose the Spark events.&nbsp;</P> Fri, 25 Jun 2021 18:55:51 GMT brickster_2018 2021-06-25T18:55:51Z https://community.databricks.com/t5/data-engineering/jobs-running-forever-in-spark-ui/m-p/19031#M12706 <P>On the Spark UI, Jobs are running forever. But my notebook already completed the operations. Why the resources are wasted</P><P></P> Fri, 25 Jun 2021 18:55:32 GMT https://community.databricks.com/t5/data-engineering/jobs-running-forever-in-spark-ui/m-p/19031#M12706 brickster_2018 2021-06-25T18:55:32Z https://community.databricks.com/t5/data-engineering/jobs-running-forever-in-spark-ui/m-p/19032#M12707 <P>This happens if the Spark driver is missing events. The jobs/task are not running. The Spark UI is reporting incorrect stats. This can be treated as a harmless UI issue.&nbsp;If you continue to see the issue consistently, then it might be good to review what is causing the Spark driver to be memory intensive and lose the Spark events.&nbsp;</P> Fri, 25 Jun 2021 18:55:51 GMT https://community.databricks.com/t5/data-engineering/jobs-running-forever-in-spark-ui/m-p/19032#M12707 brickster_2018 2021-06-25T18:55:51Z