topic Re: Databricks Job issue (run was cancelled bydatabricks and spark UI is not available after 10mins) in Data Engineering

Databricks Job issue (run was cancelled bydatabricks and spark UI is not available after 10mins)

rpaschenko — Thu, 21 Sep 2023 11:21:16 GMT

Hi!
We had an issue on 09/19/2023 - we launched job, run was started, but after 10mins it was cancelled with no reasons. The spark ui is not available (which probably means that claster has not been started at all) and I don’t see any logs even.
Could you help please.

Re: Databricks Job issue (run was cancelled bydatabricks and spark UI is not available after 10mins)

-werners- — Fri, 22 Sep 2023 07:39:46 GMT

Was it a one time only error or a recurring one?
For the former, I'd check if your vCPU quota was not exceeded, or perhaps there was a temporary issue with the cloud provider,... Could be a lot of things (lots of moving parts under the hood).

For the latter: we will have to figure out where the problem is located. code, cluster config, job timing,...
Excluding causes as much as possible.