topic Jobs suddenly stopped running with no errors in Databricks Free Edition Help

Jobs suddenly stopped running with no errors

Aminsn — Sun, 27 Sep 2026 10:06:50 GMT

I have had a few jobs scheduled to run daily since last year. There were no problems for the first 10 months, but on 31 August 2026, the jobs suddenly stopped running, with no errors or messages indicating the cause.

I rescheduled the jobs, and they started working again. However, on 20 September, they stopped running once more, again with no warnings, messages, or errors.

I am wondering what is happening. This is very unreliable, and the fact that it occurs without any information whatsoever about the cause makes the situation unacceptable.

Can someone shed some light on this issue?

Re: Jobs suddenly stopped running with no errors

ThomazNeto — Sun, 27 Sep 2026 11:22:35 GMT

Hi Aminsn,

I understand the frustration, but "stopped with no errors" is a useful clue. When a job fails, you see red runs. When nothing appears at all, the scheduler didn't try, which usually means the trigger was paused or the run was skipped before it started. Neither raises an error. The docs don't describe any case where Databricks pauses a schedule on its own, so something touched the job.

Three things to check, all in the job's own history:

1. Who edited the job on Aug 31 and Sep 20. Every pause/resume is a job update in the audit log. If someone deployed with a bundle in development mode, that alone pauses schedules: the docs say it "Pauses all schedules and triggers on deployed resources such as jobs". Terraform or a CI job re-deploying with pause_status: PAUSED does the same.

sql

SELECT event_time, user_identity.email, action_name,
       request_params.job_id
FROM system.access.audit
WHERE service_name = 'jobs'
  AND action_name IN ('update', 'reset', 'delete')
  AND event_date BETWEEN '2026-08-29' AND '2026-09-22'
ORDER BY event_time;

The system.lakeflow.jobs table also keeps every version of the job, including a paused column, so you can see exactly when it flipped.

2. Whether the runs were skipped. Open the job's Runs tab and look for pink entries. Skipped is a real status, and the docs' example is "you exceeded the maximum concurrent runs for your job or your workspace". A run that hung since Aug 31 with max concurrent runs = 1 would silently block every daily run after it. This query lists runs that never started and why:

sql

SELECT period_start_time, result_state, termination_code
FROM system.lakeflow.job_run_timeline
WHERE job_id = '<job_id>'
  AND period_start_time = period_end_time
ORDER BY period_start_time DESC;

3. The Run as user. If that user was deactivated or left, the docs say jobs "fail unless they are assigned to a new owner". That would show as failures, not silence, but worth ruling out. Set Run as to a service principal either way.
https://docs.databricks.com/aws/en/jobs/privileges

Docs:
https://docs.databricks.com/aws/en/jobs/triggers
https://docs.databricks.com/aws/en/jobs/monitor
https://docs.databricks.com/aws/en/admin/system-tables/jobs
https://docs.databricks.com/aws/en/dev-tools/bundles/deployment-modes

If you post the result of the first query, we can probably name the cause.

Re: Jobs suddenly stopped running with no errors

anuj_lathi — Thu, 08 Oct 2026 18:54:48 GMT

Silent stops like this usually come down to one of two things: the trigger got paused, or runs are being skipped because an earlier run is still active. Your job-level history and the audit log can tell you which.

1. Check whether the trigger is paused

Open the job and look at Schedules & Triggers in the Job details pane. Each trigger has its own Active/Paused status. When a trigger is paused, currently active runs continue, but no new runs start [1]. If you see "Paused", that explains the silence. Re-saving or rescheduling the job sets the status again, which would match what you saw after you rescheduled.

2. Check for skipped runs from a stuck or long-running run

By default only one run of a job can be active at a time. Runs that exceed the configured max concurrency are skipped [1]. This is the classic cause of "it just stopped running with no error":

One run hangs or runs far longer than usual, for example waiting on a lock, a stuck cluster, or an upstream dependency.
Every later scheduled run is skipped because the earlier run is still active.
Nothing looks like a failure, because nothing actually failed.

Look at the job's Runs tab around 31 Aug and 20 Sep. Check for a run from the day before that was still active, and for runs with a Skipped status. If that's the pattern, you have two fixes:

Set a timeout on the job or tasks so a hung run gets terminated.
Add a duration-warning or failure notification so you hear about it.
If overlapping runs are acceptable for your workload, raise Maximum concurrent runs under Advanced settings [1].

3. Use the audit log to see who changed what

If the trigger was paused or the job edited, the audit log system table will show it, including the identity responsible [2]. Something like this works (adjust the job ID and dates):

SELECT event_time,
       action_name,
       user_identity.email,
       identity_metadata.run_by,
       request_params,
       response.statusCode
FROM system.access.audit
WHERE service_name = 'jobs'
  AND event_date BETWEEN '2026-08-25' AND '2026-09-25'
  AND request_params['job_id'] = '<your_job_id>'
ORDER BY event_time;

Filter on event_date rather than event_time for better query performance [2]. Look for update/reset-type events around the days the runs stopped. They'll show whether a person, a service principal, or an automation (Terraform, a bundle deploy, CI/CD) modified the schedule. Redeploys that overwrite the job settings are a common cause of a job being paused unexpectedly.

4. Other things worth ruling out

Bundle or Terraform redeploys: if the job is managed as code, a deploy can reset the trigger's pause status to whatever is in the config.
Multiple triggers: if you're in the multiple-triggers Beta, each trigger is paused and resumed independently [1]. Confirm the one you expect is the active one.
Workspace trigger limit: a workspace can have up to 12,000 active triggers, and paused ones don't count [1]. This is unlikely to affect you, but it's worth knowing.

Next step

Start with the Runs tab and the audit query for those two dates. In most cases like yours, the answer is either a pause event with an identity attached or a run that stayed active and caused skips. Once you know which, set timeouts and failure/duration alerts so a recurrence isn't silent. If neither check explains it, post the job ID's audit events (with sensitive values redacted) and the run statuses for those days, and we can dig further. If it still looks like a platform-side problem, open a support ticket with those timestamps.

References

[1] Automate jobs with schedules and triggers | Databricks on AWS — https://docs.databricks.com/aws/en/jobs/triggers
[2] Audit log system table reference | Databricks on AWS — https://docs.databricks.com/aws/en/admin/system-tables/audit-logs