Job scheduled run fail alert

[alexgavrysh]

Hello,

I have a job that should run every six hours. I need to set up an alert for the case if this doesn't start (for example, someone paused it). How do I configure such an alert using Databricks native alerts?

Theoretically, this may be done using system tables and SQL Alerts. But the problem is that:

Currently no support for real-time monitoring. Data is updated throughout the day. If you don’t see a log for a recent event, check back later. link

[VZLA]

Thank you for your question! Here’s a concise workflow to set up an alert for missed job runs in Databricks:

1. Write a Query: Use system tables to identify jobs that haven’t started on time.
2. Save the Query: Save this query in Databricks SQL as a named query.
3. Create an Alert:
   1. Go to Alerts in Databricks SQL.
   2. Create a new alert based on the saved query.
   3. Set the condition to trigger when the query returns rows.
   4. Set Notifications: Add email, Slack, or PagerDuty notifications for the alert.
4. Schedule the Query: Set the query to run every 5-10 minutes to check for missed runs.

This will alert you when the job hasn’t run as expected. Let me know if you need clarification!