cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Time window for "All tables are updated" option in job Table Update Trigger

SamAdams
Contributor

I've been using the Table Update Trigger for some SQL alert workflows. I have a job that uses 3 tables with an "All tables updated" trigger:
Table 1 was updated at 07:20 UTC
Table 2 was updated at 16:48 UTC
Table 3 was updated at 16:50 UTC
-> Job is triggered. Updates happened over a ~10 hour time window.
Is there any sense of how long the window is for all tables in the trigger to have an update?

1 REPLY 1

mark_ott
Databricks Employee
Databricks Employee

There is no fixed or documented “window” time for the interval between updates to all monitored tables before a job with an "All tables updated" trigger runs in Databricks. The job is triggered as soon as every table in the set has seen at least one update since the last successful run—regardless of how much time has elapsed between the individual table updates.​

Timing Behavior

  • If Table 1 is updated at 07:20 UTC, Table 2 at 16:48 UTC, and Table 3 at 16:50 UTC, the trigger waits for the last table in that set to be updated; the job will run after Table 3's update, even though there is a ~10 hour gap between Table 1 and Table 3.​

  • There is no apparent reset or expiration window—Databricks tracks updates on all monitored tables since the last job run, and as soon as every table has received an update, the trigger fires.​

Advanced Timing Options

  • You can optionally set “Minimum time between triggers” (enforces a cooldown between runs irrespective of update activity) and “Wait after last change” (holds off execution for a set period after the most recent update in case additional batch updates occur).​

  • If both are set, Databricks waits for both timers to expire before firing the trigger.​

Practical Implication

  • If one table updates hours before others, the job will only trigger once all tables have been updated, even if this takes many hours.​