cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

**Lakeflow Connect SQL Server โ€” Snapshots Firing Outside Configured Full Refresh Window?**

lrm_data
New Contributor III

Has anyone else seen full refresh snapshots trigger outside of their configured refresh window in Lakeflow Connect?

Here's our situation:

- We have a full refresh window configured to restrict snapshot operations to off-hours
- On at least one occasion, a snapshot started outside of that window โ€” during business hours
- This caused unintended load on our SQL Server source during peak usage

The docs are clear that snapshots should only start during the next available time in the configured window, so this feels like unintended behavior. I'm still investigating whether there's a specific trigger (e.g. auto full refresh policy, a pipeline retry) that might bypass the window, but so far I haven't been able to pin down a consistent cause.

Has anyone else observed this? And if so, have you found any conditions that reliably cause a snapshot to ignore the configured window?

Would love to know if this is a known bug or if there's something in our configuration we should be looking at.

1 ACCEPTED SOLUTION

Accepted Solutions

Sumit_7
Honored Contributor III

@lrm_data This is very unlike case for the refresh to be triggered outside the configured window. Though I would still suggest to check the Configured Window and Auto Full Refresh policy once to be sure.
If still persists, then you may raise a support ticket for further resolution.

View solution in original post

3 REPLIES 3

Sumit_7
Honored Contributor III

@lrm_data This is very unlike case for the refresh to be triggered outside the configured window. Though I would still suggest to check the Configured Window and Auto Full Refresh policy once to be sure.
If still persists, then you may raise a support ticket for further resolution.

lrm_data
New Contributor III

Thank you ! I will continue monitoring and log internally if it continues happening.

lrm_data
New Contributor III

Hello @Sumit_7 ,

I have tested a few scenarios and logged a ticket with Databricks and discovered the following:

Common Misconception
The start_window setting does not define a bounded time window during which full refreshes
are contained. It is simply a "no earlier than" anchor point.

What It Actually Means
Once the configured hour is reached, Lakeflow is permitted to begin an automatic full refresh
The refresh will run as long as it needs to โ€” there is no end time or cutoff
A snapshot triggered at 9 PM can and will continue running well past that hour
What Else Can Trigger a Full Refresh (Outside the Window)
Even before the anchor time is reached, a full refresh can still be triggered by:

  • A metadata change on a table
  • A table truncation
  • Manual triggers are also always allowed regardless of the window setting.