AvailableNow Trigger and failure
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-04-2024 09:40 AM
Hi,
I wonder what is the supposed to be the behavior of spark structured streaming when using the AvailableNow Trigger and there is a query failure during the query ? More specifically, what happens to the initial end offset set ? Does it change ? While it is clear that using checkpointing the query would resume where it left off, but what happens to the end offset ? To some degree it almost amount to ask if spark structured streaming make difference between a failure and the end of the query ?