Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-19-2023 12:14 PM
Hey @therealchainman
The last backfill (lastBackfillFinishTimeMs) will be recorded as part of the checkpoint -> offset files, this helps the autoloader to know when the last backfill is triggered and to trigger the next periodic backfill.
Hope this answers your question.
Thanks,
Saikrishna Pujari
Sr. Spark Technical Solutions Engineer, Databricks
Saikrishna Pujari
Sr. Spark Technical Solutions Engineer, Databricks