Agus1
New Contributor III

Hello @Retired_mod, thank you for your answer.

I'm a bit confused here because you seem to be describing the opposite behavior of what I've seen in our checkpoint files.

Here I repost my examples to try to understand better.

First checkpoint file:

{"sourceVersion":1,"reservoirId":"4121e6a2-ab1a-4f6c-8217-6412909486c0","reservoirVersion":3716,"index":5285,"isStartingVersion":true}

All following checkpoint files:

{"sourceVersion":1,"reservoirId":"4121e6a2-ab1a-4f6c-8217-6412909486c0","reservoirVersion":3719,"index":-1,"isStartingVersion":false}

 

When you mention "when index is -1, it signifies the first checkpoint file of a stream. In this scenario, the reservoirVersion should be adjusted by adding 1 to it.":

  • as I describe in my original posting, when I see index = -1 I need to subtract 1 from the reservoirVersion not add 1, as the version present in the file doesn't even exist yet (it's 1 version above the last available version of the table). In the examples I show this is the second one, where reservoirVersion is 3719, where I need to subtract 1 as this version doesn't exist.
  • You say "it signifies the first checkpoint file of a stream", but from all the files I've seen in my implementations, never the index is -1 for the first file. Actually, in the first file is the only time I don't see index = -1.

From your comment regarding isStartingVersion: "This field is present only in the first checkpoint file of a stream (where index = -1)":

  • As you can see in my 1st example marked as "First checkpoint file", isStartingVersion is true and index is not -1. Also reservoirVersion points to the correct version of the table (no need to subtract 1). As I mentioned before, this is the behavior I've seen for all first checkpoint files. (First shared example)
  • And for all following files, isStartingVersion is always false, index is always -1, and the reservoirVersion always needs to be adjusted by subtracting 1. (Second shared example)

Can you help me understand if this behavior is normal or why I might be experiencing something different to what you mention?

Thank you very much for your help.