Suggest the best way to keep a local position

EventStoreDB
1

Hello,

What is the best way to store a “bookmark/position” when following a stream over the HTTP API?

What I am planning to do is this:

  1. Read the batch of events (with long poll) using the URL stored in step 3. If not available - start with /streams/app-events/0/forward/5?embed=body.
  2. Process the events.
  3. Get the previous link and store it in the local DB. At this point, the process can quit and start again.
  4. Repeat from step 1.
    There are two annoyances with this approach:
  • all links returned do not include the embed=body query string of the initial request meaning the app needs to amend the stored URL to add the query string;
  • the URL stored include IP or host - so if we were to change it then the “bookmark” would essentially become invalid.

The options I can see are:

  1. Store the link[rel=‘previous’] (as described above); append embed=body when requesting.
  2. Strip the hostname from link[rel=‘previous’] and store it; use hostname from the app’s configuration and append embed=body when requesting.
  3. Extract the position value from link[rel=‘previous’] and store as a number; construct full URL when requesting.
  4. Use the max(positionEventNumber) of all events, store it and construct the full URL when requesting.
    All have pros and cons. I believe the documentation recommends option one.

I want to avoid storing the hostname as a reference out of all, leaning towards option 2.

But what are people’s thoughts?

Cheers.

2

You made it clear that you don’t want to keep the hostname in your bookmark, but are ambiguous over your intention of retaining the stream id vs. (presumably) retrieving it from the app’ configuration…

In option 2 you say to strip the hostname, suggesting you will keep the rest (including the stream id).

But in option 3 you are going to save the position value as a number.

When I have to manage my own resumable subscription over HTTP API then I’ll merely store the last eventNumber seen, and resume from there. The rest of my HTTP URI will be constructed from configured values. This is closest to your option 3, other than where I take the value from.

However, where possible I will take advantage of a Persistent Subscription over the TCP API. Let ES do the lifting. Is that not an option for you?

3

Thanks,

We’ve decided to go with stories just the last number.

Persistent subscriptions aren’t going to work well for us until long polling will be supported on them:

https://github.com/EventStore/EventStore/issues/1811