Consistency Guarantees

EventStoreDB
1

Hi,

We are starting to have 3rd parties use our REST API.

We are using a similar setup to what I’ve seen others discuss here where

the read store database(s) subscribe to the all events stream and follow

along usually less than a second behind.

For many operations in our user interface that sort of catch up time is

fine. We rarely write and then read immediately since the client knows

what they just wrote.

What we would like to do for third parties is give them read-after-write

consistency guarantees for the subset of our API that is simplest.

Operations such as:

  • Approve x

  • Change the name of x

Would then be guaranteed to be reflected by the same client retrieving x by id after the write request has completed. I am happy if this slows down the experience for those API users so long as it doesn’t put a large amount of extra strain on our system.

One simple strategy I was thinking of was:

  • On write return to the client in a header something that represents

the CommitPosition/PreparePosition of the event that just got

written - probably with a 202 response code.

  • The client then sends that as a header with their subsequent requests.

  • In the case of multiple writes the client would always send the

greatest Position it has seen.

  • If a read store query comes with a Position that is greater than where

it is caught up to then it could simply return 503 with a Retry-After

header that gives some reasonable estimate of read store catch up

latency.

  • Any queries without a Position normally just returning where the

read store database is caught up to.

This feels like it allows us to have multiple read store databases that

are completely independent without the client seeing inconsistent

results. Writes can still happen quickly without waiting for the read

store to catch up. It also seems like in most cases the subsequent reads would

succeed first time and performing a comparison between the Position in

the header and the Position where that that read store is caught up to

should impose almost no extra load.

Can anyone see potential problems with this setup? One thing that we

would need to ensure is that all operations for which we provide this

guarantee are fast as if one event caused the whole read store

consistency position to lag behind then we could get a large number of

clients waiting and retrying. What I like most about this model is the simplicity of

implementing it from the client side.

Looking at the event store Client API the Position is sent as

part of the subscriber API. It does not appear that there is way to get

the Position on writing an event. Is it possible this could be added in

the future?

Looking for ideas at this point so happy to hear from anyone who has

achieved something like this in a better way.

cheers

Andrew

2

When you write you get told the position of the event in the stream. A follow up read could get this as of now.

This is also some this that could be added on the confirmation of a write we will discuss any trade offs today but I don’t see any off the top of my head when running a single replica set.

Greg

3

In looking through this there are some edge cases associated with it if done generally however simply returning the last log position would seem to be ok (eg if you write 8 events the last is returned)

4

Thanks Greg,

Returning the last position on writing a group of events would be great.

For the moment I’ll experiment with reading the position of my last write.

I presume when you say single replica set that basically means we are fine so long as each client is only dealing with events from a single cluster?

Also does the “Original” part of OriginalPosition just mean that resolvedEvents will have the position where they were originally written?

cheers

Andrew

5

So you want to use the original in that case not the resolved. Internally there is a concept known as linkTo this allows a pointer to be written to another stream back to an event in another stream (can be used to build indexes). In this case your original event (the pointer) would be different than the resolved event (the result of following the pointer). They will also have two sets of log positions (the resolved always being prior to the original).

Also when comparing the positions you always have to compare

  1. commit position > other commit position?

  2. if commit position == other commit position then use prepare position

This logic is already handled in https://github.com/EventStore/EventStore/blob/dev/src/EventStore/EventStore.ClientAPI/Position.cs for you but if you start using for your read model you would either need to use position yourself or implement something similar.

Cheers,

Greg

6

Andrew,

Can you take a look through this branch to tell me if its what you had in mind?

https://github.com/EventStore/EventStore/tree/logposition

If you look at writeresult/deleteresult you will see there is a LogPosition on them that represents where in the log (like the LogPosition on event read) it was written

This should make what you are discussing possible.

Cheers,

Greg

7

Thanks Greg,

That’s exactly what I had in mind thanks for looking into it so quickly :slight_smile:

Also thanks for the pointer about how to compare commit position and prepare position - I will need to replicate this logic in our client libraries. On the server side we’ll try and stick to using your Position struct.

cheers

Andrew

8

As of now the prepare/commit positions will always be the same but it is possible in the future they could be different.