We are starting to have 3rd parties use our REST API.
We are using a similar setup to what I’ve seen others discuss here where
the read store database(s) subscribe to the all events stream and follow
along usually less than a second behind.
For many operations in our user interface that sort of catch up time is
fine. We rarely write and then read immediately since the client knows
what they just wrote.
What we would like to do for third parties is give them read-after-write
consistency guarantees for the subset of our API that is simplest.
Operations such as:
Change the name of x
Would then be guaranteed to be reflected by the same client retrieving x by id after the write request has completed. I am happy if this slows down the experience for those API users so long as it doesn’t put a large amount of extra strain on our system.
One simple strategy I was thinking of was:
- On write return to the client in a header something that represents
the CommitPosition/PreparePosition of the event that just got
written - probably with a 202 response code.
The client then sends that as a header with their subsequent requests.
In the case of multiple writes the client would always send the
greatest Position it has seen.
- If a read store query comes with a Position that is greater than where
it is caught up to then it could simply return 503 with a Retry-After
header that gives some reasonable estimate of read store catch up
- Any queries without a Position normally just returning where the
read store database is caught up to.
This feels like it allows us to have multiple read store databases that
are completely independent without the client seeing inconsistent
results. Writes can still happen quickly without waiting for the read
store to catch up. It also seems like in most cases the subsequent reads would
succeed first time and performing a comparison between the Position in
the header and the Position where that that read store is caught up to
should impose almost no extra load.
Can anyone see potential problems with this setup? One thing that we
would need to ensure is that all operations for which we provide this
guarantee are fast as if one event caused the whole read store
consistency position to lag behind then we could get a large number of
clients waiting and retrying. What I like most about this model is the simplicity of
implementing it from the client side.
Looking at the event store Client API the Position is sent as
part of the subscriber API. It does not appear that there is way to get
the Position on writing an event. Is it possible this could be added in
Looking for ideas at this point so happy to hear from anyone who has
achieved something like this in a better way.