Hi there…
Given an event (e.g. LocationUpdated) that holds the ID (Guid) of the updated item. Is there a chance to enrich the event with a sequence number that increments in relation to the ID? So to say: I have two update events for ID 1 and one update event for ID 2. The events for ID 1 have a sequence number 1 and 2 - and the event for ID 2 has a sequence number 1 (for being the first update event to that id).
Reason for that is: I will have multiple instances of a worker service subscribing to that event stream. I want to ensure that - if worker 1 received event 1 for id 1 and worker 2 received event 2 for id 1 that worker 2 waits with processing just in case worker 1 is slow for some reason.
You can use stream revision, available in the ResolvedEvent. It’s an autoincremented, gapless number in the stream scope.
You can read more on that in my blog:
1 Like
can you tell more for the reason behind this setup ?
it seems you need ordered & blocked processing of events on stream basis…
and this is kind of hard to achieve if you have 2 independent workers listening
you need to have at least a shared thing that tells the scd one to wait on the first…
why not 1 worker just doing the work in order?
with another one that’s started if the first one dies.
I am evaluating this setup, as I two events of course have just been the example. In fact I expect thousands and thousands of events for thousands of distinct entities.
To be more concrete about the setup I am evaluating: I (will) have an endpoint (GraphQl, but that does not make any difference at all), which simply put events to the eventstore based on the submitted input data. On the other side there is a hosted service which subscribes to those eventstore streams and processes the events for read model population. As I need to do some sort of validation (e.g. a key must be unique within a tenant), I’d do a query against my read model store when validating the inputs at my endpoint - so: before the events are created. I know that there is still the risk of validations to go green when two requests with e.g. the same key are processed at the same time. I am not thinking of this right now. Anyway - the risk becomes quite higher, the longer the event processing for read model population takes.
Another option would be to populate the read model store within the request after successful validation and submission of the event to the event store. I just want to evaluate to split these two processes in order to get my “write” api endpoint faster.
it should not be necessary to check uniqueness when building your read models.
but well before appending envents to the store .
when talking tenant based you can use stream naming convention [TenantID]-[Category]-[ID]
if you want to check uniqueness it’s just a matter of checking if the [tenant]-[category]-[id] exists.
what you might want to do in your read models , is loosen relations & existence …
i.e be able to cope with out of order events.
( when using a relationall based read model , i’m usually doing upsert )
Thanks for your answer. OK - let’s assume that not only a key property needs to be unique but also - let’s say - a name property. So there are two properties that must be unique within a tenant. Do I understand you correct, that basically in this case I could also create one (or two?) projection(s) which hold the distinct key and name values? So I am not doing validation against my read model store but instead against eventstore projections?
@mario.adam, could you elaborate on why do you plan to do validation against read model?
In the canonical event sourcing approach, you can do it on the write side based on the events in the stream. You’re reading events. Apply them one by one and build your state. You’re making a business decision based on that and appending a new event to the stream. If you’re using optimistic concurrency, you can be sure that there was no conflicting update, and you’re making decisions based on the current state.
Having all of that, you can trust your events and apply them one by one to your read models (EventStoreDB guarantees the order of processing).
There are a few ways to achieve this … a persisted read model (in some store, or through an eventstoredb projection ), or by having a stream containing the constraints, or multiple streams holding the constraints or a reservation pattern , or a UUID V5
which once to choose depends on exactly what you are trying to achieve on business level…
and TBH , I’ve found that most of the time those constraints on multiple values make absolutely no sense from a business POV immediatly: i.e yes at _some point in time _ they need to be enforced , but not right away…
( and seen to many time systems enforcing those constraints… and when you ask support they have administrative ways to overule them )
let’s assume that not only a key property needs to be unique but also - let’s say - a name property.
the key uniqueness : I guess the system is assiging it ? use a UUID
the (name ,key) uniqueness : given the above, the key is already unique , you only need to check the name