handling events from subscriptions

EventStoreDB
#1

I am wondering how you guys are dealing with event subscriptions if you have at least one of the following scenarios

  • large number of events generated in a very short amount of time (bursts of events)
  • events are consumed by many consumers/projections which takes some time to complete
  • one of the consumers/projections throws an exception
    is it wise to accept the event and queue its handling using the thread pool?

Or is the internal buffer of the event store big enough to handle this situation without the client code needing to process the events asynchronously?

In the case of an exception do you just log it and continue with all other consumers of the same event?

Any tips and no no’s are highly welcome

#2

Well there are quite a few questions here :slight_smile:

bursts should be ok. They will queue. the real question is dealing with push back if you are always too slow.

How many events/second are we discussing here?

If you get an exception this depends on your projection. Some can log/recover others must stop.

Greg

#3

the bursts are the result of one command coming from a client and the events come as fast as the system can generate/process them. It is normally a couple of hundred up to a couple of thousand events.
(scenario: fulfilling/completing a receipt with several hundred items on it)

How many events does the event store queue? Is this number configurable? Is it even recommended to configure?

#4

for a few hundred to a few thousand it shouldnt be an issue. If you are talking about 100,000 etc then I would start thinking about it.

Take a look at EventStoreCatchUpSubscription in the client API. It can handle these things (providing you arent in general writing too fast for you to ever catch up :))

#5

thanks for the super fast answer!
I will have a look into the code…