Issue with SubscribeToAllFrom, same event sent to multiple subscribers

Clients
#1

Want advise on Catch-up subscription - Currently we are using catch-up subscription to subscribe to all events, the event store client API has a function called “SubscribeToAllFrom”, that will trigger the subscriber function (EventStoreOnEventCatchup in this example), and process the event.

client.Connection.SubscribeToAllFrom(
_indexPosition,
settings,
(sub, evt) =>
ReceiveHandler(sub, evt),
null,
(sub, reason, ex) =>
SubscriptionDroppedHandler(sub, reason, ex),
credentials)

Issues
I have multiple instances calling above function, that means those multiple instance receive same event independently, causing us to process same event multiple times.

Question

  1. Is there any way to configure Catch-up subscription in such way, each event will only go to one subscriber only, instead of all of them?
  2. If the answer to question 1 is NO, can the persistent subscription solve this problem? The document says that persistent subscription uses competitive consumer pattern. Does it mean once an event is picked by one subscriber, it will not be sent to other subscribers?
#2

1 / no: catchup subscription are independent of each other
2/ yes , you can partially achieve that with competing consumers
though as it’s a competing comsumers, you can get duplicate message deliveries and out of order deliveries ( more or less depending on the the strategy used )
This is due to the nature of messaging and has nothing to do with the database itself.
more info here : https://developers.eventstore.com/server/v21.6/persistent-subscriptions/#considerations

What is the problem you are trying to solve and what qualities do you want on the solution ?

#3

Thanks for your advise, the problem I try to solve is that I need such subscription- if connected with multiple subscribers, ONLY send an event to ONE subscriber, NOT ALL OF THEM, since I don’t want multiple instances to process the same even. I want the solution to be reliable.
By reading the document you refer to, looks like there is DispatchToSingle option that can solve my problem. But I am not that familiar with persistent subscription, I wonder if there is any way to route ALL EVENTS from ALL stream to my persistent subscription. Any document for that?
Currently I use SubscribeToAllFrom, which automatically route all the streams and I don’t need to set it up.

#4

, ONLY send an event to ONE subscriber, NOT ALL OF THEM, since I don’t want multiple instances to process the same even. I want the solution to be reliable.

so you want to be able to

  • scale out & the ordering of processing does not matter
  • or have failover

The server can know handle persistent subscription to $all (all events in the cluster ) but it’s not exposed yet in the clients ( should be out soon in the .net grpc client , )

another way to have failover processing is to use a regular client side initiated subscriptions and use a
distributed lock to only perfom the work on 1 process at a time
so multiple instance are up & in order to actually do the work ( begin to subscribe & do the work) they need to get a hold on the lock .

#5

Thanks for your suggestion, I was able to do some proof of concept - I created a projection to agree to a list of events I am interested at, then I created a persistent subscription, seems able to get one event at a time. I will contine to follow up this path.

Also, I have a big concern, hopefully get your explaination - I belive you are very familiar with SubscribeToAllFrom, which is used to capture live events. But I ran into scenarios where some of the event under some streams, could not be received by the eventappeared callback funciton, I tried create events manually under those streams, still could not receive the events. BUT this issue was not happening to all the events, I was able to receive some others. I am puzzled by this inconsistent behavior. ONLY thought I have is that there may be a position associated with the events created by our event source generator, unless those instance was crazy and inserted the stream/events at the wrong position? I bookmark the stream I failed to receive event, not sure if I can investigate deeper and find out why?
Unfortunately we are using very old server and client, still 4.10. We may migrate soon.But the question - is it possible to insert an event which can not be captured by subscriber in live mode, assuming that event is not filtered by our subscriber?