What is the expected behaviour if messages are acked out of order?

EventStoreDB
#1

Hi,

Imagine we have competing consumers setup and the consumers end up acking messages out of order, what is the expected behaviour when clients reconnect after event store shutdown?

Observed behaviour:

Subscriber 1 receives Message 96 and is busy doing long work

Message 97,98,99 - Processed in Subscriber 2 and ack

  1. Shutdown eventstore - When eventstore comes up 97,98,99 is resent . 96 is lost.

  2. Shutdown subscriber 1 and subscriber 2. Followed by evenstore -when eventstore comes up 96 is lost but 97,98,99 is not resent.

We are trying to come up with a pattern to handle this situation and are unsure the best way to proceed and would like to know what the expected behaviour is from ES

Thanks

Sam

#2

You should get loads of retries. Checkpoints will not be written into
the future until all messages up to that point have been processed.

EG:

1,2,3,4,5

if something is still processing 1 and 2,3,4,5 have been processed a
checkpoint will not be written past 1 until 1 is processed. Of course
if your long running thing had timed out multiple times and parked the
message the checkpoint could be written (no longer in process). This
depends on configuration settings.

Checkpoint writes are logged, checkpoints are loaded on restart again logged.

#3

Thanks Greg.

That gives us something to work with I think.

#4

Greg, in our case (as described in the example above) we are losing message 96 altogether. Is there anyway we can make sure message 96 is also resent? I do not see the message in the parked queue as well.

#5

There are bunches of tests around exactly this behaviour. Send a
reproduction and we will look at it.

#6

Hey, I have not been able to replicate this issue. I will keep looking and send a test program if I can replicate again.