ReadAllEventsForward behavior

EventStoreDB
#1

Hey hey,

I’ve been trying to recover ‘missed’ events when a denormalizer comes back online by using ReadAllEventsForward and passing the position that was last seen (using the latest dev branch).

I assumed that ReadAllEventsForward just returns me the list of events since ‘position’ up to ‘max’ (I’m passing int32.maxvalue for this) and then that’s that. When I call this method I get a sensible looking set of events back, however once I subscribe normally (SubscribeToAllStreamsAsync) I seem to receive a hell of a lot of duplicate events in the subscription handler. Just to see what the cause might be I’ve left the call to ReadAllEventsForward but do absolutely nothing with the returned events, so there should be no side effect. If I take this call out then everything works as expected (no duplicated events).

I’ve got an integration test that shoves 100k events through the EventStore and does a basic test to make sure they were all persisted in the read db, which is how I’m testing this code atm. So with the call to ReadAllEventsForward I get lots of duplicated messages, otherwise it works fine.

Am I doing something wrong here?

Cheers

Thomas

#2

Read events forward takes two longs and an int32(count). Prepare/commit position and count from that point. Are you setting them?

#3

See readallforward https://github.com/EventStore/EventStore/blob/dev/src/EventStore/EventStore.ClientAPI/Messages/ClientMessages.cs

#4

Ok I think I’m getting some odd behavior here with ReadAllEventsForward. The following happens with an empty event store:

  1. Read all events up to Position.Start. Start subscribing

  2. Write event ‘1’. Receive event on subscription with position 3423, 3317

  3. Kill subscribtion.

  4. Write event ‘2’.

  5. Start subscribing. ReadAlllEventsForward with position 3423,3317.

The last read all events forward returns both events (I was expecting just the last missed one). This is only reproducible the first time this sequence is followed though, every other time the connection is killed and recovered, it receives the correct missed events.

Cheers,

Thomas

#5

You will always get the first one twice. You are saying read from this event forward. You have no way of knowing what the next one would be. We have looked at various ways of changing this including read next events after position…

#6

Ok I see, that’s cool I can just ignore the first event.

Could you tell me if each RecordedEvent I receive over a subscription is supposed to have a unique position? If I batch up a bunch of events on the write -side the position doesn’t seem to change in the subscription handler for each event received.

Cheers

#7

Hi,

Yes, each unique event you receive over subscription has unique position. The position itself is a pair of commit position and prepare position. So if you batch multiple events into one write operation, all those events will have the same commit position, but still different prepare position. Please, verify that prepare position for each event is unique.

Hope this made that somewhat more clear.

#8

You are saying if you write 5 in a batch they all come back With the same sequence?ncan you copy/paste?

#9

Hey guys,

No you’re ok, I see the commit position is static in a batch but the prepare position IS changing per event. My bad.

Cheers