Indempotency issue

EventStoreDB
1

Seeing an issues using CatchupSubscription where duplicate events are being written to the target (a clean Event Store)

When the CatchupSubscription indicates it has caught up, we re-run the CatchupSubscription using the same ES connection object and all the events are written again to our new Event Store (no idempotent writes)

If we conduct the same test, but create a new connection, we get idempotent writes.

Can anyone shed any light on why this would happen?

Expected version is always .Any and we always catchup the same stream.

EventStore version 4.0.1 and using core.net client api.

2

Hi Steven,

I’m not able to reproduce what you are seeing on my side.

The first thing I’d like to ask is if your events are always being written using the same Event Id.

The idempotency check uses the Event Id to decide whether an event is new or not, so if this is being changed when your subscription runs the second time against the same connection, this would be the cause of what you are seeing.

If this is not the case, could you please provide us a sample of the client code you are using?

3

The same Event Id is being written:

Here is an example:

Here is the event that was initially written:

Number: 0

Data
{
“$type”: “Vme.Eposity.Organisations.DomainEvents.OrganisationCreatedEvent, Vme.Eposity.Organisations.DomainEvents”,
“OrganisationName”: “Tamworth”,
“AggregateId”: “ea836293-21b5-4986-a82d-ad285d5265c4”,
“EventId”: “b930f82d-e800-4668-bc62-e4c1e5450aa4”,
“EventTimestamp”: “0001-01-01T00:00:00”,
“MetaData”: null
}

``

And here is our duplicate being written later on in the stream:

Number: 29224

Data
{
“$type”: “Vme.Eposity.Organisations.DomainEvents.OrganisationCreatedEvent, Vme.Eposity.Organisations.DomainEvents”,
“OrganisationName”: “Tamworth”,
“AggregateId”: “ea836293-21b5-4986-a82d-ad285d5265c4”,
“EventId”: “b930f82d-e800-4668-bc62-e4c1e5450aa4”,
“EventTimestamp”: “0001-01-01T00:00:00”,
“MetaData”: null
}

``

I don’t know if there is a way to see the Event Id used by Event Store (which we get on ResolvedEvent) but we have a factory that takes the remote Event, and builds a new EventData object before we write, and the Id’s are the same.

4

using ExpectedVersion.Any does not assure idempotency it is a best
effort. By default I believe the size is around 1m events it will
check. Setting an ExpectedVersion gives assured idempotency.

5

The stream we are working with is around 30,000 events.
How could we get ExpectedVersion when we are using a CatchupSubscription to essentially seed a new EventStore?

We start the CatchupSubscription, and essentially just want a clone of that stream in our new Event Store.

Would we have to Write to the stream and keep track of the version manually (even if we had to restart the CatchupSubscription?)

Something like this;

public void EventAppeared()

{

await connection.AppendToStreamAsync(streamName, expectedVersion, eventData, this.DefaultUserCredentials);

expectedVersion++;

//TODO: probably persist this value in case the process dies

}

6

I would imaging that you would be persisting your location of the
catchupsubscription either way. Your event that you are copying
already has a stream/version associated with it just use the same.