Event Store Discuss Forum

Getting the same event several times in projected stream - cant understand why

Julian_May October 6, 2015, 9:17am 1

Hey

I have a problem with the same events being recieved several times, because the same event appears several times in the projected stream.

How do I prevent this?

The most problematic/reoccurring event is “RoomstayPlanned”, which does occur in a couple of other projected streams… Is the same event being picked up several times because of “fromAll()”?

Anyway - the projection looks like this:

fromAll().when({

ReservationPlaced: function(s, e) {

linkTo(“Read_RoomAssignments_Assignments”, e);

},

RoomstayPlanned: function(s, e) {

linkTo(“Read_RoomAssignments_Assignments”, e);

},

RoomstayWasAssignedRoom: function(s, e) {

linkTo(“Read_RoomAssignments_Assignments”, e);

},

RoomstayWasUnassignedRoom: function(s, e) {

linkTo(“Read_RoomAssignments_Assignments”, e);

},

RoomWasAdded: function(s, e) {

linkTo(“Read_RoomAssignments_Assignments”, e);

},

RoomTypeWasAdded: function(s, e) {

linkTo(“Read_RoomAssignments_Assignments”, e);

},

PersonCreated: function(s, e) {

linkTo(“Read_RoomAssignments_Assignments”, e);

},

CompanyCreated: function(s, e) {

linkTo(“Read_RoomAssignments_Assignments”, e);

},

TransientGuestWasAddedToReservation: function(s, e) {

linkTo(“Read_RoomAssignments_Assignments”, e);

},

KnownGuestWasRelatedToReservation: function(s, e) {

linkTo(“Read_RoomAssignments_Assignments”, e);

},

ReservationBecameGroupReservation: function(s, e) {

linkTo(“Read_RoomAssignments_Assignments”, e);

}

});

And i caught the phenomenon in Postman; here you can see the same event being referenced, but with different “updated” timestamps:

http://localhost:2113/streams/Read_RoomAssignments_Assignments/340/forward/10

<?xml version="1.0" encoding="utf-8"?> Event stream 'Read_RoomAssignments_Assignments'

http://localhost:2113/streams/Read_RoomAssignments_Assignments

2015-10-06T09:02:22.1623852Z

EventStore

3@ReservationAR-dd4023a8b50e4a679ddcfb8676cc7942

http://localhost:2113/streams/ReservationAR-dd4023a8b50e4a679ddcfb8676cc7942/3

2015-10-06T09:02:22.2393873Z

EventStore

$>

3@ReservationAR-dd4023a8b50e4a679ddcfb8676cc7942

http://localhost:2113/streams/ReservationAR-dd4023a8b50e4a679ddcfb8676cc7942/3

2015-10-06T09:02:22.2373874Z

EventStore

$>

4@ReservationAR-a3e7aa1e0bd9431aa49e508aa6b1fa5b

http://localhost:2113/streams/ReservationAR-a3e7aa1e0bd9431aa49e508aa6b1fa5b/4

2015-10-06T09:02:22.2313869Z

EventStore

$>

4@ReservationAR-a3e7aa1e0bd9431aa49e508aa6b1fa5b

http://localhost:2113/streams/ReservationAR-a3e7aa1e0bd9431aa49e508aa6b1fa5b/4

2015-10-06T09:02:22.2303871Z

EventStore

$>

4@ReservationAR-a3e7aa1e0bd9431aa49e508aa6b1fa5b

http://localhost:2113/streams/ReservationAR-a3e7aa1e0bd9431aa49e508aa6b1fa5b/4

2015-10-06T09:02:22.229387Z

EventStore

$>

4@ReservationAR-3844357a0659474297a1e18dee0cb8a8

http://localhost:2113/streams/ReservationAR-3844357a0659474297a1e18dee0cb8a8/4

2015-10-06T09:02:22.2223871Z

EventStore

$>

4@ReservationAR-3844357a0659474297a1e18dee0cb8a8

http://localhost:2113/streams/ReservationAR-3844357a0659474297a1e18dee0cb8a8/4

2015-10-06T09:02:22.2213873Z

EventStore

$>

4@ReservationAR-3844357a0659474297a1e18dee0cb8a8

http://localhost:2113/streams/ReservationAR-3844357a0659474297a1e18dee0cb8a8/4

2015-10-06T09:02:22.2183869Z

EventStore

$>

4@ReservationAR-44d8c45cd04e44cab9dfec88e05d11bb

http://localhost:2113/streams/ReservationAR-44d8c45cd04e44cab9dfec88e05d11bb/4

2015-10-06T09:02:22.2033866Z

EventStore

$>

2@ReservationAR-0de2c58ba7394c97a2d31d518c80b92c

http://localhost:2113/streams/ReservationAR-0de2c58ba7394c97a2d31d518c80b92c/2

2015-10-06T09:02:22.1703856Z

EventStore

$>

Julian_May October 7, 2015, 3:11pm 2

No takers? This causes a real problem problem in our write model, and redundant updates (that are less problematic) in our readmodel. Any ideas or hints would be very much appreciated!
What could cause this???

Julian_May October 8, 2015, 12:47pm 3

Is there any way to skip link’s? It seems to me that all events, once they reach a projection, is resolved is it was a link. I’ve been snooping in metadata, streamId etc. for a way to ignore “link”-events, and only linkTo(’’) for events that is not already a link - because I think that’s the reason for the “dobbelganger” events in my projected streams.

jen20 October 8, 2015, 12:55pm 4

I have a problem with the same events being recieved several times, because
the same event appears several times in the projected stream. How do I prevent

this?

Your example isn’t succinct enough to quickly understand (likely why there are no replies), but if the same event appears multiple times in the stream from which you’re running your projection it would be unusual if it didn’t get received by the projection multiple times? Moreover, the guarantees are only “at-least-once”.

Why is your even appearing multiple times?

Julian_May October 8, 2015, 1:58pm 5

Ok. Maybe I’m having problems with my explenation. I do have something very concrete though:
When I spin up my test-system, emitting events and creating subscriptions, the projection ends in the following state:

{

“coreProcessingTime”: 50,

“version”: 0,

“epoch”: -1,

“effectiveName”: “Read_RoomAssignments_Assignments_Projection”,

“writesInProgress”: 0,

“readsInProgress”: 0,

“partitionsCached”: 1,

“status”: “Running”,

“stateReason”: “”,

“name”: “Read_RoomAssignments_Assignments_Projection”,

“mode”: “Continuous”,

“position”: "C:2430835/P:2430835; ReservationPlaced: -1; RoomstayPlanned: -1; RoomstayWasAssignedRoom: -1; RoomstayWasUnassignedRoom: -1; RoomWasAdded: -1; RoomTypeWasAdded: -1; PersonCreated: -1; CompanyCreated: -1; TransientGuestWasAddedToReservation: -1; KnownGuestWasRelatedToReservation: -1; ReservationBecameGroupReservation: -1; ",

“progress”: 100,

“lastCheckpoint”: "C:0/P:-1; ReservationPlaced: -1; RoomstayPlanned: -1; RoomstayWasAssignedRoom: -1; RoomstayWasUnassignedRoom: -1; RoomWasAdded: -1; RoomTypeWasAdded: -1; PersonCreated: -1; CompanyCreated: -1; TransientGuestWasAddedToReservation: -1; KnownGuestWasRelatedToReservation: -1; ReservationBecameGroupReservation: -1; ",

“eventsProcessedAfterRestart”: 253,

“statusUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection”,

“stateUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection/state”,

“resultUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection/result”,

“queryUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection/query%3Fconfig=yes”,

“enableCommandUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection/command/enable”,

“disableCommandUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection/command/disable”,

“checkpointStatus”: “”,

“bufferedEvents”: 0,

“writePendingEventsBeforeCheckpoint”: 0,

“writePendingEventsAfterCheckpoint”: 0

}

This is wrong (there is for example 18 ReservationPlaced events in the output stream in the output here, 1 one of these being a “dobbelganger”)

If we reset the projection, it fixes the issue, and we end at this state:

{

“coreProcessingTime”: 18,

“version”: 1,

“epoch”: 1,

“effectiveName”: “Read_RoomAssignments_Assignments_Projection”,

“writesInProgress”: 0,

“readsInProgress”: 0,

“partitionsCached”: 1,

“status”: “Running”,

“stateReason”: “”,

“name”: “Read_RoomAssignments_Assignments_Projection”,

“mode”: “Continuous”,

“position”: "C:2426206/P:2426206; ReservationPlaced: 16; RoomstayPlanned: 31; RoomstayWasAssignedRoom: -1; RoomstayWasUnassignedRoom: -1; RoomWasAdded: 24; RoomTypeWasAdded: 3; PersonCreated: 49; CompanyCreated: 5; TransientGuestWasAddedToReservation: 47; KnownGuestWasRelatedToReservation: 12; ReservationBecameGroupReservation: 2; ",

“progress”: 100,

“lastCheckpoint”: "C:0/P:-1; ReservationPlaced: -1; RoomstayPlanned: -1; RoomstayWasAssignedRoom: -1; RoomstayWasUnassignedRoom: -1; RoomWasAdded: -1; RoomTypeWasAdded: -1; PersonCreated: -1; CompanyCreated: -1; TransientGuestWasAddedToReservation: -1; KnownGuestWasRelatedToReservation: -1; ReservationBecameGroupReservation: -1; ",

“eventsProcessedAfterRestart”: 199,

“statusUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection”,

“stateUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection/state”,

“resultUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection/result”,

“queryUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection/query%3Fconfig=yes”,

“enableCommandUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection/command/enable”,

“disableCommandUrl”: “http://localhost:2113/projection/Read_RoomAssignments_Assignments_Projection/command/disable”,

“checkpointStatus”: “”,

“bufferedEvents”: 0,

“writePendingEventsBeforeCheckpoint”: 0,

“writePendingEventsAfterCheckpoint”: 0

}

… And no dobbelganger-events!

I am at a loss as to why. Why does a “reset” fix it?

Julian_May October 8, 2015, 2:34pm 6

Some more facts:

The problem is very consistent (the state of the projection ), however the amount of “dobbelgangers”, and thus the "eventsProcessedAfterRestart in the stream is not, and “eventsProcessedAfterRestart” varies from 253 to 337 (out of 199 correct) in the handful runs we’ve done today.
The problem recurs in similar projections (that simply links events of specific types to another stream), but does not occur in our slightly more complex projections, where we use properties of the events to decide whether or not to link events to another stream.
The problem is reproducable on other machines (Windows 7, Windows 8 and Windows 10 - same story).
Everytime, resetting the projection solves it.

Julian_May October 8, 2015, 3:47pm 7

I just realized “succinct” means “brief and to the point” - I can see why this thread quicly suffers TL;DR…

Greg_Young1 October 8, 2015, 3:52pm 8

I don't understand what you think is happening, what you expect to
happen, or what the issue is here. I just see projection states
copy/pasted with no context and discussion about events not being
there. Without knowing what the data actually is (and output streams
etc) etc I don't think many will be able to help you. This is just
checkpoints etc.

Julian_May October 9, 2015, 9:52am 9

Ok, sorry for my chaotic stream of information.

To boil it down:

I expect projections to handle each event within scope (in this case, specified EventTypes fromAll() ) exactly once.

When I create and enable the projection Read_RoomAssignments_Assignments_Projection (see Projection.txt), the stream “Read_RoomAssignments_Assignments” contains the entries as specified in StreamBeforeReset.txt. This contains the same events several times. When i reset the projection, the stream ends up like StreamAfterReset.txt, where each event is represented exactly once, as I would expect.

My question is: How can i achive a result like “StreamAfterReset” without resetting my Projection?

StreamBeforeReset.txt (248 KB)

StreamAfterReset.txt (195 KB)

Projection.txt (1.19 KB)

Julian_May October 9, 2015, 9:52am 10

Ok, sorry for my chaotic stream of information.

To boil it down:

I expect projections to handle each event within scope (in this case, specified EventTypes fromAll() ) exactly once.

When I create and enable the projection Read_RoomAssignments_Assignments_Projection (see Projection.txt), the stream “Read_RoomAssignments_Assignments” contains the entries as specified in StreamBeforeReset.txt. This contains the same events several times. When i reset the projection, the stream ends up like StreamAfterReset.txt, where each event is represented exactly once, as I would expect.

My question is: How can i achive a result like “StreamAfterReset” without resetting my Projection?

StreamBeforeReset.txt (248 KB)

StreamAfterReset.txt (195 KB)

Projection.txt (1.19 KB)