We are evaluating whether we can migrate the calculation of our average of incoming measurement events to projections in EventStore. I up-gradate today to the latest version (coming from 0.9), and I can’t remeber I had the problem in the past with the same projections…
What we do, as test, is writing measurement values for 50 meters. And we have 4 projections calculating averages. After a while we get following exception. This can be some seconds to minutes depending on how hard we stress the writing of the events…
Emitted event caused position before last handled position
The projections looks like this:
fromAll()
.foreachStream()
.when( {
MeasurementRead : function (previousState, measurementEvent) {
if (measurementEvent.body == null) return previousState;
var timestamp = measurementEvent.body.Timestamp;
var reading = measurementEvent.body.Reading;
if (previousState.total == null) {
return createState(reading, 1, timestamp);
}
if (isDifferentTimeSlot(timestamp, previousState.lastTimestamp)) {
var name = eventStreamId(measurementEvent.streamId, previousState.lastTimestamp);
var event = createEvent(formatTimeSlot(previousState.lastTimestamp), previousState.total, previousState.count);
emit(name, "MeasurementPeriod", event);
return createState(reading, 1, timestamp);
} else {
return createState(previousState.total + reading, previousState.count + 1, timestamp);
}
};
});
The helper methods implementations are omitted for verbosity…
Any idea on how to prevent this? I can upload the sample project if this helps…
the most probably cause of this error is another projection (existing, removed, stopped) has emitted an event to the stream your current projection attempts to write to. Projections are supposed to be exclusive.
The version from the dev branch produces better messages in this case. We also added a way to restart a projection (for the development purposes only). Restarting causes the projection to ignore already existing events in the streams this projection emits to.
Earlier versions (before 1.0.3) silently ignored any events emitted to such streams until the second projection reaches the position of the last event handled by the first projection.
summary:
do not write to the same stream from different projections
the next version will provide better messages showing where the conflict occurred
you will be able to restart a projection while developing/debugging it (make sure you understand the consequences if you do this in production).
Projections attach metadata to all emitted events with a reference to the event that caused this emit. This is required to handle projections recovery when the process is terminated/crashes without proper shutdown.
Projections avoid writing the same event twice to the output stream this way.
If you need to combine events from different sources you can either:
a) use another projection to join streams
b) or, take a risk of the projection writing the same events twice to the stream when recovering after the system restart, if you write to the stream with AppendToStream (as you suggested).
Likely b) will require additional option in the future to acknowledge your intention explicitly.
a) Sounds safer. I am thinking of this in the context of an event-sourced saga / process manager / whatever. For-each Saga type you would have a projection that re-partitions by correlation id. But the Saga would also need to emit its own events, mainly when it was done.
In my case each projection writes to different streams, I add a projection specific suffix to the results streams. Eg: ‘+H’
So that shouldn’t be the problem my situation.
I enabled the 4 default projections $by_category, $by_event_type, $stream_by_category & $streams by default.
When I don’t enable these projections the problem occurs later, but it still occurs…
I did some more tests and it even happens when I have only one projection of my own. (I always start with an empty store, so previous projections should not affect my tests)
I logged the emmited events (50) when the execption occurs:
The validation fails on validating the first Checkpoint event: $projections-MeasurementReadPerHour-Meter-4-checkpoint
When _last is {485008/484154}
Any other thoughts? Let me know if you need more information…
I tried the dev branch (1.1) and I don’t have the projection problems anymore. Good job
One thing I noticed is that the default projections are gone, and a [$users](http://127.0.0.1:2113/web/view-projection.htm#http://127.0.0.1:2113/projection/%24users) projection is added. Is this normal behavior?
What does the $users projection do?
Thanks for the support…
PS: i had troubles compiling build-js1_x64.cmd from the project root, it should be src\EventStore instead of … i guess…
