how to compare data from two eventstore streams?

EventStoreDB
1

Here is the issue:

I have two streams in eventstore:

  • stream1 contains raised errors (about 96000 entries)

  • stream2 contains resolved errors (about 95000 entries)

I want to use projections to compare the two streams and return a list of unresolved errors.

i.e. get all entries in stream1 and remove entries if it matches entries in stream2

This is the projection code I have now:

fromStreams([“raised_errors”, “resolved_errors”])

.when({

$init: function() {

    return {raised_count: 0, resolved_count: 0, diff_count: 0, unresolved_errs: []}

},

"StartedErrors": function(state, event) {

    state.raised_count += 1;

    state.unresolved_errs.push(event);

},

"ResolvedError": function(state, event) {

    state.resolved_count += 1;

    var index = state.unresolved_errs.findIndex(function (t) {

       return t.errorId == event.errorId

    });

    state.unresolved_errs.splice(index, 1);

state.diff_count = state.unresolved_errs.length;
}

})

``

The problem with this is that the projection goes through all the events in raised errors before starting in resolved errors. Which means it will render all 96000 entries before removing entries. Thus crashing the projection.

Are there any other ways to compare data from two streams using projections?

2

What is the temporal relationship between the streams? Are raised errors always before resolved errors? Are all raised errors before any resolved errors?

3

Yes, all raised errors are always before resolved errors. Currently all raised errors are before any resolved errors. Thus eventstore goes through all raised errors before resolved errors.
After the initial run, there will be resolved errors in between raised errors. However, the time between raised error and resolved error will be fairly large.