Projection stream position does not move from -1

EventStoreDB V20.X
#1

We have a problem with a projection where the position does not move from -1

We have replicated this on our system (and other systems) with the following steps

  1. Truncate a stream (tried using a Delete from UI and maxAge) - note the stream tends to have millions of truncated events.
  2. Create a projection which uses this truncated stream
  3. Projection will remain at Position -1

If I take the same projection code and change the fromStream to another non truncated stream, the projection bursts into life. As soon as I introduce the truncated stream, Position returns to -1 (even on the stream that was working when using fromStreams)

It sounds similar to the github issue below:

Can anyone help on this? I am hoping it’s something I have done to cause this, but unsure where else to look.

Running Event Store 21.2.0.0

#2

Noticed this issue is the same:

#3

I have pulled down the Eventstore source code, and have a local environment which replicates this problem.
I don’t know the source code at all, so I am fumbling around in the dark trying to decipher where to look.
From what I can see ClientMessage.cs is where a lot of the action is happening.
I can see my projection being used here (as well as the checkpoint and emitted streams) but not entirely sure where to look next.

As a side note, on one of the debug sessions, my projection actually burst into life, but not been able to replicate it.

Would anyone who is familiar with the code base point me in the right direction?
Even just a few classes I could put a breakpoint on.

This issue is causing us real problems so any assistance would be appreciated.

Steven

#4

Still haven’t found the source of the problem, but I do have a workaround.
If I manually add an event to the stream, my projection bursts into life.
Replicated this multiple times now.

#5

Hey Steven , thanks for the PR ,
it’s going to get a look by some of the dev team people (probably next week, the ones that know more about this are on holiday today )
In the meanwhile : would it be possible to share he following 2 ?

  • the JS projection you’re using as well as the
  • reproduction you mentionned ?
#6

Yves,

Thanks for the reply.

I can host a Live Share for the reproduction, or alternatively, here are the steps:

  1. Truncate a stream (I have tried via the UI and maxAge on the metadata
  2. Create a projection which uses this truncated stream (see projection example below)
  3. Start the new projection.

The projection will stay at -1

For my own local work, I have added a new event to the truncated stream, and the projection bursts into life.

With the projection, I tried replacing the truncate stream with another non truncated, and the projection works fine.

Here is an example Projection:

fromStream('projections_StoreProductManager_result')
    .partitionBy(function(e) {
        if (e.data.organisationProductId) {
            return "OrganisationProduct-" + e.data.organisationProductId.replace(/-/gi, "");
        }
    })
    .when({
        $init: function(s, e) {
            return {
                storeProductActivityReceived: false
            };
        },

        $any: function(s, e) {
            // Pass to the event handler for further processing if required
            eventHandler(s, e);
        }
    });

function getStreamName(s) {
    return "projections_ExternalProjections_result";
}

function eventHandler(s, e) {
    if (e.eventType === 'StoreProductHistoryUpdatedEvent') {
        //Once we receive one Store product activity, we will pass onto our new "C Sharp projection stream"
        s.storeProductActivityReceived = true;
    }

    if (s.storeProductActivityReceived) {
        linkTo(getStreamName(s), e);
    }
}
#7

Any movement on this?
I wouldn’t mind looking at fixing it, but I am struggling where to start looking for this in the code.
Guessing it’s not a wide spread problem, but for us it’s really killing our event processing just now and have to keep coming up with creative workarounds.

#8

Might have a fix!