Projection taking a long time to populate

EventStoreDB
#1

I created a projection merging three $ce-* streams together. It’s been populating for a day and a half. At one time it got to 90 something %, and then jumped back down. In addition, every one of our other projections is also below 100%. The end of the day, our system is basically unusable. At least it was over a weekend, but the weekend is coming to a close.

Questions:

  1. Why are the other projections also populating?
  2. Why does it take so long (we’ve done this a few times in the past. Every time it gets worse and worse, from a few hours to a half a day, and now coming close to two days…obviously has much to do with the amount of events we have.
  3. Are we doing something wrong with eventstore? Should we be using the $all stream instead?
  4. Is there anyway to speed it up?
#2

Sounds pretty similar to things we do on our systems
How many events are on your category streams?
Could I have look at your projection?

#3

This is what my projection looks like:

fromStreams([
'$ce-pledge',
'$ce-payment',
'$ce-donor',
'$ce-person'])
.when({
			$any: (state, event) => {
				linkTo('pledge-reporting', event)
			}
		})

the category streams have millions of events

$ce-donor, for example, has 1,654,8577 events

#4

Unfortunately, the projection will have to go through each event, from category, and then link it to your new stream, which I am guessing is the root of your problem.
If I was managing this, I would look at reorganing the streams a little to make concurrent processing possible.
For example. is there a common Id that relates the four streams (maybe the donor?)

Using this technique, I would build a stream per relationship:

linkTo(‘pledge-reporting-’, event)

This means if you need to build one big stream, you could do it concurrently by using foreachStream.

Alternatively, could correlationId do this free of charge?

#5

Before I try to digest what you said :slightly_smiling_face:

Is there anyway we can save ourselves right now?

#6

I don’t think so, but could you run a test, and just see how long it takes query to process the events, to see if the linkTo is the source of the delay:

fromStreams([
‘$ce-pledge’,
‘$ce-payment’,
‘$ce-donor’,
‘$ce-person’])
.when({

$any : function(s,e){
return { count :0 }
},
$any: (state, event) => {
//Is there anything you can filter out?
s.count++; //see how it takes to count all your events
}
})

If that takes just as long, I guess it means you can’t avoid this.

The CorrelationId is a system projection, so I think that gets processed as a C#, rather than javascript, which might be faster than what you are trying.

#7

something weird we just noticed

it seems all events from $ce-donor (and possibly others, but that’s what we see now) are now being written to $ce-pledge!!

#8

Ah, the stream name pledge-reporting for linkTo will also write to $ce-pledge
A never ending cycle of events :o