GES 3.5.0 projections faulted

EventStoreDB
#1

After updating to 3.5.0 two of my projection have broken.

  1. Simple projection where I use stats as a source of ticks

fromStream(’$stats-10.8.58.11:2114’).

when({

‘$init’ : function(s,e) {

return { “timeMillis”: new Date().getTime() };

},

‘$statsCollected’ : function(s,e) {

var time = new Date();

var timeMillis = time.getTime();

if (s.timeMillis < timeMillis) {

s.timeMillis = timeMillis;

emit(‘clock’, ‘tick’, { “time”: time, “timeMillis”: timeMillis});

}

}

});

``

It is in state Faulted (Enabled) with error An event emitted in recovery differs from the originally emitted event. Existing(‘tick’, ‘$stats-10.8.58.11:2114: 54292’). New(‘tick’, ‘$stats-10.8.58.11:2114: 54291’)

  1. Another projection was in state starting. I tried to stop it. And now it is freezed in state stopping.
#2

For #1 how big is your db and can I send it?

#3

Sorry on phone meant to add

For #2 I think Pieter will want logs as there is a lot of additional logging added in 3.5.0

#4

Do you ask if I can send you my DB?

#5

Asked if it’s possible and the size …

#6

Just want to know if there was any feedback regarding getting the db.

#7

The size is 2,5MB. Sorry I can’t send you a db content. It’s confidential PCI DSS compliant data.
Is it possible to reset stopping projections? I can provide you with logs.
Another symptom is when I hit Cmd+R on Projections tab I get `Projections are not enabled on the node` error.

#8

Sorry it’s 2,5 GB. And that’s feels too much actually. There are 500 streams with 5 events approx.

#9

Have you limited statistics and/or scavenged?

#10

Nope. Limit how? Using metadata?

#11
  1. Are you running a cluster, if so, do all the nodes in the cluster have their configuration for projections set to the same level?
  2. Is this reproducible from your backup? I am assuming that you made a backup of the database before you upgraded Event Store. If this is the case,

a. If you run with the previous version of Event Store, does the same issue occur from the backup?

b. If you run with 3.5.0 from the backup, does it produce the same results?

#12

I’ve just ran scavenge and it saved me 1,838,102,720 bytes. That’s nice.

#13

Yes $maxAge/$maxCount on the $stats stream(s)

#14

1. Yes I run cluster. What configuration do you mean?
2. I’ll try later today.

#15

–run-projections=all

``

from the command line or any of the other means of configuring an Event Store node. More details here in the docs on configuration

#16

This flag is set using RunProjections: All