Projection stops working when handling more than 100 000 messages

EventStoreDB
#1

Hello

We have a projection which stops working when it handles a lot of messages, like more than 100 000. It does nothing more than partitioning the messages in different streams.

The projection just stops working, no errors in state. It just does not pick up anymore new messages which are saved to Event Store

fromStreams(’$ce-Dog’, ‘$ce-Cat’, ‘$ce-Bird’, ‘$ce-Rabbit’)
.when({
$any: (s, e) => {
if (!e || !e.eventType || e.eventType.startsWith(“Bulk”) || !e.streamId) {
return;
}
if (e.streamId.startsWith(“Dog-”)) {
linkTo(‘Dogs’, e);
}
if (e.streamId.startsWith(“Cat-”)) {
linkTo(‘Cats’, e);
}
if (e.streamId.startsWith(“Bird-”)) {
linkTo(‘Birds’, e);
}
if (e.streamId.startsWith(“Rabbit-”)) {
linkTo(‘Rabbits’, e);
}
}
});

``

What could here be the issue?

Many thanks

Maarten

#2

  1. What version of Event Store are you on?

  2. Are you able to provide us with logs?

  3. Are you running in a cluster?

  4. What does your environment look like? (disk, network, cpu, ram)?

#3
  1. What version of Event Store are you on?

4.0.1

  1. Are you able to provide us with logs?

Yes, see attached file.

  1. Are you running in a cluster?

Yes, 3 node cluster with gossip seeds.

  1. What does your environment look like? (disk, network, cpu, ram)?

HDD 40GB, 1Gbit, 2.4GHz Xeon 2 cores, 16GB

log.txt (21.8 KB)

#4

Maarten,

  1. The log you have provided start at 2017.08.18 08:36:00.868 UTC and ends at 2017.08.18 08:36:04.822. Would you mind attaching the logs from all the nodes and for a time period that covers where the projections stopped processing events? This is just so that we are able to diagnose and piece together the possible reasons for why the projections could have stopped?

  2. Is this environment a production or QA environment?

a. Do you perhaps have a specification for the disk (is this an SSD?), you mention HDD which would have us assume it’s a magnectic drive.

#5

Hi Pieter

I can provide the logs but I cannot post them here for security reasons. Can I mail them to you. Our company has also a paid support contract.

The disks are indeed magnetic. The servers are virtual. The problem happens on all environments (TEST, ACC and PROD)

Maarten

#6

If your company has a paid support contract, please don’t hesitate to open a ticket in freshdesk and attach the logs there.