Node unreachable


I am using event store cloud.
I have one persistent subscription with 3 consumers and 2 catch up subscription.

I am trying to process around 100k messages by adding them to a command stream and then processing them by using a persistent subscription to all with filtering on event types.
I am using the default settings for the persistent subscription.

Each message is taking about 150 - 450 ms to process.

Each message that gets processed is adding it’s own stream and adding about 10 events.

When doing this after a couple of minutes the node becomes unreachable and the connection to the event store stops working. I get a critical notification in the cloud ui with a message node 0 can not be reached.

Then after awhile the status for the cluster is “down”.

Right now I am in test so I am using a M8 cluster with 1 node but I have tried using a M16 cluster with 3 nodes and I still get the same error it is just that 1 of the 3 nodes becomes unreachable.

I do not know what I am doing wrong and what I need to do to not make the cluster stop working.

If I process 1k events instead of 100k everything works fine.

Thanks for any help I can get.

Hi @philip.oterdahl, could you make a ticket with this information in the support system
with both cluster Id & timeframe this happenned

1 Like

@philip.oterdahl timefame in UTC if possible , make it easier to search the metrics

1 Like