Failure subscribing to all

EventStoreDB
1

I have a bit of code that does something like this:

conn.SubscribeToAllFrom(null, true, (s,e) => o.OnNext(e), s => o.OnCompleted(), (s,sr,e) => o.OnCompleted(), cred);

This seems to work as expected on a test ES db.

However, on our production ES instance it stops reading some way through it. The exception I get is a RetriesLimitReachedException. Turning on client logging, I see we time out with the following errors:

[18,21:25:45.525,ERROR] EventStoreConnection ‘ES-d107f09e-5b03-4b61-a1da-e437ecd667e5’: operation never got response from server.
UTC now: 21:25:45.523, operation: Operation ReadAllEventsForwardOperation ({321f90ff-f26c-47d9-8ff4-0f7930a69a07}): Position: 263666490/263666490, MaxCount: 500, ResolveLinkTos: True, RequireMaster: True, retry count: 0, created: 21:25:38.469, last updated: 21:25:38.469.

On the server I see errors like:

[27824,06,21:25:45.540] Error while wrapping message EventStore.Core.Messages.ClientMessage+ReadAllEventsForwardCompleted.

Object reference not set to an instance of an object.

Is there something wrong with the data written to ES, a bug, or am I doing things incorrectly?

Thanks,

Karl

2

Is there a stack trace included?

3

StackTrace:

at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) at EventStore.ClientAPI.EventStoreAllCatchUpSubscription.ReadEventsTill(IEventStoreConnection connection, Boolean resolveLinkTos, UserCredentials userCredentials, Nullable1 lastCommitPosition, Nullable`1 lastEventNumber)
at EventStore.ClientAPI.EventStoreCatchUpSubscription.b__0(Object _)

InnerException (no stack trace):

Item Operation ReadAllEventsForwardOperation ({3aa3886d-d0d5-4abc-81eb-24d97934709e}): Position: 263666490/263666490, MaxCount: 500, ResolveLinkTos: True, RequireMaster: True, retry count: 10, created: 22:17:14.512, last updated: 22:18:26.390 reached retries limit : 10

4

I meant on the server side, this is just stating that it retried 10
times and failed.

Also what version/environment client/server are you running?

5

Sorry, didn’t understand.

Client 3.4

From GES server. Using 3.0.5 we get the following exception

[PID:32856:035 2016.02.17 01:37:20.630 ERROR TcpDispatcher ] Error while wrapping message EventStore.Core.Messages.ClientMessage+ReadAllEventsForwardCompleted.

System.NullReferenceException: Object reference not set to an instance of an object.

at EventStore.Core.Messages.TcpClientMessageDto.EventRecord…ctor(EventRecord eventRecord) in c:\work\EventStore\src\EventStore.Core\Messages\TcpClientMessageDtoExtensions.cs:line 30

at EventStore.Core.Services.Transport.Tcp.ClientTcpDispatcher.ConvertToResolvedEvents(ResolvedEvent[] events) in c:\work\EventStore\src\EventStore.Core\Services\Transport\Tcp\ClientTcpDispatcher.cs:line 391

at EventStore.Core.Services.Transport.Tcp.ClientTcpDispatcher.WrapReadAllEventsForwardCompleted(ReadAllEventsForwardCompleted msg) in c:\work\EventStore\src\EventStore.Core\Services\Transport\Tcp\ClientTcpDispatcher.cs:line 361

at EventStore.Core.Services.Transport.Tcp.TcpDispatcher.WrapMessage(Message message) in c:\work\EventStore\src\EventStore.Core\Services\Transport\Tcp\TcpDispatcher.cs:line 64

Running on 3.4.0 I don’t see any error, but it seems not to move forward about at the same place.

All running on windows.

perhaps related to: https://github.com/EventStore/EventStore/issues/503 ?

6

It seems to be, are you now getting an error on the client side?

7

As an update internally this has been reproduced but doesn't seem to
be reproducible on 3.4.0/3.4.0 (client + server)

8

Do not get an error at all on the client side. And in fact the catchup subscription never converts to the live subscription.

As I said the i am taking the ESdb that was created on 3.0.5 and then running it on an instance of 3.4

9

What do you get if you enable verbose logging in the client?

10

My apologies. Apparently, i had a prerelease of the client. Switching to the nuget version and it works with server 3.4 as expected. Still got error with 3.0.5.

So upgrading both the client & server to the latest is the remedy.

Thank you for walking me through all the assumptions.

11

prerelease as in 3.5.0?

12

no, a 3.4 prerelease

13

ah ok that makes sense then!