C# Client Reconnection and blocking

EventStoreDB
#1

Hi,

I can’t seem to figure out how to configure the client API for a reconnection scenario.

Whenever I set the connection to reconnect and the EventStore is down the connect operation returns immediately and then first operation just hangs ignoring all timeouts and retries.

With the event store not running

var settings = ConnectionSettings.Create()
.KeepReconnecting()
.SetReconnectionDelayTo(TimeSpan.FromSeconds(5))
.WithConnectionTimeoutOf(TimeSpan.FromSeconds(2))
.SetOperationTimeoutTo(TimeSpan.FromSeconds(2))
.PerformOnAnyNode()
.SetDefaultUserCredentials(creds);

Connection = EventStoreConnection.Create(settings, CommonUtil.Utilities.EndPointParse(config.EventStoreIp, 1113));
Connection.Connect(); //<= returns immediately. As this is the Sync version shouldn’t it block and then timeout?
var evt = Connection.ReadEvent("$users", 0,false); // <=Hangs indefinitely until event store is started

What connection options should I use to get a connection that will fail operations while the eventstore is down but will still try to reconnect?

Note: I have tried all sorts of ways to do this, the only thing that seems close to working is using the defaults, but then it won’t reconnect.

var settings = ConnectionSettings.Create()

.PerformOnAnyNode()
.SetDefaultUserCredentials(creds);

I know I could create a wrapper and handle all of the events, but it seems like the client API is already doing this.

Also is there a way to check the current state of the connection?

Thanks,

Chris

#2

I also found it a bit unintuitive that Connect() returned without actually connecting. The way I worked around this is to use the callback methods:

var settings = ConnectionSettings.Create()

.KeepReconnecting()

.OnReconnecting(OnReconnecting)

.OnDisconnected(OnDisconnected)

.OnConnected(OnConnected)

.OnClosed(OnClosed);

Then in the methods you can do stuff like logging or exiting your application based on how you intend to resolve the connection issue.

  • Morten.
#3

We will take a look at changing this

#4

Hi Chris

Did you ever decide on a strategy for this? I’m thinking of how to handle this scenario, and was wondering what you ended up doing.

I have been playing around with the settings but cannot seem to get the limit retries to work as I thought they might:

var settings = ConnectionSettings.Create()
                                  .KeepReconnecting()
                                  .SetReconnectionDelayTo(TimeSpan.FromSeconds(5))
                                  .WithConnectionTimeoutOf(TimeSpan.FromSeconds(2))
                                  .SetOperationTimeoutTo(TimeSpan.FromSeconds(2))
                                  .LimitRetriesForOperationTo(5)
                                  .LimitAttemptsForOperationTo(5)
                                  .OnDisconnected(EventStoreEventHandlers.OnEventStoreDisconnected)
                                  .OnClosed(EventStoreEventHandlers.OnEventStoreClosed)
                                  .OnConnected(EventStoreEventHandlers.OnEventStoreConnected)
                                  .OnErrorOccurred(EventStoreEventHandlers.OnEventStoreError)
                                  .OnReconnecting(EventStoreEventHandlers.OnEventStoreReconnecting)
                                  .UseCustomLogger(new EventStoreLogger());





Ideally, I'd like the following behaviour:



Automatically re-connect when ES comes back up - this is working with the above settings

Operations like connection.AppendToStream to retry the number of times specified in the settings, then return an error. Currently this just hangs until the ES is reconnected



I guess I don't really understand what .LimitRetriesForOperationTo() and .LimitAttemptsForOperationTo() do, as I presumed they would apply to actions like .AppendToStream() but they do not appear to have any effect on this. :)



Are there any docs on this? (already read https://github.com/eventstore/eventstore/wiki/Overview-(.NET-API) and https://github.com/EventStore/EventStore/wiki/Connection-Options-%28.NET-API%29)



cheers

Tom



p.s. EventStoreEventHandlers methods just log info at the moment
#5

As of now if no connection exists it is not counted as a retry (as it was never attempted). I will look through this code this evening to see if there is a work around or a new config that can help this

#6

Thanks Greg.

In my particular use case, it will be fine to just throw an exception if disconnected when trying to append to a stream (ideally after a period of retries), as the code can deal with this. An alternative would be to have a ConnectionStatus property on the connection (i.e. Connected, Disconnected etc), so I could query this before trying to append, and handle retries myself. (I could do this myself using the OnConnected, OnDisconnected events, but there is already a ConnectionState on the EventStoreConnectionLogicHandler that could potentially be exposed on the connection?)

Thanks

Tom

p.s. I’ve now read the code for LimitRetriesForOperationTo() and LimitAttemptsForOperationTo() so now understand that they essentially do the same thing, except in terms of semantics i.e. limiting once means never retry, limiting to 2 means retry once etc…

#7

Hi,
I know this is a very old thread but I am facing the same issue now.

I want my connection to reconnect automatically but the actions on the connections (AppendToStream / ReadStreamEventsForwardAsync) should immediatly fail if ther is no connection to the server so the caller knows that something went wrong.

I tried different Setting combinations but had no luck:

var connectionSettings = ConnectionSettings
    .Create()
    .KeepReconnecting()
    .LimitRetriesForOperationTo(1)
    .LimitAttemptsForOperationTo(1)
    .SetOperationTimeoutTo(TimeSpan.FromSeconds(1))
    .UseConsoleLogger()
    .Build();

Can anybody please help me? :slight_smile:

Thanks

Lars

#8

I reckon you’re looking for FailOnNoServerResponse ?

Semi-related, there’s also a new API in 4.1.1: SetQueueTimeoutTo which may be of interest when a client/server connection is overloaded in the context of serving requests.

–R