"Error starting gRPC call. HttpRequestException: Connection refused (127.0.0.1:2113) SocketException: Connection refused"

We are appending events to a stream but get some strange exceptions. Our code is like this:

await using var client = new EventStoreClient(EventStoreClientSettings.Create(options.ConnectionString));

var eventData = new EventData(
Uuid.NewUuid(),
ServiceConstants.IntegrationEventType,
Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(context, Formatting.Indented,
new JsonSerializerSettings()))
);

await client.AppendToStreamAsync(
ConstructStreamName(context.TenantId, context.JobId),
StreamState.Any,
new[] { eventData },
cancellationToken: cancellationToken
);

The connectionstring is like this: esdb+discover://eventstoredb.monitoring.svc.cluster.local:2113?tls=false&keepAliveTimeout=10000&keepAliveInterval=10000

When running this we get the following exception:

Grpc.Core.RpcException
Status(StatusCode=“Unavailable”, Detail=“Error starting gRPC call. HttpRequestException: Connection refused (127.0.0.1:2113) SocketException: Connection refused”, DebugException=“System.Net.Http.HttpRequestException: Connection refused (127.0.0.1:2113)
—> System.Net.Sockets.SocketException (111): Connection refused
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at System.Net.Sockets.Socket.g__WaitForConnectWithCancellation|281_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
— End of inner exception stack trace —
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp2ConnectionAsync(QueueItem queueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Grpc.Net.Client.Internal.GrpcCall2.RunCall(HttpRequestMessage request, Nullable1 timeout)”)

One strange thing here is that the error message does not match the connection string.

In addition, this one happens:
System.ObjectDisposedException: The CancellationTokenSource has been disposed.
?, in EventStoreClientBase(EventStoreClientSettings settings, IDictionary<string, Func<RpcException, Exception>> exceptionMap)+(ReconnectionRequired endPoint, Action onBroken) => { }
?, in async Task SharingProvider<TInput, TOutput>.FillBoxAsync(TaskCompletionSource box, TInput input)

And this last exception happens many times, about 1500 times in one hour.

We are using the EventStore.Client.Grpc.Streams nuget, version 22.0.0

What could be causing this behavior, are we missing something obvious in our code?

I really appreciate any help you can provide.

I am quite sure that the client works fine if it’s configured correctly. Could it be that you have a config file with localhost in the connection string that gets to the deployment instead of the one you want there? At least that’s what I check first - what is the actual value of the connection string variable when the application executes. If you use app settings config files, you might have wrong environment for which the config file is applied.