About the NodeIP setting

wangziyejimo · March 20, 2024, 8:25am

Hello, I encountered some issues while trying to use Eventstore.
Here is my configuration↓

Snapshot taken at 2024-03-20 16:22:14
Internal Tcp                    External Tcp                    Http                    Status  State          Timestamp (UTC)     Checkpoints
35.243.xx.40:2113               35.243.xx.40:2113               35.243.xx.40:2113       Alive   Manager       2024-03-20 14:42:29  n/a                                                                                       
35.221.xxx.213:2113             35.221.xxx.213:2113             35.221.xxx.213:2113     Alive   Manager       2024-03-20 14:42:29  n/a                                                                                       
34.146.xx.19:2113               34.146.xx.19:2113               34.146.xx.19:2113       Alive   Manager       2024-03-20 14:42:29  n/a                                                                                       
10.0.2.4:1112                   10.0.2.4:0                      10.0.2.4:2113           Alive   Leader        2024-03-20 16:22:12  L8993/W9357/C9357/E6@8715:{9095d23b-8ca7-4897-8643-1a56da13d8fc}                          
10.0.2.3:1112                   10.0.2.3:0                      10.0.2.3:2113           Alive   Follower      2024-03-20 16:22:13  L8993/W9357/C9357/E6@8715:{9095d23b-8ca7-4897-8643-1a56da13d8fc}                          
10.0.2.2:1112                   10.0.2.2:0                      10.0.2.2:2113           Alive   Follower      2024-03-20 16:22:13  L8993/W9357/C9357/E6@8715:{9095d23b-8ca7-4897-8643-1a56da13d8fc}

My first question is, why are there three additional nodes, and why are they associated with external IP addresses?

Then I tried to modify the configuration file to set the external IP as the external IP and the internal IP as the internal IP.

The error is as follows:

{"@t":"2024-03-20T06:25:28.9333725+00:00","@mt":"Hosting failed to start","@l":"Error","@i":732972521,"@x":"System.Net.Sockets.SocketException (99): Cannot assign requested address\n   at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)\n   at System.Net.Sockets.Socket.Bind(EndPoint localEP)\n   at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportOptions.CreateDefaultBoundListenSocket(EndPoint endpoint)\n   at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()\n   at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportFactory.BindAsync(EndPoint endpoint, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.TransportManager.BindAsync(EndPoint endPoint, ConnectionDelegate connectionDelegate, EndpointConfig endpointConfig, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass28_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()\n--- End of stack trace from previous location ---\n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.EndpointsStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(ListenOptions[] listenOptions, AddressBindContext context, Func`2 useHttps, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)\n   at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__15_1(IHostedService service, CancellationToken token)\n   at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)\n   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)\n   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)\n   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)\n   at EventStore.ClusterNode.Program.<>c__DisplayClass0_1.<<Main>g__Run|4>d.MoveNext() in /home/runner/work/TrainStation/Train

My external IP is fine, and the ports are also open, So, any suggestions? What should I do?

I am using the latest version 24.2

yves.lorphelin · March 20, 2024, 5:36pm

What are the ip adresses ( A records ) in the cluster DNS addrress ?

alexey.zimarev · March 21, 2024, 3:10pm

I believe Yves asked about what that blurred DNS name specified in ClusterDns resolves to.

alexey.zimarev · March 25, 2024, 8:56am

If you use that cluster DNS name, the NodeIp needs to be set to those public addresses, not internal addresses. Replication IP is good.

wangziyejimo · March 25, 2024, 8:59am

I set it up as you said. the error will be like this↓

{"@t":"2024-03-20T06:25:28.9333725+00:00","@mt":"Hosting failed to start","@l":"Error","@i":732972521,"@x":"System.Net.Sockets.SocketException (99): Cannot assign requested address\n   at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)\n   at System.Net.Sockets.Socket.Bind(EndPoint localEP)\n   at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportOptions.CreateDefaultBoundListenSocket(EndPoint endpoint)\n   at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()\n   at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportFactory.BindAsync(EndPoint endpoint, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.TransportManager.BindAsync(EndPoint endPoint, ConnectionDelegate connectionDelegate, EndpointConfig endpointConfig, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass28_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()\n--- End of stack trace from previous location ---\n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.EndpointsStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(ListenOptions[] listenOptions, AddressBindContext context, Func`2 useHttps, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)\n   at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__15_1(IHostedService service, CancellationToken token)\n   at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)\n   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)\n   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)\n   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)\n   at EventStore.ClusterNode.Program.<>c__DisplayClass0_1.<<Main>g__Run|4>d.MoveNext() in /home/runner/work/TrainStation/Train

wangziyejimo · March 25, 2024, 9:06am

One of my configration like this

---
# Paths
Db: /var/lib/eventstore
Index: /var/lib/eventstore/index
Log: /var/log/eventstore

# Certificates configuration
CertificateFile: /etc/eventstore/certs/node.crt
CertificatePrivateKeyFile: /etc/eventstore/certs/node.key
TrustedRootCertificatesPath: /etc/eventstore/certs/ca

# Network configuration
ReplicationIp: 10.0.2.4
NodeIp: 3x.2xx.1xx.x
HttpPort: 2113
IntTcpPort: 1112
EnableAtomPubOverHTTP: true
Insecure: true

# Cluster gossip
ClusterSize: 3
DiscoverViaDns: true
ClusterDns: public.dns.com

# Projections configuration
RunProjections: All

And i could see the error in error log like i said before.

yves.lorphelin · March 25, 2024, 10:20am

you need to add AdvertiseHostToClientAs: [node DNS name] to the config of each node

in term of DNS
- you also need an A record per node, with the IP address of the node
in term of Certificates make sure the cert includes
- the dns name of the nodes as SAN
  alternatively
- usa a wildcard cert

this article should help you with the complete setup
( some settings name have changed)

wangziyejimo · March 26, 2024, 8:16am

Thank you for your answers. With reference to the documentation, I am now able to read and write using the client. However, I am unsure if the appearance of ‘unknown’ here is normal. The cluster I have set up consists of 3 nodes.

yves.lorphelin · March 28, 2024, 4:23pm

@wangziyejimo i haven’t forgotten about this, I’ll hae a look tommorrow .
( sorry been busy a lot)