Upgrade path to Cloud

Hi

We are now using V5 with Eventstore.JVM and ESJC clients. And hope we can go to cloud soon. But i don’t see an upgrade path. We have 20-30 applications that use these clients. Those clients dosen’t work on V20 and the new gRpc client dosen’t work on v5. We can’t upgrade all this with downtime.

Are there any plans?

Event Store Cloud clusters have TCP enabled by default. Also, v20 TCP client works with v5 servers, unless you use features that are v20-specific, like server-side filtering (which will be just ignored for v5 servers).

The usual migration path is to move as-is, using v5 TCP client with v20 servers (cloud or not). Then, gradually move to gRPC clients, using the Event Store-supported Java library in your case.

ESJC and Eventstore.JVM dosen’t work with v20. Is there a plan to add support for v20 for those then? Eventstore.JVM dosen’t work with tls, which is required when going to a cloud provider. ESJC dosen’t have any support for v20.

Would you mind sharing the error you got when connecting to a v20 server using ESJC? Both the Rust and Haskell TCP clients were able to connect to a v20 server with no changes.

I think it should work but you have to explicitly disable TLS as v20+ is secure by default.

Its not an option to run without tls encryption.

You don’t have to disable TLS, @alexey.zimarev said it’s enabled by default now. If you were using your client with TLS then you don’t need to change anything. Considering you are using ESCJ, what error did you get last time you try connecting to a V20 server?

We did our teste this summer, so v20.10 wasn’t available. This is what i get with esjc on the server side.

node1.eventstore | [ 1,44,21:35:24.128,INF] [S"172.30.240.1:48402", L"172.30.240.11:1113"]: Exception on BeginAuthenticateAsServer.
node1.eventstore | System.IO.IOException: The handshake failed due to an unexpected packet format.
node1.eventstore | at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
node1.eventstore | at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
node1.eventstore | at System.Net.Security.SslStream.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
node1.eventstore | at System.Net.Security.SslStream.ProcessAuthentication(LazyAsyncResult lazyResult, CancellationToken cancellationToken)
node1.eventstore | at System.Net.Security.SslStream.BeginAuthenticateAsServer(SslServerAuthenticationOptions sslServerAuthenticationOptions, CancellationToken cancellationToken, AsyncCallback asyncCallback, Object asyncState)
node1.eventstore | at System.Net.Security.SslStream.BeginAuthenticateAsServer(X509Certificate serverCertificate, Boolean clientCertificateRequired, SslProtocols enabledSslProtocols, Boolean checkCertificateRevocation, AsyncCallback asyncCallback, Object asyncState)
node1.eventstore | at EventStore.Transport.Tcp.TcpConnectionSsl.InitServerSocket(Socket socket, Func1 serverCertificateSelector, Func4 clientCertValidator, Boolean verbose) in /build/src/EventStore.Transport.Tcp/TcpConnectionSsl.cs:line 153
node1.eventstore | [ 1,44,21:35:24.128,INF] ES “TcpConnectionSsl” closed [21:35:24.128: N"172.30.240.1:48402", L"172.30.240.11:1113", {4eec888e-c751-4455-a3e9-f7e9f27f7c67}]:Received bytes: 0, Sent bytes: 0
node1.eventstore | [ 1,44,21:35:24.128,INF] ES “TcpConnectionSsl” closed [21:35:24.128: N"172.30.240.1:48402", L"172.30.240.11:1113", {4eec888e-c751-4455-a3e9-f7e9f27f7c67}]:Send calls: 0, callbacks: 0
node1.eventstore | [ 1,44,21:35:24.128,INF] ES “TcpConnectionSsl” closed [21:35:24.128: N"172.30.240.1:48402", L"172.30.240.11:1113", {4eec888e-c751-4455-a3e9-f7e9f27f7c67}]:Receive calls: 0, callbacks: 0
node1.eventstore | TcpConnectionSsl::InitClientSocket(172.30.240.1:48406, L172.30.240.11:1113)
node1.eventstore | [ 1,44,21:35:24.128,INF] ES “TcpConnectionSsl” closed [21:35:24.128: N"172.30.240.1:48402", L"172.30.240.11:1113", {4eec888e-c751-4455-a3e9-f7e9f27f7c67}]:Close reason: [SocketError] “The handshake failed due to an unexpected packet format.”
node1.eventstore | [ 1,44,21:35:24.129,INF] External TCP connection accepted: [Secure, “172.30.240.1:48402”, L"172.30.240.11:1113", {4eec888e-c751-4455-a3e9-f7e9f27f7c67}].
node1.eventstore | [ 1,44,21:35:24.129,INF] Connection ‘“external-secure”""’ [“172.30.240.1:48402”, {4eec888e-c751-4455-a3e9-f7e9f27f7c67}] closed: Success.
node1.eventstore | [ 1,44,21:35:24.129,DBG] Persistent subscription lost connection from “172.30.240.1:48402”
node1.eventstore | [ 1,46,21:35:24.129,INF] [S"172.30.240.1:48406", L"172.30.240.11:1113"]: Exception on BeginAuthenticateAsServer.
node1.eventstore | System.IO.IOException: The handshake failed due to an unexpected packet format.
node1.eventstore | at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
node1.eventstore | at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
node1.eventstore | at System.Net.Security.SslStream.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
node1.eventstore | at System.Net.Security.SslStream.ProcessAuthentication(LazyAsyncResult lazyResult, CancellationToken cancellationToken)
node1.eventstore | at System.Net.Security.SslStream.BeginAuthenticateAsServer(SslServerAuthenticationOptions sslServerAuthenticationOptions, CancellationToken cancellationToken, AsyncCallback asyncCallback, Object asyncState)
node1.eventstore | at System.Net.Security.SslStream.BeginAuthenticateAsServer(X509Certificate serverCertificate, Boolean clientCertificateRequired, SslProtocols enabledSslProtocols, Boolean checkCertificateRevocation, AsyncCallback asyncCallback, Object asyncState)
node1.eventstore | at EventStore.Transport.Tcp.TcpConnectionSsl.InitServerSocket(Socket socket, Func1 serverCertificateSelector, Func4 clientCertValidator, Boolean verbose) in /build/src/EventStore.Transport.Tcp/TcpConnectionSsl.cs:line 153
node1.eventstore | [ 1,46,21:35:24.129,INF] ES “TcpConnectionSsl” closed [21:35:24.129: N"172.30.240.1:48406", L"172.30.240.11:1113", {7886bbbd-e95d-402d-b296-9efd69b4be8b}]:Received bytes: 0, Sent bytes: 0
node1.eventstore | [ 1,46,21:35:24.129,INF] ES “TcpConnectionSsl” closed [21:35:24.129: N"172.30.240.1:48406", L"172.30.240.11:1113", {7886bbbd-e95d-402d-b296-9efd69b4be8b}]:Send calls: 0, callbacks: 0
node1.eventstore | [ 1,46,21:35:24.129,INF] ES “TcpConnectionSsl” closed [21:35:24.129: N"172.30.240.1:48406", L"172.30.240.11:1113", {7886bbbd-e95d-402d-b296-9efd69b4be8b}]:Receive calls: 0, callbacks: 0
node1.eventstore | [ 1,46,21:35:24.129,INF] ES “TcpConnectionSsl” closed [21:35:24.129: N"172.30.240.1:48406", L"172.30.240.11:1113", {7886bbbd-e95d-402d-b296-9efd69b4be8b}]:Close reason: [SocketError] “The handshake failed due to an unexpected packet format.”
node1.eventstore | [ 1,46,21:35:24.129,INF] External TCP connection accepted: [Secure, “172.30.240.1:48406”, L"172.30.240.11:1113", {7886bbbd-e95d-402d-b296-9efd69b4be8b}].
node1.eventstore | [ 1,46,21:35:24.129,INF] Connection ‘“external-secure”""’ [“172.30.240.1:48406”, {7886bbbd-e95d-402d-b296-9efd69b4be8b}] closed: Success.
node1.eventstore | [ 1,46,21:35:24.129,DBG] Persistent subscription lost connection from “172.30.240.1:48406”

On the client i get this exception:

java.util.concurrent.CompletionException: com.github.msemys.esjc.ConnectionClosedException: Connection was closed.

at java.base/java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:412)
at java.base/java.util.concurrent.CompletableFuture.join(CompletableFuture.java:2044)
at com.github.msemys.esjc.AbstractEventsIterator.iterator(AbstractEventsIterator.java:39)
at com.github.msemys.esjc.AbstractEventsIterator.hasNext(AbstractEventsIterator.java:23)
at com.github.msemys.esjc.StreamEventsIterator.hasNext(StreamEventsIterator.java:13)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)
at TEster.readForsendelse(TEster.java:40)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:206)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:131)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
Caused by: com.github.msemys.esjc.ConnectionClosedException: Connection was closed.
at com.github.msemys.esjc.operation.manager.OperationManager.cleanUp(OperationManager.java:46)
at com.github.msemys.esjc.EventStoreTcp.disconnect(EventStoreTcp.java:629)
at com.github.msemys.esjc.EventStoreTcp.handle(EventStoreTcp.java:894)
at com.github.msemys.esjc.task.TaskQueue.process(TaskQueue.java:48)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.github.msemys.esjc.node.cluster.ClusterException: Failed to discover candidate in 10 attempts.

Connection code:

    eventstore = EventStoreBuilder.newBuilder()
            .clusterNodeUsingGossipSeeds(cluster -> cluster
                    .gossipSeedEndpoints(asList(
                            new InetSocketAddress("127.0.0.1", 1113),
                            new InetSocketAddress("127.0.0.1", 1112),
                            new InetSocketAddress("127.0.0.1", 1111))))
            .useSslConnection()
            .userCredentials("admin", "changeit")
            .build();

I am using the docker-compose setup from this example: https://developers.eventstore.com/server/v20/server/installation/docker.html#use-docker-compose

Yeah, but gossip goes over HTTPS, why did you specify the TCP port for it? The page clearly says that HTTP ports for that compose file are 2111, 2112 and 2113.

Because its late at night here. And i got the same error message from client and nothing logged on the servers.

The problem is probably this line: URL url = new URL(“http://” + gossipSeed.endpoint.getHostString() + “:” + gossipSeed.endpoint.getPort() + “/gossip?format=json”);

Which only has support for http and not https on gossip endpoint.

What I meant is 1111, 1112 and 1113 are TCP ports, not HTTP(S) ports.

Yeah but the connection string is still http. So it won’t connect.

We will just wait for them to support https in the gossip connect url.

If you plan moving to v20 server, I’d say you will get better off by refactoring your clients to our gRPC client library for Java. It’s supported by us and is future-proof. Besides, gRPC is proven to be much more predictable with regards to writes latency.