Hi everyone and thanks for the great work.
I really like to bring EventStore to Haskell ecosystem. But while HTTP API is quite exhaustive, I found nothing on TCP. Could someone provide some guidelines or pointers on how communication works between an EventStore server and a client over TCP ?
I’m aware of the Scala client but it seems it has been written by someone with intimate knowledge of EventStore internals, meaning the code is not very commented.
Thanks for your help,
Yorick
If I understand correctly, it’s protobuf messages over a socket. The C# client is probably also instructive.
Hi Yorick,
I would love to see a Haskell client for EventStore. If you decide to go ahead with this and want a contributor let me know.
As Sebastian points out the C# client is the best place I know of to start. I haven’t looked at the Scala client but I’ve always found the C# event store code quite easy to follow.
cheers
Andrew Browne
brownie@brownie.com.au
Hi All,
We’ve not really documented the internals of the protocol. It’s fully async (client is responsible for correlating responses with requests or timing out requests etc), and the messages which go between the client and the server are serialized protobufs framed
with a length. https://github.com/EventStore/EventStore/blob/dev/src/Protos/ClientAPI/ClientMessageDtos.proto describes the message layout. I understand this probably isn’t enough information to get going but it may be a starting point 
Cheers,
James
Writing a client is actually pretty straight forward until you get to things like reconnection/clustering/subscriptions. Have you looked through any of the existing clients (c#/js/Akka?)
Actually I think I’ll put some comments in the .proto file about usage. They could be valuable in general…
Thanks everybody for your replies.
I was aware of the protobuf protocol. What I’m not are things like Heartbeat for instance. I understand the point of the message but I don’t have a clue of how it looks like on the wire.
It seams also that Heartbeat isn’t the only message not reported in the .proto file.
All I got so far, are my generated Haskell ADT’s from the .proto file. I have also a pretty clear idea of what kind lib/tech I gonna use too.
All I need right now is how the communication works. I will follow your advice on having a look on the C# client.
I already stared at the Scala client code with no success so far.
**@**Andrew Browne
I really need that Haskell lib to happen ! Currently, I have an ‘EventStore’ like system that uses PostgreSQL
and table schemas quite close to ones Grey Young proposed in a post on eventsourcing. That does the job (for now)
but obviously not a long term solution. Help is of course welcomed, I have a blank Github object waiting to have
some code to host
Regards,
Yorick
I started writing up some docs btw (also things like cluster vs single
node, gossip etc)
Good to hear and can’t wait to have those.
From memory…
Messages like heartbeat don’t have a payload, so there’s nothing to serialize. They are just straight TCP messages that include things like auth and CorrelationId.
Brian
yes.
Also, you can find these “missing” messages in the tcp.commands.cs file.
Just be careful as there is some overlay there with the internal
networking commands (eg replication etc)
Thanks all of you for the pointers.
So far so good, I’m currently tickling the server. I’m able to parse messages coming from it. Some questions though:
Yorick
Inline.
Thanks all of you for the pointers.
So far so good, I’m currently tickling the server. I’m able to parse messages coming from it. Some questions though:
- Why the server keeps sending me CreateChunk command and it starts right after I’ve established a connection.
Test?
- Sporadically, I got a command not referenced in EventStore codebase: 0x71. Does someone know what this is ?
Test?
- Am I supposed to send a command when I’ve just established the connection ? When browsing the ClientAPI code, I don’t see any but maybe I’m missing something.
- Does HeartbeatRequests are coming from the server or have to be emitted by the client ? When browsing the code again, I see a method named manageHeartbeats. Telling me that happens in client side. But in the same time, EventStoreConnectionLogicHandler.HandleTcpPackage handles possible HeartbeatRequests from the server. So why I didn’t receive any HeartbeatRequest so far ?
You should respond the a heartbeat message. My guess is above?
Sorry Greg I don’t get most of your reply
After some investigations, it turns out I forgot to unframe data coming from the server
I’m getting a lot of heartbeat timeouts leading to disconnection from the server.
If you just open a connection and do nothing else, are any heartbeat responses received or do they all timeout? What functionality of the client are you working on?
When developing the nodejs client, I had a period where I was dealing with timeout issues, but sadly I cannot recall right now what I was working on and what the problem was. (I think it was subscriptions) I’m hoping that something you say will trigger something in my memory. 