Hi,
I want to evaluate event store for new part of system written in elixir. I’ve been listening that there’s erlang native adapter for event store, but one that is mentioned on website (available from bitbucket) has not been updated since last September and it’s stated that it’s not production safe (but README has been updated in 2013 last time).
Any experiences with it so far? Will http wrapper be safer way? There should be ~ 2M events/day and ~20 read models subscribed to them.
Thanks,
Milan
Based on our experience in the Ruby world, none of the “non-standard” adapters are anywhere near “production ready”. The .NET interface is production-ready, but the proper hooks don’t even exist in the HTTP api to write a “production ready” client (though some of them are starting to appear in the 3.1 release due out in the future).
There’s a lot of work underway to have a really solid Java client, and possibly you could lean on that. but yeah. Expect a lot of tool building if you want to use this in an Elixir environment.
the proper hooks don’t even exist in the HTTP api to write a “production ready” client
A large number of existing users disagree with you there, including the ones who built a production quality .NET client that uses HTTP underneath.
a really solid Java client, and possibly you could lean on that. but yeah. Expect a lot of tool building if you want to use this in an Elixir environment.
Really, did you even look at Elixir before posting? How exactly will a Java client help with a client on the BEAM VM?
There actually is a reasonably solid Erlang client for earlier versions of Event Store (I believe it supports the 2.0 protocol) written by a former member of the core team who now works at AWS - https://bitbucket.org/anakryiko/erles - though it’s not currently maintained. If there’s sufficient demand this is something we might consider maintaining though.
Just trying to be helpful. I guess I was mistaken about the runtime. And I stand corrected on the HTTP thing. Apparently you can use either HTTP or TCP on .NET in a “production-ready” way.
The prelude to that readme for Erles is, of course:
I’m very new to Erlang ecosystem and this is my first Erlang project, so if I’m doing something in a non-optimal, non-idiomatic or, even worse, plain wrong way, please, let me know and I’ll try to fix it.
And it doesn’t actually seem to support anything recent, so it would need a lot of tooling to get the Erlang client up to snuff. And then, you probably need to build a bunch of tooling yourself to use it in your Elixir app.
So it seems the takeaway remains, “You are not going to be able to pick something up off the shelf and use ES in a production environment.”
Cheers, and thanks for the helpfulness.
The erlang handles catch up subscriptions and it handles writing. 2m
events/day is not heavy load. It does need some updating but its not
much. I probably wouldn't use for for something serious but I might
give it a whirl (happy to help you with updating it for 3.0.5 btw). It
shouldn't be a huge amount of work (aside from things like competing
consumers but you don't really need them).
I am curious what the issues you have with the http api are? its been
reasonably stable without any drastic changes made to it.
Cheers,
Greg
Greg,
I’ve been having a lot of trouble with write forwarding, as you know (for months). James had suggested that I should not be doing write forwarding in general, and should try to write directly to the master node. But then, there is not a good way to find out what the master node is via HTTP according to James: https://github.com/EventStore/EventStore/issues/532
There is currently no good supported way of doing this over HTTP - all the TCP clients make use of the private API. Probably the best thing to do is add the state to the /info endpoint or something along those lines - I think there was a PR opened for this a while back.
The PR for it just got merged and it will be in the 3.1.0 release.
Reading that erles client, I see that they use the gossip endpoint: https://bitbucket.org/anakryiko/erles/src/e089f97997dd7a3b6540d94a1101f522258314d6/src/erles_conn.erl?at=default#cl-313
So I’m not really sure.
I’m looking forward to the 3.1.0 release, and think that with that we’ll be able to get a decent Ruby client finished up.
I just wanted to convey to the OP that my experience not been 100% rosy using something other than .NET, and I was probably more negative than I should have been, so I’m sorry if it seems I was bashing ES.
The underlying tech is great, and we’re using it as foundational technology for our company, and have been and will continue to be an advocate for it. That said, the experience of developing on top of it in the non-.NET world has not been all sunshine and rainbows. There remains a lot of work to be done to improve the DX in other ecosystems.
Justin
The .net client uses the gossip endpoint as well. This is generally
how clients get that information. That said I disagree with james on
this one and I think you should remain unaware of the cluster layout
(drop a load balancer in front and call it a day). The performance may
not be optimal (though its not bad and improving with a few issues on
github). For me its all about the simplicity of the deployment.
Thanks for the tip there. That makes a lot more sense to me as well.
The issues I mention on the list btw are
1) move to libuv/libev
2) forward over tcp not over http
#2 is more important than #1 my guess is a 3 node cluster with a lb in
front will hit > 25k events/sec writing then
Cheers,
Greg
I don’t want to hijack this thread much more, but can you link me to some docs or anything about forwarding/load balancing over TCP rather than HTTP? To be honest, I don’t understand what either #1 or #2 mean - at least in this context.
Thanks,
Justin
Thanks for answers. Even though 2M events/day is what we expect, that stands for part of system that should be proof of concept for event sourcing and event store. If everything succeeds some other parts of system should be rewritten but they have much greater traffic. That being said, I can’t rely on unmaintained client and maybe even http connections.
Are there any plans for supported erlang/elixir client in near future?
If we have demand. I don't mind coding in erlang; its a matter of time.
Right now we have exactly one user of the erlang client.
The http stuff is pretty simple to use and will hit a few thousand
transactions/second. As I said I would be happy to help on this I am
personally starting to play with elixir as well (see my recent post on
elixir list).
Cheers,
Greg
Greg, it’s great to hear that you are starting with elixir as well. That’s confirmation to me as well that I’m going right way after years of using C# and Ruby. If you think that erles library is good enough as a starting point I’ll invest more time in reading it’s source and trying to create elixir client.
Since I’ve never been doing anything similar, do you have any suggestions or links regarding ES protocol or anything else I need to know and understand in order to create elixir client?
Thanks,
Milan
Hi Greg,
We are working on new erlang/elixir app which should connect to EventStore. What I noticed that C# ClientAPI is using some protobuf types which are not defined in https://github.com/EventStore/EventStore/blob/oss-v3.0.5/src/Protos/ClientAPI/ClientMessageDtos.proto.
Currently, we didn’t do much, only write and read from stream and moved developing persistent subscriptions to event store. Could please let us know if ClientMessageDtos.proto is up to date. Also it would be good if you could share with us some paper which describes communications clinet <—> server on TCP level. Anyhow, some guidance would be helpful 
Thanks,
Milan Jaric
Which messages are you referring to the persistent subscription ones?
The same file on dev has those.
"Also it would be good if you could share with us some paper which
describes communications clinet <---> server on TCP level. Anyhow,
some guidance would be helpful :)"
I will write something longer up but basically its full duplex async
messaging. Request/responses are correlated using a correlation id.
Most of the message flows are pretty self explanatory (req/resp).
Subscriptions/persistent subscriptions will basically keep the same
correlation id as the create for all future delivered messages.
Hi Greg,
Looks like I was in wrong branch. I looked C# code from dev branch in github and saw couple proto messages which were defined in ClientMessage.cs file but not in DTO.proto file.
Thanks,
Mialn Jaric
Hi,
I just wanted to let you know that Elixir adapter for EventStore - Extreme is published to Hex. It’s used in staging 24/7 working environment for 3 months here and works great so far. Write, read one/many, delete soft/hard are tested as well as subscription is. It’s been working with single node but recently cluster support is added as well.
If you are interested, test it and let me know if you’ve found some bugs or you have some improvement ideas.
Thanks,
Milan
Great! will look probably later next week