we are currently using eventstore to store our business domain events. however, we also have data of a more raw transient nature. this data will put into message queues to move around the system. we may want to archive this data, but it’s unlikely to be looked at very often. would it make sense to use eventstore as the message queue, or would rabbit or something similar make more sense.
thanks,
Have you seen competing consumer support? Also for that type of data
you can explicitly tell it to be transient e.g. maxAge 5000 (in
seconds)
Cheers,
Greg
So. Just to be clear. Eventstore is a suitable platform for message queuing even where those messages are raw unprocessed data and not domain events. Latency and scale are also an issue
Sure you can even mark them as being transient and able to be
scavenged later. The competing consumer stuff works exactly like AMQP
(in fact there is even an AMQP adapter under way). In terms of
performance the initial numbers have been very good (faster than
catchup subscriptions). Here is some code I wrote for trying to
reproduce something the other day that can help you get started:
https://github.com/EventStore/EventStore/issues/771
class Program { private static string group = "foo"; private static
string stream = "bar"; static void Main(string[] args) { using (var c
= EventStoreConnection.Create(new IPEndPoint( IPAddress.Loopback,
1113))) { c.ConnectAsync().Wait(); //CreateCompetingGroup(c);
c.ConnectToPersistentSubscription(stream, @group, (sender,e) => {
//resolved event Console.WriteLine("Received " +
e.OriginalEventNumber); }, (sender, reason, exc) => {
Console.WriteLine("error " + reason); }); Thread.Sleep(500); for (var
i = 0; i < 20; i++) { c.AppendToStreamAsync(stream,
ExpectedVersion.Any, new EventData(Guid.NewGuid(), "foo", false, new
byte[100], new byte[100])).Wait(); } Thread.Sleep(5000); } } private
static void CreateCompetingGroup(IEventStoreConnection
eventStoreConnection) {
eventStoreConnection.CreatePersistentSubscriptionAsync(stream, @group,
PersistentSubscriptionSettings
.Create().ResolveLinkTos().StartFromBeginning(), new
UserCredentials("admin", "changeit")).Wait(); }
Ok. That's great. Thanks very much!
My reading of the documentation for competing consumers was that, unlike rabbit for example, cc doesn't guarantee ordering or at least once delivery to all consumers.
Have I misread?
That sounds fantastic for many applications! I couldn't find any
references to this in the docs, is there any for maxAge, or any
examples? Any other such settings that we should know about, and where
can we find out more about them?
cheers, Rickard
Competing consumers as a pattern does not assure ordering (its
actually impossible to insure ordering to multiple consumers where you
are load balancing between them).
If you want assured ordering to all consumers just use a catchup
subscription this is exactly what it does.