Event Store and RabbitMQ

EventStoreDB
#1

Hi,

We are considering Event Store for our microservices platform.

We use the cqrs pattern for our services where we use RabbitMQ for messaging (command and events). RabbitMQ takes care of the commands and events, pub-sub, etc.

However when reading on Event Store it looks like RabbitMQ could become obsolete for processing events and pub-sub?

We would still need messaging to send commands?

I found some posts on RabbitMQ, however nothing really in depth so far. Anyone any experience or thoughts on this?

Thanks!

Cheers,

Peter

#2

IMO you wouldn’t really need both here. For example you can make one or more streams that represent command queues, then use the persistent connection feature to make sure unhandled commands don’t disappear.

#3

You could use event store internally for your event sourcing then publish external domain events via rabbit. Not every service may use event store

#4

So RabbitMQ is usually used to write to EventStore, then EventStore persists the Events, and publishes to Subscriptions for things like ReadModel generation.
You can also publish these events back to RabbitMQ, which is useful if you use things like Process Managers / Sagas in NServiceBus.

#5

RabbitMQ is usually used to write to EventStore

This is not true, just adds another moving piece, increasing latency and complexity

#6

Agreed that RabbitMQ is not needed.

You can use the EventStore concept of competing consumers to listen to and process events that are added to EventStore, thereby not needing the Queueing functionality of RabbitMQ.

#7

I’d chuck that both are used into the mix. EventStore within a bounded context. Rabbit as a domain event publishing system.

That way AMQP can be used outside the boundaries for systems that are consuming outbound events and messages, but not necessarily eventsourcing

#8

The key difference is that EventStore is a database.

#9

Yup agreed that’s the correct way to look at it

#10

Event Store can make message brokers obsolete. Look at the feature set for your message broker to see if it provides anything that you can’t model in the event store. RabbitMQ is not as compelling to use with Event Store as MQTT, for example.

You want a message broker if:

  • you need services talking to each other directly

  • you need features like mqtt’s QoS

The price for using or not using a message broker is complexity on way or another. I personally find it easier to pass all events in the domain into the event store rather than have to manage who is talking to who.

The reason is once you’re in Event Store, you can use the projections API for handling events and can see your changes easily during dev-time. You’re also in memory at the db level so you have a large amount of flexibility in creating inexpensive (and performant) complex event handling.

The need for pubsub can come later in your external read models.

#11

Its important to note that you generally would never want to use something like rabbitmq for read models. There is nothing wrong with rabbitmq here but the needs are different.

Read models generally have a requirement of in order delivery which is provided by catchupsubscriptions. They assure 100% in order delivery, this is done for simplicity purposes, handling out of order messages gets … complicated. You will not get this assurance from most queueing systems and its a real PITA to try to code around! There are however two even larger secondary problems with using something like rabbitmq for projections to read models!

The first issue which is rarely thought of is quite simple … How do you do … a replay? You should be replaying projections a lot. Any change to one would require a full replay… no?! I often give people the advice of replay all on every release (until it becomes prohibitively expensive!). This process is convoluted by having a queue in the middle yet is trivial if using a catchupsubscription! With a catchupsubscription you just say start at Position.Start and continue until … live … what level of coordination would you need on this with say rabbit?

The second issue is that very often there is a fan out here. There might be 14 consumers associated each at different locations in $all (its common to have even multiple per read model!) … How precisely will you model this with say rabbitmq? I guess there could be routing to varying queues but this quickly becomes nasty to deal with!

Keep in mind as well that event store currently also supports competing consumers which is a similar model to rabbit. When there are actions that you want to be taking (say emailing when event E occurs) this is another model that can be used without requiring bringing in further infrastructure.

Cheers,

Greg