EventStore as a replacement for RabbitMQ in EventBus role

EventStoreDB
#1

I’m planning to use EventStore as an
EventBus.

Currently I’m using RabbitMQ for event
delivery.

Let’s consider a system in production
that operates for over one year, and we want to introduce new Saga.

This saga
would be responsibile for sending notifications to users and it’s important
that we send notifications not only

in response to events that happened after
saga introduction but also before.

General approach would be to reject
current saga state (remove state transition stream in ES) and create catch-up

subscriptions for the event types that saga is updated by. This however introduces problem with
ordering.

How should I join multiple streams with a
meaningful order ?

Current implementation handles only live
events sent through topic exchange in RabbitMQ.

On the code level i say “I
want for this saga to get events of type A and B(but only if B.someProperty ==
true)”.

I’m searching for effective method to recreate this functionality
with capability to recreate saga state.

Best regards,

Greg

#2

In event store you can have a global ordering (single replica set) subscribe to all does this for you. Look at the position (prepare/commit positions). These are also comparable.

https://github.com/EventStore/EventStore/blob/dev/src/EventStore.ClientAPI/Position.cs

You can compare positions between things to provide arbitrary ordering

#3

Btw in many scenarios like this if you are low load it’s actually simpler (though more expensive networking wise) to just maintain a subscription to all and drop the ones you are not interested in.

Btw have you seen competing consumer support for use cases like these?

#4

I haven’t seen an example yet, but from what I understand those will be perfect for event denormalization to Redis for example.
In my case I was thinking about enabling built in projection which filter events by types, and using those filtered streams to update Saga.

If I could use event position to sort those I think it would work.

Projections could also help (I think), but it would be difficult to manage those in different environments.

Have you considered (or maybe it’s already available) something like projection on demand ?

In RabbitMQ while subscribing to a topic, client provides all the details.

If a queue(or binding) is missing it will be created.

If saga could request to subscribe to output stream of a specific projection or create it if it’s missing it would be a lot easier.

I very much like the fact that i can use the position to keep all those subscriptions in proper order.

Have you encountered similar use case earlier or people use different approaches ?

Thanks for the help, I really appreciate it :slight_smile:

W dniu wtorek, 28 lipca 2015 13:30:26 UTC+2 użytkownik Greg Young napisał:

#5

You can get the same behaviour as rabbit using competing consumers
(server driven subscription)

#6

I’ve read http://docs.geteventstore.com/introduction/competing-consumers/
It’s an equivalent of queue with single binding as far as I understand.

I’m trying recreate queue with multiple bindings.

For example, lets assume we have streams A and B.

Stream C is formed as a result of joining streams A and B (order maintained)

I would want my saga to subscribe to stream C and have checkpoint like capability,

pretty much like with server driven subscriptions.

W dniu wtorek, 28 lipca 2015 16:34:31 UTC+2 użytkownik Greg Young napisał:

#7

Summing up there are 3 options if I’m correct:

  1. Every saga type would require custom projection which would join multiple streams into one.

This should work fine, but managing many evolving projections will get time consuming and possibly error prone.

  1. Use subscribe to all and ignore events that are not important. Fairly easy approach, but probably not very effective.

Could make sense if the event stream was shared between different saga types as more events would be used.

  1. Subscribe to individual streams and join them locally, but I haven’t figure out jet how to do it using JVM client.

Is there any other way ?

Best regards,

Greg

W dniu wtorek, 28 lipca 2015 18:13:56 UTC+2 użytkownik Grzegorz Duszyński napisał:

#8

"2. Use subscribe to all and ignore events that are not important.
Fairly easy approach, but probably not very effective.
Could make sense if the event stream was shared between different saga
types as more events would be used."

Yes the idea would be to have one listener for all your "sagas".

In terms of performance this is fine up to +- 10k events/second then
you should probably consider something else. As I mentioned this is
"code smart, network stupid" :slight_smile:

Greg

#9

Thank you very much Greg :slight_smile:
I’ll start hacking then.

W dniu środa, 29 lipca 2015 15:02:45 UTC+2 użytkownik Greg Young napisał: