EventStore Reliability

EventStoreDB
1

My company has implemented a project using event sourcing mainly for replication.
Events are PULLED from 80 different locations by replication system running on a central registry where read models were built.

We have used NEventStore on top of MS SQL Server for event store databases.

Now we are about to implement much larger project which would store between 500M and 1B of aggregates / streams (up to 50 events per stream in 5 years).

We find that EventStore has many features that would help us a lot to solve some problems:

1.) Classify events by using linkTo and creating streams for each microservice / bounded context

2.) Classify events by using linkTo and creating streams for each event type

3.) Have one stream for all events (using linkTo)

4.) Subscribe (permanent subscription) to streams interested by microservice (all, bounded context or event type)

5.) Remove service bus for integration (except for some other messages that are out of scope of this problem)

We also considering option where all microservices would use the same EventStore DB.

This project is mission criticalt, technology must be reliable, performant and must allow regular back-ups.

Can EventStore satisfy our requirements?

2

3.) Have one stream for all events (using linkTo)

There is a separate option to ReadFromAll as opposed to using linkTos here which is likely a better option no?

3

Thanks Greg!

I know that there is an option to read all events from the store.

We are thinking to have aggregate streams named as BoundedContext[aggregate guid], eg. CitizenEnrollment[d0f5593f-b010-4b7a-919e-c776f012c87d], where CitizenEnrollment is microservice/bounded context and d0f5593f-b010-4b7a-919e-c776f012c87d is an aggregate ID.

Event types would be something like: PersonEnrolled, BirthCertificatePrinted etc., and in EventStore they would be saved as BoundedContext-EventType (e.g. CitizenEnrollment-PersonEnrolled).

Then we would have only one projection, something like:

fromAll().when({

$any: function(state, event) {

if (event.streamId.indexOf("[") !== -1 && event.streamId.indexOf("]") !== -1) {

linkTo(“all”, event);

linkTo(event.streamId.substring(0, event.streamId.indexOf("[")), event);

linkTo(event.eventType, event);

}

}

});

After that it would be possible to subscribe to:

(i) all to all events,

****(ii) CitizenEnrollment for citizen enrollment bounded context events,

(iii) CitizenEnrollment-PersonEnrolled for events of specific type.

Similarly we could extend this to count on aggregate type, e.g. CitizenEnrollment-Citizen-PersonEnrolled.

The main question is how this could perform for a whole country (80M citizens with about 5 - 7 aggregate types).

4

this one

linkTo(“all”, event);

isnt needed. Doing another write for every write seems silly :slight_smile: there are already all streams (readfromall with tcp and $all on http/atom)

5

Thanks Greg, I agree :).

We need to find a better solution to integrate microservices using events, and subscriptions on EventStore are perfect fit since they keep and handle checkpoints for every group.

Using service bus only we had problems to extend application with new bounded contexts; loading history required much work and we needed to pull events, manage checkpoints etc.

Additionally, we are thinking to build a new service that would POST events to a specific API: microservice subscribes to a stream through this service, sets HTTP endpoint for PUT/POST, and then all events are POSTed over HTTP REST service which returns ACK after they’ve been handled.

This would remove need for explicit ES HTTP API calls from microservice, since this new service would POST events on its behalf.