Long time no see, indeed 
I actually have a couple of event sourced systems running in production today. And more to come - most likely. Much of it based on what I learned from you earlier, Alexey
It’s basically built on Scylla and Kafka. Storing all events in Scylla and publishing it to Kafka. This is risky, I know. Since this operation is not transactional. Which is one of the reasons why I want to migrate to EventStore. And now that EventStore is available as cloud solution in AWS - this is really getting interesting for us. Even though the price is a bit steep 
So right now I’m doing some research by refactoring one of these systems to EventStore. In the same process I’m looking into the projection side as well (infrastructure, event processing / projection management).
This is a system which is handling orders; placement of orders, accepting orders, closing orders, modifications of orders, cancellations etc. So we have Order as the aggregate root, very simplified - let’s say it looks something like this: Order(id, orderLines) where an order line maps to a specific product. And a couple of the projections that we have:
- OrderProjection - which is more or less a reflection of the aggregate root (current state)
- ProductTurnoverProjection - which keeps track of the accumulated turnover for each product.
Very high level how the projection management has been designed:
Projectionist(projectors):
onEvent:
projectors.forEach:
fetch the projector's current position
if the event position matches the expected position:
projector.project
projector.updateCheckpoint
Projector:
project:
fetch document to update (can be multiple documents to update pr event - but each document is updated sequentially using optimistic concurrency; each document is versioned)
update document (if matches expected document version)
This has worked really well. But I see a big issue that the Projectionist can use ~200ms pr event. When we are using a partitioned kafka topic the projection code can scale - and this works for us at the moment. But I really need to address the projection management handling, and perhaps the infrastructure we use read side. Which is Scylla. I assume it get’s troublesome because of lots of read/write pr event. Which kind of is required since it does not support transactions. We are also pushing OrderProjection (current state) into Elastic - so that we are able to do more querying.
Another side note to further complicate:
- The ProductTurnoverProjector depends on getting the current state from the OrderProjection (which then is required to have projected the incoming event beforehand). If this was not the case all the projectors could run in parallell.
So I’m a bit curious if you have any input to this flow, databases suitable for read side. I believe you’re a fan of RavenDB? Which is both queryable and supports transactions. But this is quite expensive, right? Also it might be that we would be better off designing our projections differently.
I saw you mentioning in another post here that you have a event sourced subscrition handling ~1M events per hour. That’s quite impressive! Can’t be much interaction with infrastructure in the event handling here?
This post got both a bit long, and a bit off topic But any input is appreciated 