Generating report-like output from EventStore

EventStoreDB
#1

I am using
EventStore to store events that affect account holdings for a business that
invests for their clients in equities. A
holding is identified by the combination of account id, security id, and
holding type (long or short). Currently,
the events for each holding (e.g. account1 bought x shares, account1 sold x
shares, etc…) are held in separate streams.
A projection calculates the current quantity of shares for any of the holdings. Each account may have several hundred
holdings.

I need to
provide in a report and/or user interface grid which displays all the current
holdings of any of the accounts. A user
would specify the account and then submit the request. I’m able to generate a table like this from
the client by repeatedly invoking my projection for each of the holdings in the
account using WebClient().DownloadString(projectionUrl). However, the time required to generate this
result is longer than I can expect a user having just clicked a button to
wait.

I’d like to
solve this problem by using the capabilities of EventStore, if possible, but so
far I haven’t been able to come up with anything. Are there any strategies for creating this
sort of result that have worked for others?
Are there any examples of this I could see?

Thanks,

David
Madrian

#2

My initial instinct reading your post is that this would be better served by a specialized read model?

#3

Being pretty new to EventStore, I don’t know what a specialized real model is. (Did some searching on-line, but didn’t turn up anything.) Can you tell me more about this, or refer me to any existing information about specialized real models?

#4

user interface grid
specialized real model

SQL maybe?

#5

Basically setting up a catch up subscription to a bit of code that
receives the events and writes them to some other model (say a sql db)
that you then use for your queries.
https://www.youtube.com/watch?v=GbM1ghLeweU explains more

#6

Thanks for the link to the video Greg. I watched it a couple of times and feel it really helps me better understand the potential to benefits of polyglot data for the query capabilities of a system like the one I’m working on.

I do still have a question about how to proceed with building the SQL view model. I already have an EventStore projection that generates current holdings from underlying streams of domain events generated from our holding aggregate. The holding aggregate generates additional events from the incoming ones in order to perform the accounting (e.g. offsetting cash transactions, etc…) for these holdings specified by our business requirements. These domain events underlying my current holdings projection should not be replayed but are all derivable from the incoming holding events.

Here’s are my questions:

  • Generating the read model: In order to generate this SQL view model, should I ignore the projection with its underlying streams of domain events and instead generate that SQL read model directly from the incoming events (or by replaying the events previously received)? Because this view model needs to show both current holdings and holdings as-of any previous point in time, I think it may work better for me to generate this from the replayable events rather than from the projection and streams of domain events generated from the incoming holding events, but I’m not sure.
  • Generating as-of date read models/Utilizing snapshots: Assuming the right course of action is to generate the SQL real model directly from the replayable events, are there any examples or guidance for how to do this? I would want to generate holdings as-of any time and be able to take advantage of the snapshots made by the EventStore to help with performance once we have a lot of events. The API has support for playing messages up to a certain
  • API options: How should I go about deciding whether to use the EventStore .NET API or the HTTP API? Our organization is pretty much a .NET/Microsoft environment. If I’m just starting development now, should I choose the .NET API based just on our current technology focus, or are there other factors to consider?
  • Stream Design: Each of our current streams of domain events represents a single position (identified by accountId, SecurityId, holding Type (long/short). This results in potentially 70K+ streams, with probably less than 5-10 events added to the stream per day. On the other extreme, I could just drop all the replayable events into a single stream. Maybe there’s no simple answer to this question, but I’m trying to make sure I’m aware of the most important factors I should be considering when making this sort of design decision about the structure of the streams.
    I probably can’t expect detailed answers to all these questions, but would be happy to get any help I can.

Thanks,

David