Create a projection which takes the last event from every other stream in the category

EventStoreDB
#1

I have a series of streams in a category with ids like:

category-instanceid

the events in the stream are all of the same type. what I want is a way to get the last event of every instance stream in the category

I can project them into a stream which has every event based on the category, then read all of those events and just take the last one for each instance id in my aggregate, but I want to know if there is a way I can make my projection only take the last event in each instance stream to save me loading all events.

Any thoughts?

#2

If you use the $stream_by_category projection, you can get the list of streams since that keeps the first event (not the last, unfortunately), but you can go through the list of streams and pull out the last event from each of those. Does that make sense?

#3

Define ‘last’ event. What happens after you create your projection, then append an event to one of these streams? You might be better off using the client api and ReadStreamBackwards.

#4

What I would like is a stream which always contains the last event from each stream in the category. So when I add a new event to one of the streams in the category then when I next read the projected stream it has been updated to contain the new event in place of which ever event was in the projection from that stream previously.

Obviously that’s only my explanation of one implementation, but the main goal is to be able to get the last event in every instance stream in the category without:

a. Reading every event in the category (this is the naive implementation I currently have)

b. having to iterate each instance stream in the category and taking the last (Titus made this suggestion but this seems like a select n+1 issue, which ideally I want to avoid)

c. Have a subscriber to the event stream which creates a new stream with the exact data I want (I would rather have this happen via a projection if possible)

Thanks!

#5

then when I next read the projected stream it has been updated

Here’s where you’re going wrong. There is no such thing as ‘updating’ a stream, you can only append to it. however you might be able to get this done by leveraging $maxAge. However I have no idea if you can set streamMetadata via a projection.

#6

The problem with creating a projection that contains only the last event in all related streams is that it would require deleting earlier events in that projection as new events come in from the related streams. There’s simply no way around it.

#7

This sounds like a perfect place to use a read model.