How would one go around to create one stream with all events?

Let’s say I’m creating one stream per aggregate in my system. How would I go from there to get one stream with all the event? Do I have to create a projection that takes all the events I’m writing and link them to a new stream with all the events?

I talked too Greg about it at NDC and he said I could write to multiple stream, but I must have misunderstood what he meant since I got another answer on twitter from James Nuget where he said it isn’t possible.

There is the all stream that gives you all events /streams/$all or from all overloads in client

I think what I was looking for was the $ce-category stream. If I just put a category on all my events I could get them all from that stream it looks like. That is awesome!

To clarify - you categorise streams not events - you likely want the $all stream and to filter client side

Thats one way of doing such things what exactly are you trying to achieve?

We are working on a proof of concept of the the “people registry” (folkeregisteret) in Norway, where we want to show things that has happened as an atom feed. In the first version of the PoC I was thinking I could just show the feed from ES as it is quite simple. As of the moment I’m creating one stream per aggregate (person aggregate so far), as such, This is heavily inspired by the code you had put up us an example. Since I want to expose all the events as one stream and as well as one stream per person I thought a simple solution was to use categories. If I just prefix/postfix all the streams I create with something that would give me all the events in that stream as well as one stream per person.

So basically I write to streams named: folke-aggregate type-unique id. After updating the category projection to take the first word as a category I know get one stream called $ce-folke that contains all the events that I think should go into the “all” feed and I don’t have to do any client side processing.

Am I doing something totally wrong? If you look through the code I know I might be doing something wrong since I’m using signalr to update my live feed, which I also have, when I could just consume the feed from the web client directly using long polling as Greg suggested to me at NDC.

@James, I meant category on the streams :).

How many messages/second are we discussing? My guess is low in which case a projection is just fine to aggregate them

I think it is quite low as well. Norway is not a big country. I think it is around 2.000.000 changes through a year, so not that much.

So you recommend doing a projection instead of using a category? Isn’t that almost the same as using a category, I mean, a category is a built in projection that you provide? That way I don’t have to write the projection myself :slight_smile:

the category is a projection :slight_smile: I would just write a custom one.

Ok, sounds reasonable. So I’ll try that, I think I’ll add some extra metadata to the events I want in my “all” stream to make the filtering easier, then I can just listen to all events and emit or link those events with the metadata to the my “all” stream

Is there a way to create projections from code, or to map the projections to a git source for versioning? I’m not a big fan of going into a product and write some thing without version control, I’ve the same issue with stored procedures in sql.