Linking to multiple streams

EventStoreDB
#1

Hi,

If I have 10 streams called:

typeofstream-1

typeofstream-2

typeofstream-3

…

typeofstream-10

And I have a projection that processes an event of a certain type, can I issue a linkTo call that puts the event in all streams of a certain category?

Thanks

#2

If you looped yourself yes but this would be an expensive projection (each event of type x would write n events) so the write amplification in this case would be 10-1. What are you trying to accomplish here? What are the message loads?

#3

The message loads are small right now but they won’t be for long. Sounds like it won’t work.

The scenario is partitioning messages for different users. A users join the system, a stream is set up for all types of event that the user might be interested in. This particular type of event would be considered as a system wide message that all users are interested in, therefore the event should be placed on all user streams.

Make sense? Is there a better way to achieve this or does it start to bend GES in a way it was not designed?

Thanks

#4

How many event types are there?

#5

No more than 10 currently. And there is only 1 event type that should it be sent to GES would need to be linked to all user category streams.

#6

Why not just have the users read a stream with those events in it?

#7

I want the order in which all these events were raised to be used. So a system wide message could appear in the middle of a list of other even types/messages. So the order of the stream is important and ideally I would like to read from a single stream that represents a particular user.

#8

You could do the equivalent of fromstreams() and join two streams.

How many events/sec are being discussed?

#9

It will be really low for now. Would that merge taking into account the date each event was raised to create an ordered stream?

How might the projection(s) look?

#10

I mean to do the equivalent from the outside (we have considered exposing fromstreams to the outside). Basically fromstreams just does a join of n streams.

If your message volume is low writing a link is no problem but it does not scale well imagine 1000 users where you have each event written 1000 times)

#11

From the outside it would be perfect. I’d have a stream per user and then I could join on the system wide stream too. Messages in the system wide stream would be minimal.

Is there scope to expose this sort of thing?

#12

You can also do it yourself. Listen to two streams with a slight delay to allow reordering. You can reorder the vents into correct order.

#13

Tis true :slight_smile:

Thanks, food for thought.

#14

I’ve a slightly more complex variant of this now that would make polling multiple streams externally a bit of none goer. Unless there is another way of looking at the problem…

Given a set of users, they can post messages to a central location, call it a wall for now. In posting a message they effectively join that wall and are interested in seeing further updates from it. If I look at the stream for the wall I should be able to see all interaction i.e. the messages from different users. This should work fine.

As a user I can view a separate stream of my own activity (again, this should be fine), plus I can see another view that aggregates all my interactions and other people’s interaction across the walls I am interested in. This is where it falls down.

So on receiving message events into the system I can linkto new streams of:

wall-[id]

user-[id]

But then I need to produce other streams that represent a view for each user, including messages from other users, which we made on walls the base user is interested in.

I’m thinking of these things as a stack of items, which is wrong. It feels like there should be a solution but I may be looking to bend GES in a direction it was not designed for. Please say so if this is the case!

Thanks