Is one stream per user a good idea?

EventStoreDB
1

Hi guys, please excuse if my question is too trivial. I’ve only been dealing with Event Sourcing for a day and I’m trying to understand it in the context of EventStoreDb.

In this post by Greg Young it is mentioned that EventStoreDb is designed to handle millions of streams.

I only have experience with traditional message brokers where I maintain one queue per topic and process the messages where are very few queues. Now for EventStoreDB: Let’s say I have an application that creates continuous financial statistics for its users. The user makes an input about a business transaction (emits Event to EventStore), these events would all be stored in a stream (let’s call it “business-activities”).

Since the cost of a stream (see post) is very low. Wouldn’t it make sense to have one stream per user (e.g. “user-business-activities-1235”)? Or does it make sense or should I keep the main “business-activities” stream, get a user_id from the event payload and read all specific user events for further processing?

My thoughts:

Pro for main stream:

  • Ease of use
  • Easy migration by replaying events from the event log to populate the new readmodel tables.

Con for main stream:

  • With large numbers of users, the sum of events in the stream increases enormously

Pro for user stream:

  • Uncluttered and small sum of events per stream

Con for user stream:

  • Increased complexity
  • Migrations to a new readmodel requires knowledge about the individual user_ids to iterate over the respective streams.

Thanks a lot!

2

If I was building this, I would have a stream per user, then use a projection (either the built in javascript) or external to create the stats.
For the read model, it’s still easy to rebuild because you can access the many users stream either by:

$all
$ce-users

your financial stats stream

The only time I would consider a “big” stream is if the users had business rules tied to each other.
So a simple example might:

  1. Only accept the user stats if we have received less than 10 entries today

Assuming the streams do not need to know about each other, it points to a stream per user.
On top of that, if you want to delete users (or their stats) it becomes much easier.
Perhaps you only need to keep the last days stats for a user. A stream per user makes this very simple.

Hope that helps.

1 Like
3

If you’re going to read all the activities of a single user from time to time (event store style), you need a stream per user. If you’re only producing events to consume them (broker style), you can use one stream.

Remember if you have a single stream, you’d need to set some retention period on it as you won’t be able to delete events for a single user if, say, you get a GDPR “forget me” request. With a single stream per user it’s trivial and you can keep the events as soon as you are allowed (as the Data Processor).

1 Like