Maximum size of streams

Hi

What is the maximum size of a stream in evenstore? Given small event sizes. I just whatched the video 'A deep look into the Event store' where Greg mentions that streams containing more than 2 billion events are going to be a problem. Is it possible to distribute a stream across multiple machines?

Thanks.

It’s a limit so no you can however use multiple streams

Thank you for the quick reply. Can you expand the answer a little bit? We are in the process of evaluating event store for our infrastructure. We are looking at storing lots of events produced over many years..

I’ve noticed the stream pointer is an int while the global positions are long, so I’m guessing this is by design?

/Peter

Yes. It is intended to have thousands or millions of streams not one stream for the system. As an example in a bank you would have a stream/account not one stream for the entire bank. In many systems the creation of a stream is an expensive operation in event store it is essentially free.

Cheers,

Greg

Is it still the limit now ?
Thanks.

Since version 4.1 (I think) stream version is a long so supports 2^63 -1 events. There are other limits you’ll reach before that.

Do we know where in the documentation these limitations are mentioned ?

They are not documented .
The theoretical upper bound of the total number of streams and events can be calculated though :

  • the absolute theoretical max is long.max ( 2^63 -1 )

  • the second max bound is the number of physical data file we can have : 1E6 files => ~2^50 max

    • in reality it is hard to physically achieve that: 1E6 files is about 245TB ; without counting the indexes
  • the real max has to do with the size of your events & stream names and is hard to calculate

    • you’re still in orders of magnitude bigger than billions though.
  • the limit on the number of events in a stream is int.max (2^31 -1 and that would be a terribly long stream )

1 Like