Maximum size of streams

EventStoreDB
1

Hi

What is the maximum size of a stream in evenstore? Given small event sizes. I just whatched the video 'A deep look into the Event store' where Greg mentions that streams containing more than 2 billion events are going to be a problem. Is it possible to distribute a stream across multiple machines?

Thanks.

2

It’s a limit so no you can however use multiple streams

3

Thank you for the quick reply. Can you expand the answer a little bit? We are in the process of evaluating event store for our infrastructure. We are looking at storing lots of events produced over many years..

4

I’ve noticed the stream pointer is an int while the global positions are long, so I’m guessing this is by design?

/Peter

5

Yes. It is intended to have thousands or millions of streams not one stream for the system. As an example in a bank you would have a stream/account not one stream for the entire bank. In many systems the creation of a stream is an expensive operation in event store it is essentially free.

Cheers,

Greg

6

Is it still the limit now ?
Thanks.

7

Since version 4.1 (I think) stream version is a long so supports 2^63 -1 events. There are other limits you’ll reach before that.

8

Do we know where in the documentation these limitations are mentioned ?

9

They are not documented .
The theoretical upper bound of the total number of streams and events can be calculated though :

  • the absolute theoretical max is long.max ( 2^63 -1 )

  • the second max bound is the number of physical data file we can have : 1E6 files => ~2^50 max

    • in reality it is hard to physically achieve that: 1E6 files is about 245TB ; without counting the indexes

  • the real max has to do with the size of your events & stream names and is hard to calculate

    • you’re still in orders of magnitude bigger than billions though.

  • the limit on the number of events in a stream is int.max (2^31 -1 and that would be a terribly long stream )

1 Like