For us, separating streams has been VERY handy in the past, when the ES was corrupted. Instead of the whole ES dying, only that stream died. Since it was physically separated by file/db, it could be rebuilt without side affects to other ES’s.
Maybe GetEventStore will never corrupt, but if it does, I fear the whole system throwing poison, instead of just one stream.
Storing into separated physical backing stores would help me sleep at night.
Thoughts?
@ScottCate
602-418-0770
You want a separate file set per stream?
I’m just thinking about this with a use case we have spent some time on. 100,000,000 streams 
I don’t necessarily “want” anything, just assessing risk.
In the past we’ve had a few ES corruptions. Since they were singled out in storage, there was not side affects to other Streams.
Maybe it’s not forced, but an option? With an optional storage config, we could easily separate streams by customer, product, etc. Then it would be up to us, instead of just having a single storage, or being forced to setup a whole second system.
Also, contractually, some of us can’t mix data from customer to customer.
Thanks for your thoughts.
@ScottCate
602-418-0770
I can absolutely see the value in separating files for customers (there is a thread now about best ways to handle this). We would love to hear your input on it.
For doing a file store per stream. There are many issues with this. As of now in a single vnode group as an example we can give absolute ordering ($all stream). This would become impossible. On separated customer storage it would be per customer which is I think perfectly what people would expect. Another difficulty would be making the writes consistent (though not impossible)
There is also the problem of indexing. Putting each stream in a file would be good if the only query were read all events in stream. If there is an index there would then either be an index set per stream and no cross stream queries which is a bit difficult or we have moved the corruption problem from the stream to the index (if index is corrupt we have troubles with many streams).
In the case of a corrupted file (we don’t see very many bugs in the files as we are writing but it can still happen due to things like sectors going bad on hard disks). We currently use hashing to at the least identify the hd issue. If there is an issue there is an internal tool (there has been talk about moving it to the OSS side) that is capable of looking inside of a given chunk to figure out what may be wrong and very often recover from it (or at least the best we can, only so much you can recover from a bad sector on the disk through software)
That said in the multinode version you have n copies of the data on disk so the main worry would be a bug that corrupted the file on disk on all of the nodes. While not impossible by any means it’s rather unlikely to happen and I believe a risk in any system (we might just move it around as to where it is). I believe for many of these kinds of issues even you will find today that they would affect only the stream that was corrupted. The only case it could affect many is if the file were unreadable. This is risk is even being mitigated by incremental hashing (where only a few records would be corrupted not all). This might be an interesting thing to blog about actually…
Cheers
Greg