We have an EventStoreDB running that’s taking up 100GB, and we didn’t understand where that data is going, so we wrote a small Python tool to scan the
$all stream and count up events per stream. A $projections-ProjectionName-order stream is taking up one of the top spots, even though the projection itself is truncated.
I’ve performed the following reproduction steps:
- start a new EventStoreDB (5.0.11 in my test)
- Create a projection that links two streams, Test1 and Test2, into a stream Projection
- Dump 5000 events in Test1 and Test2
- Observe a lot of $projections-Projection-order events being created detailing the offsets in Test1 and Test2 for every event in Projection.
- Set a $maxCount of 1000 in Projection
- Dump events into a filler stream until the chunk is written
After these steps, only 1000 events from Projection remain in $all, but all 10000 events from $projections-Projection-order are still present.
What are we doing wrong? Also: how do we clean those up too?