streams projection output


When should I expect to see some output from the ‘streams’ projection? It has been running for a couple of hours and is at about 20%…


It’s at 100 % now but I can’t seem to get any output from it… I must be missing something… I should be able to read the output from ‘projection/$streams/state’ or ‘projection/$streams/result’?

$streams does not keep state nor produce a result it writes link to
events (its an indexing projection).

Source is here
it writes to $streams

It’s an index over all streams right?

Not sure what you expect "an index over all streams" to mean

The logic of it is right

It outputs the first event in a stream to a stream named $streams

Ok, first event. All I need is a list of all streams… We’re investigating why we’re using so much disc space at the moment…
All streams should have a maxage of 2 days, but scavenge doesn’t seem to release much space…

Doing a

.\curl.exe -i -H “Accept:application/vnd.eventstore.atom+json” ‘$streams

Results in

‘HTTP/1.1 408 Server was unable to handle request in time’

Thats my experience as well.

Did a migration to a new eventstore yersterday. Did not migrate all the events (dropped maybe 30%, will work out the exact numbers later today), and havent started any projections in the new one, but went from over 70gb to 5gb. Just doesn’t add upp for me… (there are maxage on stats, and scavenge has been run)


Scavenging doesn’t remove space, if anything it adds space. You would have to delete the old chunks yourself.

Missed that, will have a look…


That’s news to me! How do you identify the chunks that can be deleted?

there exists a file with the same name and a smaller number in the extension. And actually I am not sure if you no longer need that file, @greg or @jen20 should weigh in :slight_smile:

We dont’ have any with the same name… e.g.: we have chunk-000162.000001 but no chunk-000162.000000…

My mistake then, looks like the obsolete chunk gets deleted

Ok, no problem. What seems to be the problem in our case then, is that scavenge is not effectively freeing any space… Are there some special cases where events > maxage are not removed from disc?

(This is eventstore 3.0.1 by the way…)

Scavenging removes the old chunks after creating the new ones.

no. You should be able to look at the stream and see whats gone btw
(it won't show up)