Hi
When should I expect to see some output from the ‘streams’ projection? It has been running for a couple of hours and is at about 20%…
Thanks.
It’s at 100 % now but I can’t seem to get any output from it… I must be missing something… I should be able to read the output from ‘projection/$streams/state’ or ‘projection/$streams/result’?
$streams does not keep state nor produce a result it writes link to
events (its an indexing projection).
Source is here https://github.com/EventStore/EventStore/blob/dev/src/EventStore.Projections.Core/Standard/IndexStreams.cs
it writes to $streams
It’s an index over all streams right?
Not sure what you expect "an index over all streams" to mean
The logic of it is right
https://github.com/EventStore/EventStore/blob/dev/src/EventStore.Projections.Core/Standard/IndexStreams.cs#L57
It outputs the first event in a stream to a stream named $streams
Ok, first event. All I need is a list of all streams… We’re investigating why we’re using so much disc space at the moment…
All streams should have a maxage of 2 days, but scavenge doesn’t seem to release much space…
Doing a
.\curl.exe -i -H “Accept:application/vnd.eventstore.atom+json” ‘http://192.168.5.129:2114/streams/$streams’
Results in
‘HTTP/1.1 408 Server was unable to handle request in time’
Thats my experience as well.
Did a migration to a new eventstore yersterday. Did not migrate all the events (dropped maybe 30%, will work out the exact numbers later today), and havent started any projections in the new one, but went from over 70gb to 5gb. Just doesn’t add upp for me… (there are maxage on stats, and scavenge has been run)
/Peter
Scavenging doesn’t remove space, if anything it adds space. You would have to delete the old chunks yourself.
Missed that, will have a look…
/Peter
That’s news to me! How do you identify the chunks that can be deleted?
there exists a file with the same name and a smaller number in the extension. And actually I am not sure if you no longer need that file, @greg or @jen20 should weigh in 
We dont’ have any with the same name… e.g.: we have chunk-000162.000001 but no chunk-000162.000000…
My mistake then, looks like the obsolete chunk gets deleted
Ok, no problem. What seems to be the problem in our case then, is that scavenge is not effectively freeing any space… Are there some special cases where events > maxage are not removed from disc?
…
(This is eventstore 3.0.1 by the way…)
Scavenging removes the old chunks after creating the new ones.
no. You should be able to look at the stream and see whats gone btw
(it won't show up)