hi,
we are having some issues with $streams projection. For some reason we can get specific stream by its full name, but when we try to find it in all $streams ( by calling ReadStreamEventsBackwardAsync("$streams", …)), we can not find it. Maybe someone has experienced this issue?
Event Store version 3.0.5.0
Greta Mikneviciute
Adform
To clarify you mean calling
ReadStreamEventsBackwardAsync("$streams", ....))
in a loop? There are likely tens of thousands to millions of events in
that stream.
Yes,loop. there are >500’000 events. We load all of them and try to find this specific steam and we cant
Is this stream recently created or very old?
actually there is not one steam,there are more than 100… most of them created 2016-06 and later. Last events differ, some of them are from today, others from few months ago
You only get new records in $streams when new streams are created.
I can come up with a few theories.
One thing to try is looking at the actual link events instead of
resolving them (resolve linktos = false). Inside of the link events
that the $streams stream holds you will see a body of the form
0@my-stream (this is a link). This will give you the original links as
opposed to resolving those links to be the events they were.
There are a few reasons why there might be a link but it does not get
resolved. The first is that the event has been deleted (either by
setting $tb (soft delete) or by using $maxage/$maxcount).
It could also be some bug in projections but it would take me a while
to look through 1.5 years of commits to see if there is something
relevant.
Providing its not the first thing, an interesting experiment would be
to re-run the projection on the latest version (you can rebuild
projections at any time) and to see if the issue remained.
Cheers,
Greg
hi
I tried to retrieve events from $streams with resolve linktos = false and i found them, streams and events are not deleted and streams are not huge ( less than 200 events) and i can’t find any events from those streams.
I guess we will try to upgrade our ES to latest version and check then 
"I tried to retrieve events from $streams with resolve linktos = false
and i found them, streams and events are not deleted and streams are
not huge ( less than 200 events) and i can't find any events from
those streams."
You found them? Then you should be able to read them. They likely are
formatted something like 0@foo ... Can you go look at event 0 in foo?
Non of these streams have 0@ event, all of them starts from 1st or higher. These streams might be deleted half a year ago, scavenge ran and now new streams with same ID created. Maybe this is related?
That is why they are not being found in $streams. $streams has a link
to the first event in the stream. If they have been soft deleted they
will not resolve. In this case you are better off using resolvelinktos
= false and use the raw link data than having it resolve.
Cheers,
Greg
Okey,got it, thanks. Never thought about that
We are using this projection to get all streams and stream types, maybe you have suggestions how to solve that? Create our own projection or there is some different solution?
So what this projection does is find any event with a 0 version and
link it. This effectively gets you streams. You could write your own
projection that does this as well (writing out a new event say
"streamcreated". Or you can just read $streams with resolvelinktos =
false (as you are now) then just grab the stream name.
Hi,
thanks, it worked. Before i didn’t realized that i can get info from byte array in Event.Data.