Stream missing from $streams stream

We are using eventstore 2.0.1, and we are facing an issue where the $streams stream is missing one of the streams.

I’ve reset the projection and re-run it, but the stream in question hasn’t appeared.

I know that one of the answers will be “upgrade to the latest and see if it works”, however I’m loath to do this unless i know it will work, and also I cant find any instructions on how to ugprade an instance.

Any ideas on how I can debug this?

thanks,

Matt

So "upgrading and instance" is putting new code on the old data. If
running other projections (eg js) use the projections upgrade tool
(stop them first)

Easiest way to test. Bring down node. Copy your data folder. Copy new
binaries over old (or use a different directory). Startup new system.

Cheers,

Greg

/s/and/an

Okay, thanks, I will try that.

As there are breaking changes, an upgrade will be a bit painful, so I'd like to explore other options as well.

Is this an issue that you've come across before?
Do you have any suggesions on how to debug?

thanks,
Matt

Hi

I've copied it to a different server, stopped the projections, upgraded to 3.0.1, and re-enabled the projections.
(I didn't use the projections upgrade tool that you mentioned, as it said it was designed for 1.x->2.x)

I then deleted the $streams stream.

Issue 1: after deletion, the stream re-appeared in the browser without me doing anything. Maybe a caching issue?

I then re-ran the $streams projection.

Issue 2: I now get a log full of:
[PID:03908:015 2014.11.12 17:04:11.038 INFO ProcessingStrategySe] Projection '$streams'(22016ae4-bb1c-4a50-988b-f20904873425) restart has been requested due to: 'The '$streams' stream has be written to from the outside'
[PID:03908:015 2014.11.12 17:04:11.116 INFO CoreProjectionCheckp] Failed to write events to stream $streams. Error: WrongExpectedVersion

Issue 3: I also got an error on loading eventstore:
[PID:03908:013 2014.11.12 16:39:43.813 ERROR QueuedHandlerMRES ] Error while processing message EventStore.Core.Messages.ClientMessage+ReadStreamEventsBackwardCompleted in queued handler 'Projections Master'.
System.Exception: The projection cannot be loaded as stopped as it was stored in the old format. Update the projection query text to force prepare
   at EventStore.Projections.Core.Services.Management.ManagedProjection.CreateBeginCreatePreparedMessage(ProjectionConfig config) in c:\EventStore\src\EventStore.Projections.Core\Services\Management\ManagedProjection.cs:line 803
   at EventStore.Projections.Core.Services.Management.ManagedProjection.PrepareWriteStartOrLoadStopped() in c:\EventStore\src\EventStore.Projections.Core\Services\Management\ManagedProjection.cs:line 985
   at EventStore.Projections.Core.Services.Management.ManagedProjection.LoadCompleted(ReadStreamEventsBackwardCompleted completed) in c:\EventStore\src\EventStore.Projections.Core\Services\Management\ManagedProjection.cs:line 594
   at EventStore.Core.Messaging.RequestResponseDispatcher`2.Handle(TResponse message) in c:\EventStore\src\EventStore.Core\Messaging\RequestResponseDispatcher.cs:line 64
   at EventStore.Core.Bus.MessageHandler`1.TryHandle(Message message) in c:\EventStore\src\EventStore.Core\Bus\MessageHandler.cs:line 33
   at EventStore.Core.Bus.InMemoryBus.Publish(Message message) in c:\EventStore\src\EventStore.Core\Bus\InMemoryBus.cs:line 324
   at EventStore.Core.Bus.QueuedHandlerMRES.ReadFromQueue(Object o) in c:\EventStore\src\EventStore.Core\Bus\QueuedHandlerMRES.cs:line 121

Which, I must admit, was happening on the old server too. How do I find out what projection this was talking about?

Issue 4: while running the $streams projection, the events/sec field goes negative occasionally: from 0.0 to 822.7 to 177.6 to -871.1... How can it process negative numbers of events per second?

Issue 5: The stats table column widths jump around every time the page updates.

Issue 6: the projection buttons are all different sizes.

Issue 7: I think because of issue 2, the projection is not working - its alternating between -1.0% done and 1.6% done

I don't really care about any of the other issues - I just want my $streams projection to contain all streams.
Any suggestions on how to debug?

thanks,
Matt

Hi

Any suggestions on how to continue with this?

If I can't trust $streams to return the all streams, that means there is no way to know what streams are available.

Regards,
Matt

How are you measuring that its missing one?

@pieter do you see the ui issues at your side?

I¹ve got a projection that is creating new streams based on the name of
the machine name that created the event (they are entries from a log).
If I access the stream (created by the projection) directly, then its
there. If I look through the $streams stream, its not there.

Regards,
Matt

Is it a first/last event? randomly in the middle? Is the projection
reset and run in 3.0.1 or run in 2.0?