Hello,
We are trying to figure out how to best projections, and scaled our use of projection from around 10 to 2 who create links to existing streams. I am joining the code for these projections below.
Unfortunately these projections seem to fail when they get close to going through all the events in the event store.
The event store crashes on a regular basis with the messages included below.
I was wondering whether any one here might have had an exception of this type while using projections ?
Any help would be appreciated.
For information we went from trying to use projections to browsing the fromAll stream in order to avoid crashing the production server which seems to happen pretty often with the use of projections
Best regards
Omar Msaaf
Projections print screen
Error messages
[PID:03596:011 2017.06.20 18:05:13.446 FATAL GLOBAL-LOGGER ] Global Unhandled Exception occurred.
System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.Dictionary2.ValueCollection.Enumerator.MoveNext() at System.Linq.Enumerable.Any[TSource](IEnumerable1 source, Func`2 predicate)
at EventStore.Projections.Core.Services.Processing.EventByTypeIndexEventReader.IndexBased.Handle(ReadTimeout message) in c:\projects\eventstore\src\EventStore.Projections.Core\Services\Processing\EventByTypeIndexEventReader.cs:line 293
at EventStore.Core.Messaging.SendToThisEnvelope.ReplyWith[T](T message) in c:\projects\eventstore\src\EventStore.Core\Messaging\SendToThisEnvelope.cs:line 20
at EventStore.Core.Services.TimerService.ThreadBasedScheduler.DoTiming() in c:\projects\eventstore\src\EventStore.Core\Services\TimerService\ThreadBasedScheduler.cs:line 68
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
[PID:06060:017 2017.06.20 18:37:46.377 ERROR StorageReaderWorker ] Error during processing ReadAllEventsForward request.
System.ArgumentOutOfRangeException: Non-negative number required.
Parameter name: count
at System.IO.BinaryReader.ReadBytes(Int32 count)
at EventStore.Core.TransactionLog.LogRecords.PrepareLogRecord…ctor(BinaryReader reader, Byte version, Int64 logPosition) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\LogRecords\PrepareLogRecord.cs:line 152
at EventStore.Core.TransactionLog.LogRecords.LogRecord.ReadFrom(BinaryReader reader) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\LogRecords\LogRecord.cs:line 50
at EventStore.Core.TransactionLog.Chunks.TFChunk.TFChunk.TFChunkReadSide.TryReadForwardInternal(ReaderWorkItem workItem, Int64 actualPosition, Int32& length, LogRecord& record) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\Chunks\TFChunk\TFChunkReadSide.cs:line 470
at EventStore.Core.TransactionLog.Chunks.TFChunk.TFChunk.TFChunkReadSideUnscavenged.TryReadClosestForward(Int64 logicalPosition) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\Chunks\TFChunk\TFChunkReadSide.cs:line 82
at EventStore.Core.TransactionLog.Chunks.TFChunkReader.TryReadNextInternal(Int32 retries) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\Chunks\TFChunkReader.cs:line 52
at EventStore.Core.Services.Storage.ReaderIndex.AllReader.ReadAllEventsForward(TFPos pos, Int32 maxCount) in c:\projects\eventstore\src\EventStore.Core\Services\Storage\ReaderIndex\AllReader.cs:line 57
at EventStore.Core.Services.Storage.ReaderIndex.ReadIndex.EventStore.Core.Services.Storage.ReaderIndex.IReadIndex.ReadAllEventsForward(TFPos pos, Int32 maxCount) in c:\projects\eventstore\src\EventStore.Core\Services\Storage\ReaderIndex\ReadIndex.cs:line 96
at EventStore.Core.Services.Storage.StorageReaderWorker.ReadAllEventsForward(ReadAllEventsForward msg) in c:\projects\eventstore\src\EventStore.Core\Services\Storage\StorageReaderWorker.cs:line 303
[PID:06060:005 2017.06.20 18:42:30.596 ERROR StorageReaderWorker ] Error during processing ReadAllEventsForward request.
System.ArgumentOutOfRangeException: Ticks must be between DateTime.MinValue.Ticks and DateTime.MaxValue.Ticks.
Parameter name: ticks
at System.DateTime…ctor(Int64 ticks)
at EventStore.Core.TransactionLog.LogRecords.PrepareLogRecord…ctor(BinaryReader reader, Byte version, Int64 logPosition) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\LogRecords\PrepareLogRecord.cs:line 145
at EventStore.Core.TransactionLog.LogRecords.LogRecord.ReadFrom(BinaryReader reader) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\LogRecords\LogRecord.cs:line 50
at EventStore.Core.TransactionLog.Chunks.TFChunk.TFChunk.TFChunkReadSide.TryReadForwardInternal(ReaderWorkItem workItem, Int64 actualPosition, Int32& length, LogRecord& record) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\Chunks\TFChunk\TFChunkReadSide.cs:line 470
at EventStore.Core.TransactionLog.Chunks.TFChunk.TFChunk.TFChunkReadSideUnscavenged.TryReadClosestForward(Int64 logicalPosition) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\Chunks\TFChunk\TFChunkReadSide.cs:line 82
at EventStore.Core.TransactionLog.Chunks.TFChunkReader.TryReadNextInternal(Int32 retries) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\Chunks\TFChunkReader.cs:line 52
at EventStore.Core.Services.Storage.ReaderIndex.AllReader.ReadAllEventsForward(TFPos pos, Int32 maxCount) in c:\projects\eventstore\src\EventStore.Core\Services\Storage\ReaderIndex\AllReader.cs:line 57
at EventStore.Core.Services.Storage.ReaderIndex.ReadIndex.EventStore.Core.Services.Storage.ReaderIndex.IReadIndex.ReadAllEventsForward(TFPos pos, Int32 maxCount) in c:\projects\eventstore\src\EventStore.Core\Services\Storage\ReaderIndex\ReadIndex.cs:line 96
at EventStore.Core.Services.Storage.StorageReaderWorker.ReadAllEventsForward(ReadAllEventsForward msg) in c:\projects\eventstore\src\EventStore.Core\Services\Storage\StorageReaderWorker.cs:line 303
[PID:06060:023 2017.06.20 19:59:13.716 ERROR StorageReaderWorker ] Error during processing ReadAllEventsForward request.
EventStore.Core.TransactionLog.Chunks.TFChunk.InvalidReadException: Log record at actual pos 1 has too large length: 33554432 bytes, while limit is 16777216 bytes. In chunk #0-0 (chunk-000000.000000).
at EventStore.Core.TransactionLog.Chunks.TFChunk.TFChunk.TFChunkReadSide.TryReadForwardInternal(ReaderWorkItem workItem, Int64 actualPosition, Int32& length, LogRecord& record) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\Chunks\TFChunk\TFChunkReadSide.cs:line 472
at EventStore.Core.TransactionLog.Chunks.TFChunk.TFChunk.TFChunkReadSideUnscavenged.TryReadClosestForward(Int64 logicalPosition) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\Chunks\TFChunk\TFChunkReadSide.cs:line 82
at EventStore.Core.TransactionLog.Chunks.TFChunkReader.TryReadNextInternal(Int32 retries) in c:\projects\eventstore\src\EventStore.Core\TransactionLog\Chunks\TFChunkReader.cs:line 52
at EventStore.Core.Services.Storage.ReaderIndex.AllReader.ReadAllEventsForward(TFPos pos, Int32 maxCount) in c:\projects\eventstore\src\EventStore.Core\Services\Storage\ReaderIndex\AllReader.cs:line 57
at EventStore.Core.Services.Storage.ReaderIndex.ReadIndex.EventStore.Core.Services.Storage.ReaderIndex.IReadIndex.ReadAllEventsForward(TFPos pos, Int32 maxCount) in c:\projects\eventstore\src\EventStore.Core\Services\Storage\ReaderIndex\ReadIndex.cs:line 96
at EventStore.Core.Services.Storage.StorageReaderWorker.ReadAllEventsForward(ReadAllEventsForward msg) in c:\projects\eventstore\src\EventStore.Core\Services\Storage\StorageReaderWorker.cs:line 303