Exception in writer - Idle GES on AWS

As an evaluation, I am running GES 3.3.0 on Windows 2012 R2 on an AWS instance.

It had been running idle for 4 days, after which the following occured in the log and GES died:

[PID:01540:008 2015.12.09 19:21:34.760 ERROR StorageWriterService] Exception in writer.

System.IO.IOException: The device is not ready.

at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)

at System.IO.FileStream.WriteCore(Byte[] buffer, Int32 offset, Int32 count)

at EventStore.Core.TransactionLog.Chunks.TFChunk.TFChunk.WriteRawData(WriterWorkItem workItem, Byte[] buf, Int32 len) in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\TransactionLog\Chunks\TFChunk\TFChunk.cs:line 668

at EventStore.Core.TransactionLog.Chunks.TFChunk.TFChunk.TryAppend(LogRecord record) in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\TransactionLog\Chunks\TFChunk\TFChunk.cs:line 641

at EventStore.Core.TransactionLog.Chunks.TFChunkWriter.Write(LogRecord record, Int64& newPos) in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\TransactionLog\Chunks\TFChunkWriter.cs:line 37

at EventStore.Core.Services.Storage.StorageWriterService.WritePrepareWithRetry(PrepareLogRecord prepare) in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\Services\Storage\StorageWriterService.cs:line 603

at EventStore.Core.Services.Storage.StorageWriterService.EventStore.Core.Bus.IHandle<EventStore.Core.Messages.StorageMessage.WritePrepares>.Handle(WritePrepares msg) in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\Services\Storage\StorageWriterService.cs:line 269

[PID:01540:008 2015.12.09 19:21:34.917 FATAL StorageWriterService] Unexpected error in StorageWriterService. Terminating the process…

System.Exception: FlushFileBuffers failed with err: 21

at EventStore.Common.Utils.FileStreamExtensions.b__2(FileStream f) in c:\buildAgent\work\47590a33512c319d\src\EventStore.Common\Utils\FileStreamExtensions.cs:line 57

at EventStore.Core.TransactionLog.Chunks.TFChunk.TFChunk.Flush() in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\TransactionLog\Chunks\TFChunk\TFChunk.cs:line 679

at EventStore.Core.TransactionLog.Chunks.TFChunkWriter.Flush() in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\TransactionLog\Chunks\TFChunkWriter.cs:line 86

at EventStore.Core.Services.Storage.StorageWriterService.Flush(Boolean force) in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\Services\Storage\StorageWriterService.cs:line 661

at EventStore.Core.Services.Storage.StorageWriterService.EventStore.Core.Bus.IHandle<EventStore.Core.Messages.StorageMessage.WritePrepares>.Handle(WritePrepares msg) in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\Services\Storage\StorageWriterService.cs:line 301

at EventStore.Core.Bus.MessageHandler`1.TryHandle(Message message) in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\Bus\MessageHandler.cs:line 33

at EventStore.Core.Bus.InMemoryBus.Publish(Message message) in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\Bus\InMemoryBus.cs:line 315

at EventStore.Core.Services.Storage.StorageWriterService.CommonHandle(Message message) in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\Services\Storage\StorageWriterService.cs:line 160

[PID:01540:008 2015.12.09 19:21:34.979 ERROR Application ] Exiting with exit code: 1.

Exit reason: Unexpected error in StorageWriterService: FlushFileBuffers failed with err: 21

Is this a bug? Is it something AWS related? Do I create a GH issue for this?

Best regards,

David

Pretty clear what the issue is (device is not ready) [PID:01540:008
2015.12.09 19:21:34.760 ERROR StorageWriterService] Exception in
writer.
System.IO.IOException: The device is not ready.

What storage mechanism are you using? Looks like windows might have
not been able to write to the disk?

Regular EBS disk mounted to the instance, nothing exotic.

David Cumps

// Mail sent from mobile device

Well the OS is returning "Device not ready" I might look to see if
some smart happens with windows on AWS like not keeping the disk
active during idle times

I’ll follow it up, it has been running again for 3 days now :slight_smile:

David Cumps

// Mail sent from mobile device