Stats collection InvalidOperationException

EventStoreDB
#1

When updating a customer site, I noticed the following error appearing in the xxxxx-single-node-err.log several times. We are using EventStore 2.0 and in this environment, and in this particular environment, there are multiple instances of single-node running simultaneously.

[PID:02960:022 2014.08.19 14:10:29.982 ERROR MonitoringService ] Error while collecting stats

System.InvalidOperationException: Instance ‘EventStore.SingleNode’ does not exist in the specified Category.

at System.Diagnostics.CounterDefinitionSample.GetInstanceValue(String instanceName)

at System.Diagnostics.PerformanceCounter.NextSample()

at System.Diagnostics.PerformanceCounter.NextValue()

at EventStore.Core.Services.Monitoring.SystemStatsHelper.GetSystemStats() in

A search of this group didn’t reveal anything, though it appears that a solution to another Stats collection issue was published roughly a year ago.

If there’s no immediate solution in 2.0, is there a way to turn down (or off) stats collection so the log files do not fill so rapidly?

Many thanks in advance.

#2

This is usually corrupt windows performance counters through we have seen it coming out of hibernation until a restart as well. Does it continue if you restart the node (hibernation does not). If so try this command lodctl /R

#3

Did you mean LODCTR /R?

Will give that a try and let you know.

Thanks.

#4

Yes I believe thats the one to reset the windows perf counter db

#5

Yes it is. Unfortunately, I know that command well.

#6

Neither option helped the problem. Is there a way you could save performance counter settings for a machine that does not have this issue and we could try loading those? Or just turn down stats collection?

#7

It should fail a few times and then stop.

These are not our perf counters they are windows perf counters we are reading. We connect to them on every restart, does the issue persist between restarts?

#8

If I understand the question correctly, yes. The errors continue to show up regularly. I am just becoming more familiar with this issue, but I believe that other users in the company have seen disks fill up with the error. I will be following up more now that it has my direct attention.

Are all windows instances set up with the same perf counters or are we possibly missing a setup step that might be considered basic? This is occuring on a VM (EC2-hosted)

#9

We do not create an perf counters this is reads of windows internal perf counters. And actually the code for stopping may be in 3.0 its been a long time but may not be in 2.0. You can also tell it to do stats less often on the command line.

Cheers,

Greg

#10

I’ve only seen the performance counter issue in two places:

1 - hibernating laptops after resumption (after quite some time - 1-2 days)

2- on virtual machines

Other software sees the same issue and suggests lodctl /R - unfortunately this doesn’t seem like our issue (if it were it would be a nice easy fix!)

James