Performance Counters - Which are needed?

David_Cumps · December 25, 2015, 2:34pm

My logs files are full with the following:

[PID:03436:016 2015.12.09 00:33:45.633 ERROR MonitoringService ] Error while collecting stats

System.InvalidOperationException: The Counter layout for the Category specified is invalid, a counter of the type: AverageCount64, AverageTimer32, CounterMultiTimer, CounterMultiTimerInverse, CounterMultiTimer100Ns, CounterMultiTimer100NsInverse, RawFraction, or SampleFraction has to be immediately followed by any of the base counter types: AverageBase, CounterMultiBase, RawBase or SampleBase.

at System.Diagnostics.CategorySample.GetCounterDefinitionSample(String counter)

at System.Diagnostics.PerformanceCounter.NextSample()

at EventStore.Core.Services.Monitoring.Utils.PerfCounterHelper.GetGcStats() in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\Services\Monitoring\Utils\PerfCounterHelper.cs:line 119

at EventStore.Core.Services.Monitoring.SystemStatsHelper.GetSystemStats() in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\Services\Monitoring\SystemStatsHelper.cs:line 66

at EventStore.Core.Services.Monitoring.MonitoringService.CollectStats() in c:\buildAgent\work\47590a33512c319d\src\EventStore.Core\Services\Monitoring\MonitoringService.cs:line 127

How do I install the required perf counters?

Best regards,

David

Greg_Young1 · December 25, 2015, 2:37pm

Welcome to windows perf counters. They like to become corrupted.

This should reset it for you : lodctr /R

We have seen some people get to the point with an image where they
just constantly keep corrupting. With a different image they never do.
I don't even want to know what is going on under the covers.

Greg

David_Cumps · December 25, 2015, 2:44pm

Thanks, will give that a shot.

David Cumps

// Mail sent from mobile device

David_Cumps · December 26, 2015, 8:16pm

The errors are gone, seems something weird is going on with the .NET perf counters:

[PID:03336:005 2015.12.26 13:50:19.269 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR Exceptions’, counter=’# of Exceps Thrown / sec’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

[PID:03336:005 2015.12.26 13:50:19.284 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR LocksAndThreads’, counter=‘Contention Rate / sec’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

[PID:03336:005 2015.12.26 13:50:19.284 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR Memory’, counter=’# Gen 0 Collections’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

[PID:03336:005 2015.12.26 13:50:19.284 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR Memory’, counter=’# Gen 1 Collections’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

[PID:03336:005 2015.12.26 13:50:19.284 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR Memory’, counter=’# Gen 2 Collections’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

[PID:03336:005 2015.12.26 13:50:19.284 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR Memory’, counter=‘Gen 0 heap size’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

[PID:03336:005 2015.12.26 13:50:19.284 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR Memory’, counter=‘Gen 1 heap size’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

[PID:03336:005 2015.12.26 13:50:19.284 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR Memory’, counter=‘Gen 2 heap size’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

[PID:03336:005 2015.12.26 13:50:19.284 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR Memory’, counter=‘Large Object Heap size’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

[PID:03336:005 2015.12.26 13:50:19.284 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR Memory’, counter=‘Allocated Bytes/sec’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

[PID:03336:005 2015.12.26 13:50:19.284 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR Memory’, counter=’% Time in GC’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

[PID:03336:005 2015.12.26 13:50:19.284 TRACE MonitoringService ] Could not create performance counter: category=’.NET CLR Memory’, counter=’# Bytes in all Heaps’, instance=‘EventStore.ClusterNode’. Error: Category does not exist.

I’ll try to find something to repair that

David_Cumps · December 26, 2015, 8:20pm

For reference:

unlodctr .NETFramework.

lodctr C:\Windows\Inf.NETFramework\corperfmonsymbols.ini

Greg_Young1 · December 26, 2015, 9:53pm

The perf counters are extremely weird in windows is all I can say,
there are tons of issues even with MS products (which is where we
figured out how to deal with them)

David_Cumps · December 30, 2015, 3:13pm

Does this have to do something with stats too?

[PID:03732:014 2015.12.30 01:33:57.317 ERROR MonitoringService ] Maximum rebuild attempts reached. Giving up on rebuilds.

The perf counter errors are back to Any way I can disable them, or are they needed?

Greg_Young1 · December 30, 2015, 4:30pm

They are included in stats.

Likely trying from a fresh windows image will resolve the issue.

You can also disable stats (set stats interval to a very large number
say once every 5 years)