Hi ES team!
We have cluster with 3 nodes, version 22.10.2
We caught an exception
{
"@t": "2024-09-25T20:12:47.0954226+03:00",
"@mt": "Host terminated unexpectedly.",
"@l": "Fatal",
"@i": 2897046276,
"@x": "System.IndexOutOfRangeException: Index was outside the bounds of the array.n at EventStore.Core.TransactionLog.Chunks.TFChunkManager.AddChunk(TFChunk chunk) in /home/runner/work/TrainStation/TrainStation/build/oss-eventstore/src/EventStore.Core/TransactionLog/Chunks/TFChunkManager.cs:line 154n at EventStore.Core.TransactionLog.Chunks.TFChunkDb.Open(Boolean verifyHash, Boolean readOnly, Int32 threads) in /home/runner/work/TrainStation/TrainStation/build/oss-eventstore/src/EventStore.Core/TransactionLog/Chunks/TFChunkDb.cs:line 150n at EventStore.Core.ClusterVNode1 logFormatAbstractorFactory, AuthenticationProviderFactory authenticationProviderFactory, AuthorizationProviderFactory authorizationProviderFactory, IReadOnlyList1 additionalPersistentSubscriptionConsumerStrategyFactories, CertificateProvider certificateProvider, IExpiryStrategy expiryStrategy, Nullable1 instanceId, Int32 debugIndex) in /home/runner/work/TrainStation/TrainStation/build/oss-eventstore/src/EventStore.Core/ClusterVNode.cs:line 639n at EventStore.Core.ClusterVNode.Create[TStreamId](ClusterVNodeOptions options, ILogFormatAbstractorFactory1 logFormatAbstractorFactory, AuthenticationProviderFactory authenticationProviderFactory, AuthorizationProviderFactory authorizationProviderFactory, IReadOnlyList1 factories, CertificateProvider certificateProvider, Nullable`1 instanceId, Int32 debugIndex) in /home/runner/work/TrainStation/TrainStation/build/oss-eventstore/src/EventStore.Core/ClusterVNode.cs:line 84n at EventStore.ClusterNode.ClusterVNodeHostedService..ctor(ClusterVNodeOptions options, CertificateProvider certificateProvider) in /home/runner/work/TrainStation/TrainStation/build/oss-eventstore/src/EventStore.ClusterNode/ClusterVNodeHostedService.cs:line 83n at EventStore.ClusterNode.Program.Main(String[] args) in /home/runner/work/TrainStation/TrainStation/build/oss-eventstore/src/EventStore.ClusterNode/Program.cs:line 169",
"ProcessId": 1980470,
"ThreadId": 1
}
and all nodes was stopped.
In src we found that MaxChunksCount = 400_000;
There are files with name chunk-400000.000000 in data folder on servers
Obviliously, we have exceeded the limit of 400k chunks.
As a temporary solution, we build the EventStore.Core.dll with a 600k limit and replace it on servers. But this only postponed the problem.
Are there ways to bypass this limitation?