Event Store node startup time proportional to amount of data stored?

EventStoreDB
1

Hi

Our experience with an 800 GB cluster has lead to the suspicion that a nodes startup time is proportional to the amount of data stored… Which is about an hour in our case.

Is this the case? And if so any plans to improve in this area?

Thanks!

2

What is the size of the index? I am guessing this is where your time
is being spent (loading/validating index) also what version?

3

1 index is 30 gb, another is 20 gb, a third is 5 gb. And then we have 5 which are between 20-300 mb.

Version is 4 rc1 (just updated).

4

During start up you should see messages printed about indexes being
loaded/verified is this what is taking the time? Also are you
validating checksums on transaction files at the same time
(skip-db-verify is set?)

5

skip-db-verify is true. And yes, looks like it’s the indexes being loaded and verified that takes the time:

.03.01 14:37:44.433 TRACE PTable ] Loading PTable (Version: 2) ‘7b6f74c8-5462-4467-8b84-2c17e2d30231’ (1000234 entries, cache depth 16) done in 00:00:01.7738979.

[PID:35596:015 2017.03.01 14:37:44.433 TRACE PTable ] Loading and Verification of PTable ‘96ffb792-a520-48ea-bcaa-7b0d4672046e’ started…

[PID:35596:015 2017.03.01 14:37:48.592 TRACE PTable ] Loading PTable (Version: 2) ‘96ffb792-a520-48ea-bcaa-7b0d4672046e’ (2000318 entries, cache depth 16) done in 00:00:04.1582695.

[PID:35596:015 2017.03.01 14:37:48.592 TRACE PTable ] Loading and Verification of PTable ‘cf726707-086c-4a58-ada5-126e757c5b24’ started…

[PID:35596:022 2017.03.01 14:37:53.760 DEBUG HttpEntityManager ] Error during setting content length on HTTP response: This operation cannot be performed after the response has been submitted…

[PID:35596:015 2017.03.01 14:37:56.883 TRACE PTable ] Loading PTable (Version: 2) ‘cf726707-086c-4a58-ada5-126e757c5b24’ (4001012 entries, cache depth 16) done in 00:00:08.2905010.

[PID:35596:015 2017.03.01 14:37:56.883 TRACE PTable ] Loading and Verification of PTable ‘26c64f8d-3f22-4522-b42a-a696ac5c772d’ started…

[PID:35596:015 2017.03.01 14:38:10.453 TRACE PTable ] Loading PTable (Version: 2) ‘26c64f8d-3f22-4522-b42a-a696ac5c772d’ (8002016 entries, cache depth 16) done in 00:00:13.5687496.

[PID:35596:015 2017.03.01 14:38:10.453 TRACE PTable ] Loading and Verification of PTable ‘7efcbe58-4c43-4f5a-87a6-0b02b8c062eb’ started…

[PID:35596:015 2017.03.01 14:38:38.435 TRACE PTable ] Loading PTable (Version: 2) ‘7efcbe58-4c43-4f5a-87a6-0b02b8c062eb’ (16003178 entries, cache depth 16) done in 00:00:27.9820139.

[PID:35596:015 2017.03.01 14:38:38.435 TRACE PTable ] Loading and Verification of PTable ‘c1825092-9a78-478d-a001-4bd5f207c916’ started…