Poor persistent subscriptions performance

EventStoreDB V20.X
#1

Hello Community!

I’m using .net gRPC Client with EventStore DB 20.6.1 server (single node). Test workload contains 1KB events. I’m using persistent subscription to read events. Performance is poor: only 650 msg/sec.
EventStoreDB and test application are on the same machine, network traffic is via localhost loopback. Machine specs: Intel i5-8350U@3.60Ghz, 16GB RAM, 500GB SSD (CPU is < 25% and disc I/O is < 5% during test).

Are persistent subscriptions slow by design? Any hints to improve? - I need at least 3000 msg/sec.

Thank you.

#2

Persistent subscriptions write events for checkpoints as the checkpoint for each subscription is stored on the server. As it introduces additional write load to the cluster, I’d suggest tuning the checkpoint settings to avoid writing checkpoints too often. The risk there, of course, is that in case of a failure, you’ll get some events again. As usual, it’s a balance between performance and consistency.

These are the settings:

Setting Description
CheckPointAfter(TimeSpan time) The amount of time the system should try to checkpoint after.
MinimumCheckPointCountOf(int count) The minimum number of messages to write a checkpoint for.
MaximumCheckPointCountOf(int count) The maximum number of messages not checkpointed before forcing a checkpoint.
#3

Yes, I was able to achieve the required performance by changing various subscription parameters.
I set: Live Buffer Count: 10000, Buffer Size: 10000, Read Batch Size: 2000, CheckPoint After (ms): 10000, Min CheckPoint Count: 10, Max CheckPoint Count: 50000. Performance jumped to 3000-4000 events/sec.

1 Like