Worker threads recommendation

EventStoreDB
#1

Hello,

As the title states, I’m wondering what would be the recommendation on the number of worker threads.

Are those threads only CPU-bound (by delegating the task to write on the disk to other threads) or do they perform themselves the I/O operations?

I’m asking this because in the former case I would assume a number close to the number of processors might be the best configuration. In the latter case, I would increase it (4 times the number of processors?).

Any suggestion is more than welcome!

Cheers

#2

You need to share more context - are you hosting embedded or just thinking about the client ? For the former, that’s a larger It Depends (but go read the source - typically there is a thread or timer per type of activity rather than any e.g. spawning of a thread or a Task per stream or anything else non-scalable). For the latter, ES Client typically uses very few.

A good load test rig will pay back its worth by letting you inspect this IRL with oyur data and your system vs reading outline answers on mailing lists :wink:

#3

I’m speaking about the configuration of an Event Store node, not a client.

Le mar. 7 août 2018 à 08:54, Ruben Bartelink bartelink@gmail.com a écrit :

#4

Any suggestion then?

#5

There is a single writer thread and by default IIRC 4 reader threads which I believe is the setting you are referring to. Projections are a different story which also can be set. What are your overall goals?

The reader threads basically do what you would think. Are you on a high latency disk? Usually defaults should be fine or lowered! Are you running for instance on a multiple replica disk system (as example 5 linked ssds etc)?

#6

Thanks for your response.

My goal is to maximize the efficiency of an Event Store node by applying the best configuration possible for -WorkerThreads and -ReaderThreadsCount. I’m not using the projections.
If for instance, I’m running on an 8-CPU core machine, I just want to make sure the default parameters are under-optimized.

For the reader threads, I may understand it will heavily depend on the disk etc.

For the worker threads, I was not sure about their tasks actually. If they are only doing a CPU-bound task, I was assuming a parameter close to the number of CPU cores would be the best then.

#7

Worker threads don’t do that much in terms of cpu bound load except for projections, they are mostly io bound for reads etc. This of course obviously gets more complicated taking a step down as some disks can do multiple concurrent reads where as others cannot etc.