Performance on Single-node, 2 cores (2.3 gHz), 2GB ram?

Caveat…this is also running in a virtual machine…also running v3 rc9 of the event store.

I’ve got a simple test, sending 1000 messages (fairly small, really just an integer), using Json.NET for serialize/deserialize…

Anyway, seeing times of around 4-5 seconds to send, receive and print integer to console; that is all 1000 messages take a total of about 4-5 seconds. I haven’t bothered putting a watch on it yet, b/c this just seems much slower than I expected.

Granted, I’m not giving it a lot of resources, plus there is the vm angle.

Does this sound about right?

Am I just asking too much, or is there something I should look at to tune the setup?

There is a perf test in testclient for writes. Try WRFL 10 100000 (10 threads 100k messages).

My guess is your test is synchronous? :slight_smile: This means each write equals:

Send to server

Server writes

Receive from server

Loop

As such you also have the communications overhead. Try either a) sending more than one message at once b) having more than one thread pushing messages.

Cheers,

Greg

I’ll back up even further here and run the perf test you referenced and then move forward from there.

Thanks for responding :slight_smile:

Cool, so I’m seeing numbers now that make sense, testing from same box as that running event store. I’m happy now that it’s certainly something in my setup or code. My test wasn’t synchronous like you describe, but will use perf test as example code to feel things out.

Question: is there a way to use perf test to hit a remote machine? How about auth?

Another question: When we subscribe to a stream, I had assumed (probably wrongly) that each received message would be spun off into its own pooled thread. Do subscription handlers run on the main thread, or do I need to manually spin each off into its own thread if that’s what I want? Is it best practice to use one or multiple connections?..it’s feeling like we need to open multiple connections.

I’m sure examining perf will illuminate me on some of these issues.

I think it’s on a thread pool thread

It is. You probably don’t want a thread/event (its fairly expensive)

re: remote machine if you look at --help on the command line there are options for instance --ip 192.168.0.4 to connect to remote machines etc localhost is just the default

Thanks, dropping down into that repl threw me off a bit.

So, my understanding so far is that for best performance for pushing out multiple writes quickly we want to multi-thread the application. For subscriptions we want to just subscribe and let client library handle the threading.

When I started in on all of this my design was as below, and involves akka dotnet:

  1. One main stream to which I have one subscriber

  2. As certain events are posted to that stream my subscriber will spin off a set of actors

  3. Each set of actors will create at least one stream of its own and possibly one or a few more streams

-Each stream that is being written to will have its own actor

  1. As the set of actors does its work, it will write to the streams created in #3

If I understand all the pieces correctly, this initial design I had should be close to what would be best for performance for EventStore. The application as a whole was far from performant, though.

I know bringing in akka dotnet adds another variable, but am I missing something wildly obvious here that would’ve caused a problem?

Either way, I’m going to write another test utilizing client api library (perf tests provided use other classes and types not in client api) and see how many writes AND reads (via subscription) I can send through a single node (will probably try clustered as well). Ultimately my goal as far as event store goes is to try to get some kind of number and feel so that I can know how well we can scale this thing. I also need to make sure I understand how to use it correctly in a performant way.

If you are interested in akka I would consider using actual akka. We have a client api written in akka btw :slight_smile:

Otherwise things seem pretty reasonable and yes you basically just need multiple threads writing.

“Either way, I’m going to write another test utilizing client api library (perf tests provided use other classes and types not in client api) and see how many writes AND reads (via subscription) I can send through a single node (will probably try clustered as well).”

Feel free to drop your code here

Congratulations on the new release!

Thanks for the sanity check on the design. Here is the code I am now using to explore perf. It’s pretty basic, but I believe operates in roughly same manner as perf test client included in downloads. I can get up to about 1500 messages/sec with current rig, depending on how I set params. Next step for me is to try out different hardware configs and see how that affects things.

using EventStore.ClientAPI;

using System;

using System.Collections.Generic;

using System.Diagnostics;

using System.Threading;

namespace MyEventStorePerf

{

class Program

{

class StreamTracker {

public string StreamName = “Perf-” + Guid.NewGuid().ToString();

public int WriteCount = 0;

public int HowManyWriters = 1;

public int HowManyMessages = 1;

public Stopwatch Watch = new Stopwatch();

}

static void Main(string[] args)

{

Console.WriteLine(“started”);

var conn = EventStoreConnection.Create(new System.Net.IPEndPoint(System.Net.IPAddress.Parse(“192.168.164.132”), 1113));

conn.Connect();

var starts = new List();

//500 30 per

int howManyStreams = 500;

int howManyMessages = 30;

int howManyWritersPerStream = 10;

var totalTime = new Stopwatch();

var streamCount = 0;

Action onFinished = () => {

Interlocked.Increment(ref streamCount);

if (streamCount == howManyStreams) {

totalTime.Stop();

var totalRequests = howManyStreams * howManyMessages;

var reqPerSec = (totalRequests + 0.0) / totalTime.ElapsedMilliseconds * 1000;

Console.WriteLine(“Total {0} requests completed in {1}ms ({2:0.00} reqs per sec).”, totalRequests, totalTime.ElapsedMilliseconds, reqPerSec);

Console.WriteLine(“finished”);

conn.Close();

}

};

Func<StreamTracker, Action> getStarter = _tracker => {

var tracker = _tracker;

var thread = new Thread(new ThreadStart(() =>

{

writeEvents(tracker, conn);

}));

return () => {

if (!tracker.Watch.IsRunning) {

tracker.Watch.Start();

}

thread.Start();

};

};

for (int i = 0; i < howManyStreams; i++) {

var tracker = new StreamTracker { HowManyWriters = howManyWritersPerStream, HowManyMessages = howManyMessages };

listenEvents(tracker, conn, onFinished);

for (int k = 0; k < howManyWritersPerStream; k++)

{

starts.Add(getStarter(tracker));

}

}

totalTime.Start();

foreach (var writeThread in starts) {

writeThread();

}

Console.ReadLine();

}

static void listenEvents(StreamTracker tracker, IEventStoreConnection conn, Action onFinished)

{

var count = 0;

conn.SubscribeToStream(tracker.StreamName, false, (sub, evt) =>

{

Interlocked.Increment(ref count);

if (count == tracker.HowManyMessages)

{

tracker.Watch.Stop();

var reqPerSec = (tracker.HowManyMessages + 0.0) / tracker.Watch.ElapsedMilliseconds * 1000;

Console.WriteLine("{0} requests completed in {1}ms ({2:0.00} reqs per sec).", tracker.HowManyMessages, tracker.Watch.ElapsedMilliseconds, reqPerSec);

onFinished();

}

});

}

static void writeEvents(StreamTracker tracker, IEventStoreConnection conn)

{

var _dataBytes = System.Text.Encoding.UTF8.GetBytes(“1”);

for (int i = 0; i < tracker.HowManyMessages; i++)

{

var msg = new EventData(Guid.NewGuid(), “PerfEvent”, true, _dataBytes, null);

Interlocked.Increment(ref tracker.WriteCount);

if (tracker.WriteCount <= tracker.HowManyMessages)

{

conn.AppendToStream(tracker.StreamName, ExpectedVersion.Any, msg);

}

else {

break;

}

}

}

}

}

``

Must be very weak hardware? My 1.5 yr old ultra book does ten times this. What are you running on?

Well, I’ve got two VMs running on my MacBook Air, one for the store, other for everything else.

I’ve given them each two (virtual) cores and 2 GB of RAM. The store is running on an ubuntu box and the rest is a Win7 box.

When I get home, I’m going to offload one or both onto a much beefier machine so I can allocate more resources and see what that does. At some point I also want to put it up on Azure and see what that looks like, as that would be truer to what we would eventually do.

Try not running in a vm (I am running native linux) also usually performance is much better with “directly connected” drives for a vm than through the vm file system.

Cheers,

Greg

Will definitely try that as well. I also suspect the virtual filesystem might be slowing things down significantly. My ultimate target is actually Azure or AWS. Somewhere I’ve seen some info on running eventstore on azure; will be sure to look that up when I get to that point.

I don’t suppose you all have any plans to offer ES as a service at some point?

There are plans for this