Basic questions + slow queues

EventStoreDB
#1

Hi all,

I know this is a pro forum, and I have newbie questions. I appreciate any help.

First of all I wonder if there is any preferred setup for a cluster on Azure. I’ve been playing around with a windows VM but now I read this part from James Nugent:

Be aware that configuring in Azure is a shit show unless the boxes are either in the same cloud service or virtual network thanks to it’s history as a PAAS with IAAS bolted on and still making use of “cloud services”.

So, I’m wondering if it’s a total nono al together, or if you’d have a recommended setup? The idea is to have each node on a separate location.

Second, I experience very slow queues when I run ES, both on a VM on my machine and in an Azure VM. It’s like 224 ms to 3000 ms, and even up to 7000 ms. I don’t really get how 2000 iops could be possible with this. What could be the problem here?

Thanks

#2

Re slowness latency per operation and throughput have nothing to do with each other as many operations can be baatched together. The latency is normally controlled by your disk access times.

Try running the test client and run a wrfl command in it. 2000/sec would be ridiculously slow btw. My 2 year old ultrabook is at about 15k and my mac at about 30k. These are by default single record writes of roughly 700 bytes. Obviously you could go much faster by writing 10 100byte records per write.

Greg

#3

Btw for wrfl try

Wrfl 10 50000

Thats says 10 threads for 50000 writes

#4

ok I got it, awesome G

So it’s the disk access time that sucks here.

Remember I’m a newbie on this, so my questions might be a bit strange, but in effect, when having an application with 100 users logged in issuing commands, how would these slow disk access times affect the overall performance?

This post http://abdullin.com/post/migrating-to-eventstore-in-windows-azure/ gives a couple of options, where they preferred hosting ES as a windows service in a VM.

Could I have, say 3 separate cloud services (hosting an ES clusternode in a worker role), each in a different region, and make a cluster of them? And then a fourth for the application itself. Would that be an ok setup?

#5

They would affect the latency of the operation. Why you are seeing 7 second disk accesses is a whole different story.

James knows more about azure setups than i maybe he can chime in here. Generally though you dont want wan or slow connections though

Greg

#6

Yeah, I don’t know what is up with that disk access…

Thanks again.

I’ll be trying something out here and update with my findings. I’ll start looking at having the cluster in the same Cloud Service (i.e. same geo-location). James, should you have any general advice, don’t hesitate :slight_smile:

#7

A friend told me yesterday that they were having azure disk issues. They could confirm a difference in oops between various VMs they were running with the same software. The difference turned out to be VM size.

Just going to a larger size made a remarkable difference.

They were consider an ssd instance, but didn’t need to go that route.

They had numbers from a tool to validate the experience.

Kind of a shame that changing “size” dramatically changes disk performance.

#8

It depends on the size of VM, Minimum production acceptable performance is Standard A3, I even tried to switch to the new D2 having SSD temp drive, and accessing storage account was much slower! D3 would be better but more expensive. Note also that soon Azure would be releasing SSD storage (Premium Storage) that should make a big difference.

#9

Azure VM IOPS is documented here.

I’ve had pretty good performance using an Standard A2 with 4 data disks in RAID 0.

http://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-configure-raid/

#10

Iops dont equate to latency per iops.

#11

Yes Kijana, that’s it!
I was trying this on the Standard A0. Hadn’t guessed that it would be THAT slow…

And I agree about the size.

Thanks all for the input.