A while ago (August 15, 2013), Windows Azure MVP Wely Lau posted in Cloud Zone a comparison of different IaaS platforms from the performance perspective. The comparison was very limited, only comparing single instance Ubuntu Linux performance on 5 different cloud providers. That being said, it was still interesting to see how different cloud providers measure up against each other in simple one dimensional performance test. The original post can be found here.  In it he reaches a conclusion that Windows Azure performance is about 3x better than Amazon AWS performance.

I stumbled upon this comparison just couple of days ago, and one thing that I immediately saw is that the Amazon EC2 instance used was m1.medium – this instance type has been on offer rom Amazon for many years already (starting somewhere in 2009 if I remember correctly). Today (April, 2014) Amazon lists this as previous-generation instance. There are current generation instances available that roughly match the configuration options (CPU, RAM) of the previous generation instance.

Given that, I decided to do the same test myself. Since I have ready access only to AWS and Azure immediately at hand, I did the tests only on these 2 cloud environments. For virtual machines, I chose the same Ubuntu 12.04 LTS image as original tester, and booted 2 virtual Machines:

• c3.large on Amazon EC2. It has 2 vCPU-s and 3,75GB of RAM
• A2 on Azure. It has 2 vCPU-s and 3,5GB OF RAM

These two instance types are not exact match, as c3.large has 0,25GB more memory, but most importantly, they have equal number of vCPU-s. What that means in practice, we shall see next.

I ran the latest version UnixBench (5.1.3) on both machines twice, and here are the results:

Number on the Y-axis shows UnixBench score.

Quite frankly, the results are stunning. UnixBench test suite shows that Azure instance is approximately 50% faster than comparable Amazon EC2 unit with Linux guest OS. Looking at the detailed test scores, it seems that the biggest difference (about 4x) is in file copy operations and in pipe-based context switching operations. That being said, all other tests show significantly better performance on Azure as well.

Here are the detailed test results:

Azure A2 – Run1
Azure A2 – Run2
EC2 c3.large – Run1
EC2 c3.large – Run2

Looking at the processor specifications inside the instances of Azure A2 and EC2 c3.large do not explain this performance difference either. Azure uses (at least for the host where my test machine was running) AMD Opteron 4171 HE processor, whereas EC2 uses Intel Xeon CPU E5-2680 v2. I could not find any performance test results for Opteron 4171 HE processor, but the Intel E5-2680 v2 sits at the top of the CPU Benchmark list of high-end processors. So, at least VM running on this Intel processor should not be considerably slower than the one running on AMD processor.

It seems that although Amazon has come out with newer instance types after Wely Lau posted his test results in August 2013, not much has changed. It is not 3x the performance difference anymore, but almost 2x difference is still huge.

The hourly cost of Amazon EC2 c3.large (in EU region) is $0.120. Hourly cost for Azure A2 is $0.094. Given that A2 cost per hour is 78,3% of the comparable EC2 c3.large, and that A2 performance is on average 45,5% better than EC2 c3.large, then my conclusion form this limited test is that if single instance performance is important, then Azure is much more cost effective choice. In other words, you can run 45,5% more CPU intensive workload on Azure A2 for 78,3% of the cost compared  to Amazon EC2. And this conclusion is coming from someone who has been Amazon AWS customer and fan of their service since the beginning of 2009.

Granted, I did not look at the other aspects, such as data egress, storage, etc., and these factors will affect the cost at the end. Also, this was testing only Linux OS instances – it would be interesting to do similar tests with Windows virtual machines and with other flavors of Linux – Red Hat for example.