Top Cloud Service Providers… Absolutely Horrible Speed, Latency & Consistency

image

Recently an article was posted by someone else who claimed these numbers as “showing how good” the CSP speed was however the truth is just the opposite if you know anything about network infrastructure and the types of consistency and latency a real private cloud Data-Center/Infrastructure can generate for your business, these results are absolutely horrendous and one of the primary reasons, if you made the mistake of throwing your entire business SaaS or otherwise onto the public cloud, why your experiencing so many complaints from your customers or the one administrator you hired is scratching his head and pointing his finger at the software development team… Little does he realize, the marketing hype that states the cloud is “perfect and without fault” is nothing but exactly that…. A BUNCH OF MARKETING HYPE

I found your business DEPENDS on performance and consistency of performance then the LAST PLACE you should ever put it is in a fully managed hosted environment, and that is exactly what these services are, they may not be fully managed but they still allow zero input or even insight into normal everyday aspects of any performance network such as routing, backbone connectivity, caching, utilization levels, saturation levels, etc… Not even going to go into the massive security risks associated with these types of environments.

There are a number of hidden sites around the Internet which actually monitor things like Amazon Uptime Percentage which, is a lot worse than the advertised 96% (about as bad as you can get and still be in business) and their massive latency issues which have been a constant complaint for years now and one of the things I am called on to investigate, as an executive consultant often.

To the numbers now, and the following text was taken from the article and not written by me, I have removed the authors commentary as I said he is an Amazon vendor and his bias was glaringly evident in his confused take on  these absolutely terrible results, results I would have been ashamed of when I owned my own ISP for five years in Los Gatos, CA beginning in 1999:
Methodology

The testing was done using the iperf tool on Linux. One server acts as the client and the other as the server:

Server: iperf -f m -s

Client: iperf -f m -c hostname

The OS was Ubuntu 12.04 (with all latest updates and kernel), except on Google Compute Engine, where it’s not available. There, I used the Debian Backports image.

The client was run for three tests for each type – within zone, between zones and between regions – with the mean average taken as the value reported.

Amazon networking performance

t1.micro (1 CPU)
135 Mbits/sec
101 Mbits/sec
19 Mbits/sec

c3.8xlarge (32 CPUs)
7013 Mbits/sec
3395 Mbits/sec
210 Mbits/sec

us-east-1 zone-1a us-east-1 zone-1aus-east-1 zone-1a us-east-1 zone-1dus-east-1 zone-1a us-west-1 zone-1a

Amazon’s larger instances, such as the c3.8xlarge tested here, support the enhanced 10 GB networking, however you must use the Amazon Linux AMI (or manually install the drivers) within a VPC. Because of the additional complexity of setting up a VPC, which isn’t necessary on any other provider, I didn’t test this, although it is now the default for new accounts.

However, the consistency of the performance wasn’t so good. The speeds changed quite dramatically across the three test runs for all instance types, much more than with any other provider.

You can use internal IPs within the same zone (free of charge) and across zones (incurs inter-zone transfer fees), but across regions, you have to go over the public internet using the public IPs, which incurs further networking charges.

Google Compute Engine networking performance

f1-micro (shared CPU)
692 Mbits/sec
905 Mbits/sec
531 Mbits/sec
140 Mbits/sec
137 Mbits/sec

n1-highmem-8 (8 CPUs)
2976 Mbits/sec
3042 Mbits/sec
2678 Mbits/sec
154 Mbits/sec
189 Mbits/sec

us-central-1a us-central-1aus-central-1b us-central-1bus-central-1a us-central-1bus-central-1a europe-west-1aus-central-1b europe-west-1a

Google doesn’t currently offer an Ubuntu image, so instead I used its backports-debian-7-wheezy-v20140318 image. For the f1-micro instance, I got very inconsistent iperf results for all zone tests. For example, within the same us-central-1a zone, the first run showed 991 Mbits/sec, but the next two showed 855 Mbits/sec and 232 Mbits/sec. Across regions between the US and Europe, the results were much more consistent, as were all the tests for the higher spec n1-highmem-8 server. This suggests the variability was because of the very low spec, shared CPU f1-micro instance type.

I tested more zones here than on other providers because on April 2, Google announced a new networking infrastructure in us-central-1b and europe-west-1a which would later roll out to other zones. There was about a 1.3x improvement in throughput using this new networking and users should also see lower latency and CPU overhead, which are not tested here.

Although 16 CPU instances are available, they’re only offered in limited preview with no SLA, so I tested on the fastest generally available instance type. Since networking is often CPU bound, there may be better performance available when Google releases its other instance types.

Google allows you to use internal IPs globally

Rackspace networking performance

512 MB Standard (1 CPU)
595 Mbits/sec
30 Mbits/sec
13 Mbits/sec

120 GB Performance 2 (32 CPUs)
5539 Mbits/s
534 Mbits/s
88 Mbits/s

Dallas (DFW) Dallas (DWF)Dallas (DFW) North Virginia (IAD)Dallas (DFW) London (LON)

Rackspace does not offer the same kind of zone/region deployments as Amazon or Google so I wasn’t able to run any between-zone tests. Instead I picked the next closest data center. Rackspace offers an optional enhanced virtualization platform called PVHVM. This offers better i/o and networking performance and is available on all instance types, which is what I used for these tests.

Similar to Amazon, you can use internal IPs within the same location at no extra cost but across regions you need to use the public IPs, which incur data charges.

When trying to launch x2 120 GB Performance 2 servers at Rackspace, I hit our account quota (with no other servers on the account) and had to open a support ticket to request a quota increase, which took them about an hour and a half to approve. For some reason, launching servers in the London region also requires a separate account, and logging in and out of multiple control panels soon became annoying.

Softlayer networking performance

1 CPU, 1 GB RAM, 100 Mbps
105 Mbits/sec
105 Mbits/sec
29 Mbits/sec

8 CPUs, 2 GB RAM, 1 Gbps
911 Mbits/s
921 Mbits/s
61 Mbits/s

Dallas 1 Dallas 1Dallas 1 Dallas 5Dallas 1 Amsterdam

Softlayer only allows you to deploy into multiple data centers at one location: Dallas. All other regions have a single facility. Softlayer also caps out at 1 Gbps on its public cloud instances, although its bare metal servers do have the option of dual 1 Gbps bonded network cards, allowing up to 2 Gbps. You choose the port speed when ordering or when upgrading an existing server. They also list 10Gbit/s networking as available for some bare metal servers.

Similarly to Google, Softlayer’s maximum instance size is 16 cores, but it also offers private CPU options which give you a dedicated core versus sharing the cores with other users. This allows up to eight private cores, for a higher price.

The biggest advantage Softlayer has over every other provider is completely free, private networking between all regions whereas all other provider charge for transfer out of zone. When you have VLAN spanning enabled, you can use the private network across regions, which gives you an entirely private network for your whole account.

Excerpts written by David Mutton CEO of “Server Density” a large Cloud service vendor, no wonder he is “touting” these as “good” numbers

By Jarrett Neil Ridlinghafer 
CTO of the following –
4DHealthware.com
Synapse Synergy Group
EinDrive.com
HTML5Deck.com
PerfectCapacity.com
CSPComply.com
Chief Technology Analyst, Author & Consultant
Compass Solutions, LLC
Atheneum-Partners
Hadoop Magazine
BrainBench.com
Cloud Consulting International

Advertisements