Rackspace FirstGen vs. OpenCloud

[UPDATED 10/9/2012]

There’s Good News and There’s Bad News

So the good news is yet to be determined – I’m sure there are lots of good things about the Rackspace’s move to OpenCloud… I mean seriously it has ‘open’ right in the title… that must be good. Right?! Unfortunately I’m here to report the bad news.

Here is a quick visual chart of the benchmark results:

Rackspace FirstGen vs. OpenCloud Benchmark

Rackspace FirstGen vs. OpenCloud Benchmark

As you can see the first generation Rackspace servers perform far better than the new OpenCloud serves. That is of course an understatement, basically it’s sucktown, U.S.A!!! Sigh…

I’ve been trying to engage Rackspace at every level I can to try to address this obvious downgrade, and hopefully avert a disaster to my beloved Rackspace! Here are some of the communications I’ve had thus far with the folks at Rackspace:

Greetings,

It was a pleasure chatting with you. The reason for this ticket is because you ran a benchmark test on your open cloud servers vs first gen servers. You have sent be a google docs page with your results, and it shows that the first gen servers are significantly better than the open cloud servers.

https://docs.google.com/spreadsheet/ccc?key=0Au6q9_68v25vdFZHMjBRZDZGclRtejJhYzFpMXVhMXc

Although this shouldn’t be the case, and you were wanting to know if there was another benchmark testing suite that you could use to test the servers more. I am passing this over to a higher level tech to point you in the right direction. If you would like any additional assistance, please don’t hesitate to ask.

Thank you for your time.

Best Regards,
Eric [REMOVED]
Rackspace Support
(888) 480-7640

Need an answer, quick?
Use rackspace chat, now available in the portal!

Great customer support as always! Now for the response, I’m sure it will be informative:

Sean,

OpenCloud servers have different vCPU allocations than the FirstGen cloud servers, which explains the differing results you see when performing CPU-centric benchmarks. To view the vCPU allocation of our OpenCloud servers, simply go to http://www.rackspace.com/cloud/public/servers/pricing/ and expand the given server size.

A 512MB OpenCloud servers is allocated 1 vCPU, while all FirstGen Linux cloud servers are allocated 4 vCPUs (with the exception of 30GB servers that are allocated 8). Based on your results, the OpenCloud server performed as expected achieving 29% (over 25%) of the CPU performance of your FirstGen server.

Per our FAQ at http://www.rackspace.com/cloud/public/servers/faq/#overview_faq_11 the vCPU allocations were changed in order to unify the vCPU settings and scale CPU performance along with the size of the server.

If you have any other questions or concerns, please let me know!

Thanks,
Mark [REMOVED]
Linux Administrator
Rackspace Cloud Support

… WHAT?! Wait, what? You’re telling me that you’ve downgraded my server and everything is as expected. WTF? Mark closed the ticket, so I lodged a complaint to Rackspace:

I am deeply troubled by the performance benchmarks I’m seeing form my new NextGen OpenCloud servers. I feel like my servers are being downgraded, and I’m ultimately being forced to pay more for the same performance.

I’m planning on publishing my benchmarking results and would like to give Rackspace an opportunity to respond and address my concerns – all of which I will publish, along with my commentary. You may reference Ticket Number: 604951 [above] for more details.

I have always received excellent customer support and value from Rackspace, but in this matter I do not feel that my interests and needs as a customer are not being met. I suspect that once more people are aware of these performance downgrades inherent in the NextGen OpenCloud servers they will think twice before becoming new customers. I, for one, have universally recommend to friends and colleges going with Rackspace cloud servers, that however will not be the case anymore.

The only satisfaction that I as a customer, and Rackspace proselytizer, would feel appropriate in this matter would be to charge the same amount of money for the same performance. If that means you assign 4 vCPUs as a minimum, then so be it – anything less would be a downgrade!

Thank you, Sean Conner

[UPDATE: Still no response to my original complaint as of October 17th, 2012]

Here is a chat I recently to probe about getting in touch with someone more directly, rather than the provided channels on the Rackspace Website:

Welcome to the Rackspace Cloud! My name is marshall.h, how may I help you?
marshall.h: hey there
marshall.h: how can i help today?

Sean Conner: Marshall, before you go, I have a gripe that I’d like to share with you.
marshall.h: sure
Sean Conner: I’ve been experimenting with the new OpenCloud servers, and doing some benchmarking tests on them. And I’m sorry to say, it’s not good.
marshall.h: yea?
marshall.h: i hear that the legacy servers benchmark better
marshall.h: im not sure why that is
Sean Conner: one sec
Sean Conner: Sorry- call. So ya the new servers benchmark at about 27% of the old ones.
Sean Conner: It’s because the vCPU in the old ones are x4 and in the new ones are only x1
Sean Conner: My gripe it that I really feel like I’m getting a downgrade with the open cloud, and ultimately being made to pay more for the same performance.
marshall.h: yea i can understand
Sean Conner: I submitted feedback a week ago, asking for a formal response from someone with the power to effect change at that level, but nothing yet. Do you know of an effective way to contact upper management?
marshall.h: to be honest, just through ticket, or calling in
marshall.h: and asking to be escalated
Sean Conner: How high can escalation though that channel get? I would imagine the final tier of escalation through that route would be a systems administrator?
marshall.h: well we have managers that you can get escalated to, or account managers
Sean Conner: Do they have the power to assign 4 vCPUs to all of my OpenCloud servers and only charge me the same amount as my non-open cloud servers?
marshall.h: that i do not know of to be honest, i dont think they would be able to do that unless you get a larger server, which come with the x4 procs
marshall.h: like the 15gb servers
marshall.h: see even with the legacy servers
marshall.h: smaller instances dont use the whole 4 procs
marshall.h: for instance a 512mb slice will only use 3.1% of the 4 vcpus
marshall.h: with nextgen you use the whole 1x proc
Sean Conner: Well that’s puzzling, I don’t think that’s the whole story then – because when you test the old and the new side-by-side, the performance data would suggest what you said is in fact not the case.
Sean Conner: Anyway- it’s not important right now.
Sean Conner: I just wanted to the opportunity to vent a little. Thank you for listening to me.
marshall.h: absolutely
marshall.h: if you need anything else, please be sure to let us know
Sean Conner: I will do, thank you. And have a great Saturday.
marshall.h: you do the same sir
marshall.h: goodbye
Thank you for contacting the Rackspace Cloud Support Team

… To be continued

[UPDATE: I have some responses as of October 19th, 2012 -- as follows]

Hello Sean,

The first gen and next gen cloud servers actually get the same amount of CPU cycles assigned to them by the scheduler, but in next gen, it is all on one virtual core, so some synthetic benchmarks which use multithreading can indicate a false result. I can assure you that you are getting similar to same performance as your first gen servers.

The reasoning behind this change is to simplify the cloud server platform, by having a single virtual core allotment for both Linux and Windows. Each server of a specific size gets a specific percentage of the host CPU, whether on first or next gen. The only difference is the lack of multi threading.

Thanks for being a Customer!

Best Regards,
Benjamin [REMOVED], RHCSA
Rackspace Cloud Servers

My response:

Hello Benjamin, you’ll have to excuse me as I don’t fully understand how a benchmark can be “synthetic” — what would be a non-synthetic benchmark in this arena? Perhaps you could make some suggestions, preferably with clear metrics such as execution time.

I will attempt to come up with some examples of my own and post those on this ticket thread, as well as my blog post here: http://seancombinator.com/rackspace-firstgen-vs-opencloud

Thank you,
Sean Conner

The other response:

Hello Sean,

We are reaching out to you regarding your recent feedback about the rackspace-firstgen-vs-opencloud server performance.
I have contacted you via phone at 646-450-9657 and was unable to reach you.
We would like to have our Tech Team talk to you about the Server CPU usage differences and to be able to answer any additional questions you might have.

We will continue to try to contact you and in the mean time if you have any additional questions or comments please update this ticket or contact us at Support: 1-877-934-0407

Kindest Regards

Michelle [REMOVED]
Cloud Account Manager
Rackspace Hosting

My response:

Hello Michelle, thank you for trying to follow up. The number I had in my account was out of date. My personal line is (—) — —-

That said, if you’re contacting me to rehash discussions already had I would prefer you address the real issue. To bring yourself up to date on my main concern you can read my blog post on the topic: http://seancombinator.com/rackspace-firstgen-vs-opencloud

Best Regards,
Sean Conner

[UPDATE: 10/9/2012]

After several back and forth phone calls and emails with Michelle and Patrick from Rackspace we agreed that the best thing to do would be post side-by-side results for FirstGen vs. OpenCloud servers for the larger sizes. The concession that Patrick and Michelle made was that while it’s clear that the FirstGen servers far out perform OpenCloud at the smaller sizes, performance is better for OpenCloud when you measure the largest server sizes available.

Here are the results of this investigation:

RackSpace Cloud Server Benchmarks – FirstGen vs. OpenCloud – Ubuntu – 4GB RAM

 

RackSpace Cloud Server Benchmarks – FirstGen vs. OpenCloud – Ubuntu – 8GB RAM

 

 

RackSpace Cloud Server Benchmarks – FirstGen vs. OpenCloud – Ubuntu – 16GB RAM

 

So basically these results confirm my original hypothesis that switching from FirstGen to OpenCloud means a very significant downgrade in performance, with the caveat that if you choose a 16GB (15GB according the pricing chart and Rackspace’s website) or bigger server configuration you will see a noticeable bump in some of the benchmark metrics.

If you’d like access to the raw data for these last three sets of benchmarks they can be found here: FirstGen Raw Data and here: OpenCloud Raw Data

 

12 Responses to Rackspace FirstGen vs. OpenCloud

  1. Hammad October 24, 2012 at 5:24 pm #

    Here are my simple tests for FirstGen VS OpenCloud. I imaged a new centos 6.3 but after doing some testing, I think I am sticking with FirstGen for now and maybe find another alternative. If OpenCloud is a Wow then I am expecting it to blow it away with its speed as well.

    OpenCloud CPU load test:
    ———————————
    [root@borg sysconfig]# time echo “scale=5000; a(1)*4″ | bc -l > /dev/null
    real 0m42.473s
    user 0m42.309s
    sys 0m0.011s

    AGAIN
    [root@borg sysconfig]# time echo “scale=5000; a(1)*4″ | bc -l > /dev/null
    real 0m42.450s
    user 0m42.330s
    sys 0m0.008s

    FirstGen CPU load test:
    ——————————
    [root@rs-pluto sysconfig]# time echo “scale=5000; a(1)*4″ | bc -l > /dev/null
    real 0m40.218s
    user 0m39.930s
    sys 0m0.000s

    AGAIN
    [root@rs-pluto sysconfig]# time echo “scale=5000; a(1)*4″ | bc -l > /dev/null
    real 0m39.842s
    user 0m39.740s
    sys 0m0.000s

    OpenCloud Disk Write:
    ——————————-
    [root@borg sysconfig]# time dd bs=1M count=5000 if=/dev/zero of=/tmp/dd.test
    5000+0 records in
    5000+0 records out
    5242880000 bytes (5.2 GB) copied, 51.9691 s, 101 MB/s

    FirstGen Disk Write:
    —————————–
    [root@rs-pluto sysconfig]# time dd bs=1M count=5000 if=/dev/zero of=/tmp/dd.test
    5000+0 records in
    5000+0 records out
    5242880000 bytes (5.2 GB) copied, 30.4147 seconds, 172 MB/s

    real 0m30.663s
    user 0m0.000s
    sys 0m5.810s

    OpenCloud Disk Read:
    ——————————-
    [root@borg sysconfig]# time dd bs=1M count=5000 if=/tmp/dd.test of=/dev/null
    5000+0 records in
    5000+0 records out
    5242880000 bytes (5.2 GB) copied, 46.5742 s, 113 MB/s

    real 0m46.671s
    user 0m0.017s
    sys 0m6.746s

    FirstGen Disk Read:
    —————————
    [root@rs-pluto sysconfig]# time dd bs=1M count=5000 if=/tmp/dd.test of=/dev/null
    5000+0 records in
    5000+0 records out
    5242880000 bytes (5.2 GB) copied, 22.9185 seconds, 229 MB/s

    real 0m23.616s
    user 0m0.010s
    sys 0m0.740s

    • Sean Conner November 9, 2012 at 2:43 pm #

      Thank you very much for taking the time to post your results Hammad!

      -Sean

  2. Jason Jonas November 15, 2012 at 8:36 am #

    I moved my personal website (along with some web apps) into the NextGen environment. It’s a simple 512MB Linux (Ubuntu 12.04), PHP and MySQL system. Admittedly, I haven’t run any benchmarks on the system. What I’m experiencing are brief periods of no response. When talking to support on Oct 8 they indicated it’s a known issue. Here’s the chat reply…

    ——
    ok, so it does look like this is related to the OVS issue on NextGen that we have been seeing lately. This is an issue on our side, and has been brought to our engineers and developers attention to be resolved. Please keep in mind that the NextGen environment is a extremely new environment and we are finding and resolving issues on the daily basis. This specific issue is the most prioritized issue that we are seeing at the moment since there is some degradation being caused by it.
    ——

    I didn’t ask what OVS means or pressed him on the issue. Figured it was known and is being addressed.

    Using pingdom.com here’s what we’re currently experiencing regarding outages…

    https://share.pingdom.com/banners/78378e5d

    And here’s the response times we’re experiencing…

    https://share.pingdom.com/banners/e46b2bd6

    Obviously, there are issues here.

    I have a 4GB FirstGen production server that doesn’t incur an outage unless I’m performing an upgrade and forget to disable the pingdom.com check. :) The response times are between 150ms and 350ms. There’s no way I’m upgrading the 4GB server to a NextGen server right now.

    I wasn’t expecting these issues when moving to the NextGen servers and am considering moving back to a FirstGen server if this “OVS” issue isn’t addressed soon.

  3. Jason Jonas November 16, 2012 at 2:35 pm #

    An update, if it matters… RSC found an network-related issue on the host server and migrated my system to another host. Zero outages and much-improved and acceptable response times since.

    Jason

    • Sean Conner November 16, 2012 at 9:24 pm #

      Jason, thank you for sharing your your issue! I’m glad you got they figured out there was a hardware issue. A similar thing happened to a couple of my servers whilst I was on vacation. I came home to find a string of emails explaining how my servers had stopped responding, been rebooted, then there was a hardwares failure, then they were migrating them, and finally it was all fixed… and I missed it all.

      I have to admit I don’t know what OVS means either… I just searched and I think maybe it means “Open vSwitch”… and still I don’t really know what that is.

      Cheers,

  4. Sam Kim December 15, 2012 at 9:09 pm #

    I migrated my servers (about 10) to Next Gen few months back. So far I experienced host failure(3hr), ddos(few hours), network degradation(loss of packets), load balancer failure (90min), resizing failures, with less CPU power.. Hmm maybe i am the unlucky one, but I am seriously considering going back to first-gen because I didn’t have this much of problem back then. But one concern is the support quality (inc. hardware, network, tech support, etc.) of first-gen servers since they are going to retire it at some point.
    Can anyone comment on future support of first-gens?
    I like their DNS service which allows 5min TTL for free. But again their DNS is out not too long ago so I guess I will wait little more before start using them.

    I have used other public cloud services as well (still using them), but all seems not so mature at this point. Even with some problems, Amazon and Rackspace seems to provide most usable public cloud so far.. just my 2 cents

  5. David Walton January 21, 2013 at 7:25 am #

    Looks like I need to have another option ready to go. When they close down the first-gen, it will be time to LEAVE RackSpace altogether.

    They are forcing us to buy a 512MB option for 16.06/mo instead of the 256MB option for 10.95 – a 46.7% cost increase!

    And for WHAT? 3 fewer vCPU’S??? A major degredation in service?

    Almost 50% more cost for almost 75% less service?!?!

    WHAT ARE THEY SMOKING????

  6. webdesign January 21, 2013 at 12:58 pm #

    I think this link explains the crux of the issue best:

    http://www.rackspace.com/knowledge_center/comment/169448#comment-169448

    Basically they have removed burst from instances, and they are given the resources they are paying for, and nothing more, even if there are more unused resources available on the server, whereas the old servers were happy to give up unused horsepower to smaller instances, though as mentioned in the link, this sometimes proved problematic for other instances not getting the resources they should have, or getting equal resources at least.

    As stated in the link above, they are trying to provide a more “guaranteed” resources environment, where you almost always get what you pay for at a minimum, though now, you don’t necessarily get anything more than that, while if you stay with Firstgen servers, you gain that advantage of the burst, but may not be guaranteed to get even what you are paying for at all times if another virtual machine is hogging the resources.

  7. Anthony May 8, 2013 at 8:16 am #

    Looks like this blog post is still generating conversation 6 months on! Here’s something we’ve written in our Knowledge Centre which addresses some of these issues. It’s a long read but hope this explains some of your questions:

    http://www.rackspace.com/knowledge_center/whitepaper/cpu-in-next-generation-cloud-servers

    • Walker Traylor July 15, 2013 at 11:08 pm #

      The link keeps timing out. Seems apropos!

      • Sean Conner August 20, 2013 at 2:04 pm #

        Sorry Walker, I don’t check the comments too often. Which link seems to be timing out?

  8. Wale August 2, 2013 at 6:56 am #

    Gee, Thanks…

    I was gonna get a server upgrade, but at this rate, I think I’d stick to getting bigger 1st gen servers instead – until I consider scaling above 16GB of RAM

Leave a Reply