Microsoft Research Paper Measures Limelight and Akamai’s Network Performance

At the end of this month, folks from the Microsoft Research team are presenting results at the Internet Measurement Conference from a recently completed CDN study with the Polytechnic Institute of NYU entitled “Measuring and Evaluating Large-Scale CDNs“. While the event is not for a few weeks, the technical study is now public and available from the Microsoft Research website. Having read through it, the paper is one of the most comprehensive technical white papers I have seen on the subject in quite a few years that is available in the public domain. (UPDATED: On October 15th, I was informed by Microsoft Research that they “decided to withdraw the paper from the Internet” and as a result, I was required to remove my links in this post to the paper.)

For the study, Microsoft Research and the Polytechnic Institute of NYU conducted extensive and thorough measurements to accurately characterize the performance of Akamai and Limelight. Their measurements included charting the CDNs (locating all their content and DNS servers), assessing their server availability, and quantifying their worldwide delay performance.

The purpose of their measurements was to shed light on two radically different design philosophies for CDNs: the Akamai design, which enters deep into ISPs; and the Limelight design, which brings ISPs to home. They compared both CDNs with regards to the numbers of their content servers, their internal DNS designs, the geographic locations of their data centers, and their DNS and content server delays.

In speaking to one of the authors of the paper yesterday, he commented that their measurement techniques can be adopted by CDN customers to independently evaluate the performance of CDN vendors and can also be used by a new CDN entrant to choose an appropriate CDN design and to locate its servers. While the Microsoft Research team was originally going to study hybrid CDNs and P2P delivery, they quickly decided that comparing Akamai and Limelight different network architectures would be a great subject to tackle.

While some might be under the impression that the divisions inside Microsoft that use Akamai and Limelight for delivery commissioned the paper, this is not the case. They were certainly interested in the results, but the report was initiated solely from Microsoft Research and the Polytechnic Institute of NYU.

I will be highlighting some of the findings of the technical paper in multiple blog posts over the course of the next two weeks.

  • shaun nol

    great Dan,
    will be anxiously awaiting the results and analysis of this paper as this is basically exactly what we were talking about before, this is going to be VERY interesting….

  • grinsandfun

    It is interesting to see the “Technical review” vs. the “Marketing pitch” on AKAM. I also found it interesting that you would commonly hear about route optimization from AKAM (but it is apparent they are routing on cost vs. quality from this paper) also 36K servers – but it seems they take down a considerable chunk to maintain the environment. Also interesting to read about the virtualization usage on the core for some functions…
    As for the LLNW part – seeing the international footprint – quantity of servers really exposes a weakness if your content requires pushing packets that direction… (esp in Asia (less than 250 for the whole region) / Australia (1) & even Miami (111) which they serve most South America countries from)
    This is a very interesting read & information… thanks Dan!

  • Gifford Hesketh

    It is also worth mentioning that the data in the paper pertains only to those “LDNS” servers (or “resolvers”) open to outside queries, and is unweighted with regard to the volume of actual clients associated with them.
    A substantial number of resolvers that serve very large Internet communities (e.g., cable modem and DSL subscribers) are NOT open to the public, so the report is skewed unrealistically.
    This does not necessarily mean that its findings are inaccurate, but its methodology has an intrinsic filter and bias that may not be obvious to everyone.

  • term

    Certainly looking forward to the results and more great posts from you.
    Thanks Dan.

  • spenser

    Too bad about the paper being retracted. This is about the fourth site I have visited trying to track it down.

  • RPS

    very interesting blog and a good posting !!!

  • Kelvin

    Microsoft has done a very great job of ensure that all students are better able to make use of microsoft word to correct their papers. I am a specialist of one such program that is meant to proofread and grammar check their content.