Apple’s CDN Now Live: Has Paid Deals With ISPs, Massive Capacity In Place

Since last year, Apple’s been hard at work building out their own CDN and now those efforts are paying off. Recently, Apple’s CDN has gone live in the U.S. and Europe and the company is now delivering some of their own content, directly to consumers. In addition, Apple has interconnect deals in place with multiple ISPs, including Comcast and others, and has paid to get direct access to their networks. Doing trace routes on OS X downloads from multiple ISPs now shows them coming from directly from Apple’s CDN, as you can see with the example below.

- te-0-7-0-9-sur02.lowell.ma.boston.comcast.net (68.87.159.37)
- be-21-ar01.needham.ma.boston.comcast.net (68.85.106.45)
- he-1-12-0-0-cr01.newyork.ny.ibone.comcast.net (68.86.90.253)
- he-4-15-0-0-cr01.ashburn.va.ibone.comcast.net (68.86.87.197)
- he-0-11-0-1-pe04.ashburn.va.ibone.comcast.net (68.86.82.222)
- as714-2-c.ashburn.va.ibone.comcast.net (66.208.228.214)
- usqas1-vip-sx-003.aaplimg.com (17.253.0.223)

From ISPs I have spoken with, they tell me Apple has put a massive amount of capacity in place, with many saying that Apple has more than 10x the capacity they are using today, all ready to go. With Apple planning to release the beta version of their next desktop OS today, Yosemite (10.10), and with iOS 8 expected to come out this fall, Apple’s putting in place a lot of capacity to support upcoming software releases. Apple is still using Akamai and Level 3′s CDN services for iTunes (Akamai), Radio (Level 3) and app downloads, but over time, much of that traffic will be brought over to Apple’s CDN. It’s too early to know how much traffic will come over and when, but Apple’s already started using their own CDN much faster than I expected. The pace of their build out and amount of money they are spending on infrastructure is incredible. Based on my calculations, Apple has already put in place multiple terabits per second of capacity and by the end of this year, will have invested well more than $100M in their CDN build out.

While Apple will probably never completely move away from third-party CDNs, like Netflix did, they will rely less on third-party CDNs over time, just like we have seen with Microsoft, YouTube and others. Level 3 will be able to make up for lost CDN business as they are one of the vendors that Apple is buying wavelengths, IP transit, fiber and other infrastructure services from. From a revenue perspective, Level 3 benefits more from Apple building out their own CDN and buying network services from them, as opposed to using Level 3′s commercial CDN platform. Akamai will see the most negative impact over time since almost 10% of their revenue comes from Apple and they can’t sell Apple wavelengths, transit, co-location or other network related products. When YouTube, Microsoft and Netflix all took their CDN delivery to in-house platforms, it took them about 18 months before they moved enough traffic away from third-party CDNs to impact their business. Apple has now been working on their CDN build out for about 12 months now, and are quickly scaling their network. So while I’m not predicting doom and gloom for Akamai overnight, make no mistake, it will have a negative impact on their business at some point.

It is also important to point out that decisions around who (Apple/Akamai/Level 3) delivers what (updates, streaming, apps, radio) to whom (ISP customers, different devices etc.) are under Apple’s control. These content providers manage these choices according to their business rules and usually don’t inform the ISP when something changes. If adequate server and network capacity exists, switching from one CDN provider to another (such as moving iTunes downloads from Akamai to Apple’s CDN) or changing encode rates should not impact performance, so we should not expect consumers to even notice the change. What CDN to use – internal versus external – depends on cost, contracts, capacity, and technical readiness to in source among many other factors. Companies like Apple, and previously Netflix, make variations in CDN choice by device, by ISP, by service – download, streaming, etc.

Apple already controls the hardware, the OS (iOS/OS X) as well as the iTunes/App store platforms. Right now they control the entire customer experience, except for the way content is delivered to their devices, and they are quickly working to change that. While Apple doesn’t own the last mile, paying to connect directly to it (in some places) and delivering content from their own servers allows them much more control over the user experience, especially for cloud based services. Over time, this is something that will make the experience and performance for consumers even better – and Apple’s only just getting started.

Latest List Of Vendors In The Content Delivery Ecosystem

It’s been eight months since I last updated my list of vendors in the CDN ecosystem and since then, some changes have taken place. Below is my most up to date list, with many new companies like Aryaka, Comcast, Hibernia Networks, MileWEB, MaxCDN and others having been added. (You can easily find this list at anytime by going to www.cdnlist.com)

The term CDN means many things to different people and today, is an umbrella term that covers a lot of different types of content delivery services. Video streaming, software downloads, web and mobile content acceleration, licensed/managed CDN, transparent caching, FEO, and services to measure CDN performance, load balancing and analytics and cloud intelligence. It’s a complex ecosystem with a lot of vendors both large and small. You also have some CDNs that cross over into other industries like security and WAN optimization, two segments that for the most part, are not included in my list.

This list only includes commercial services in the market and does not include companies who have their own CDNs, like Netflix, Google, Microsoft, Apple, Twitch, Facebook etc. If you think your company should be added to any of these lists, see the bottom of the post for instructions.

Vendors In The CDN Ecosystem

We hear a lot about telcos and carriers in the CDN market, but the vast majority of them have built out CDNs for their own internal use and are not selling it as a commercial CDN service. So it’s not accurate to say that they all compete with traditional service based CDNs. There are a few exceptions like Level 3, Verizon (EdgeCast), Comcast and Tata who offer commercial CDN services and compete against other commercial CDNs, but most telco and carrier based commercial CDN services are based off of reselling a traditional CDN, for example AT&T reselling Akamai. This telco/carrier list is far from being complete and many more still need to be added.

Telco/Carrier Based CDN Deployments

In addition to the current crop of vendors in the market, I think it’s important to remember how the CDN industry got to where it is today. Many CDNs raised tons of money but didn’t have a business model, some only focused on selling at the lowest price and many had technology that simply didn’t work. Lots on CDNs went under, some within a short time of launching. The CDN market has been through a lot of hard times over the past 19 years and here’s a running list of those who got acquired or went under.

CDN Related Vendor Acquisitions/Closures

Each time I make a list of vendors, for any solution or service in the market, I always get emails from companies asking why they are not on the list. If you think you should be added to the list, please add it to the comments section but note that I am not listing regional hosting providers or companies who get most of their sales from $100 a month customers. Also, just because you are not on this list doesn’t mean you don’t have a valid solution in the market, but the companies listed are the ones I get asked about most often, get mentioned in the media, are included in major RFPs and promote and market their services to medium and large customers.

Limelight’s Network Outage Shows The Importance Of Third-Party Performance Benchmarks

About two weeks ago, on Thursday July 17th, Limelight Networks suffered a major outage of their network that for some, measured 76 minutes of downtime. I got multiple calls from customers about the outage and Cedexis, a company that provides performance measurements from the standpoint of end users around the world, discussed the details of the outage on their blog. While they didn’t name the CDN, it was Limelight Networks, and the outage was so widespread that Cedexis said, “Little to no traffic was being passed at this time from this CDN. If this were a storm in the Atlantic it would get a name.” For a breakdown on the outage, visit the Cedexis blog post.

It’s a shame that Limelight had such a major outage, considering they are once again trying to get customers to think of them when it comes to being a reliable CDN with good performance. However the bigger point is not that CDNs have downtime but rather that when it comes to customers measuring performance amongst CDNs, it is crucial that they use a third-party service that takes real user measurements (RUM). On the day of the outage, Cedexis collected more than 2 billion measurements across 40,000 networks and 100 cloud and CDN providers worldwide. This type of measurement data is crucial for content owners, especially the large ones, who use a multi-CDN strategy and want to load balance amongst CDNs based on real performance data, from actual end users.

Cedexis has a platform that many content owners now use, called Cedexis Openmix, that allows them to improve the availability, latency and throughput of their content, website and mobile apps. It allows them to dynamically ingest this real user measurement data as a way to allow content owners to load balance across multiple CDNs and avoid outages. In addition to content owners, all of the CDNs now use Cedexis to compare their performance against their competitors and some even use Cedexis performance data in their service level agreements (SLA) with customers. While Cedexis may not be known by many outside of the infrastructure market, the company is quickly becoming part of the fabric of the CDN ecosystem and I would argue, the most important piece. If you can’t measure your performance, for any service, then you don’t really know what you are paying for and what you are getting in return.

More and more customers I talk to, especially the larger ones that CDNs would define as enterprise, are now using Cedexis. Founded in 2009 by two Akamai veterans, Cedexis is really starting to gain traction in the market and while they aren’t as big as Gomez, Catchpoint or Keynote, they are much more focused and have much better insight into what’s really taking place on the Internet. Gomez, Catchpoint and Keynote aren’t inside the last mile networks, don’t have buy in from cloud, backbone and infrastructure providers and take only a small number of real user measurements per day. They can’t truly compare network performance amongst CDN and cloud providers and Cedexis providers far superior data and analytics to what others offer.

If I had to make a list of the top five hottest companies right now, Cedexis would be at the top of the list. Not only because of what they are doing, and how important it is to the market, but also because they share so much of the data – for free. You can look at the results of a few of their measurements taken over the last 24 hour period here and more data is available within their free benchmarking community, which you can sign up for on this page. It’s only a matter of time before someone in the market acquires them and makes their data and methodology part of a larger platform of web performance monitoring.

Cedexis did a recent presentation at the Content Delivery Summit in May and has a really good slide deck you can download, with lots of details on how they measure, what they measure and their methodology. You can also see a video of that presentation below.


ChinaNetCenter Enters The Mobile Web Acceleration Market

As the market for web acceleration services continues to grow, driven primarily by mobile usage, ChinaNetCenter has thrown their hat into the ring with a new mobile application acceleration solution. The company, which has more than 500 POPs in China and more than two dozen POPs outside of China, is using an adaptive policy based solution. It integrates their mobile application acceleration SDK, what they call mSDK, with an intelligent context aware technology designed to get information from the mobile end, such as the brand, OS version, network type and app’s quality of service.

The company says it’s very easy to integrate their mSDK to iOS and Android and that it takes one developer less than 30 min to integrate it into their native app with less than four lines of code. Because all of their CDN cache servers have what they call an mSmart component that communicates with the mSDK, it allows them to optimize every TCP/UDP connection. Their platform also decides the priority of the content being delivered, ensures that the highest priority content is transmitted first, and selectively queues content in the system buffer.

The company said they found that regular connection optimization methods in the industry like TCP/Http Keep Alive and Google SPDY contributes very little improvement, because they have the side effect of involving more competition for the limited available bandwidth. So ChinaNetCenter uses a proprietary intelligent adaptive protocol to control the connection that decides the adaptive policy for transmission and content and optimization parameters based on environment information gathered by the mSDK.

By the end of 2013, China had over 618 million Internet users, 80 percent of which access the Web via their smartphones. At the same time, a huge amount of startups are bringing mobile apps on the market and traditional enterprises are also working to migrate existing application to the mobile platform. In early 2013, one of the biggest online map application companies came to ChinaNetCenter for help to improve their mobile end-user performance. The customer complained that while they were already using a CDN, when they looked at the performance numbers, it simply wasn’t effective for mobile clients. They were considering disabling mobile viewing all together as they didn’t want to provide a bad user experience.

This is something I hear often from content owners, that traditional CDNs aren’t providing a true platform for mobile content acceleration. While some CDNs like Akamai have been in the mobile acceleration market for a while, and smaller CDNs like EdgeCast and Fastly are also focusing on this segment, most CDNs don’t really address the performance issues around delivering content to mobile. It’s one of the reasons why new startups like Instart Logic have emerged in the industry, specifically to focus on improving the performance of content delivered across mobile devices.

ChinaNetCenter said their customer’s install base of 100M users for their online map application app saw an average performance gain of over 40%, and in some low speed wireless network environments, the performance gain was as high as 100%. Earlier this year, the biggest online reading application in China also started using ChinaNetCenter’s mobile application acceleration platform and saw their app’s utilization ratio increase by 20%. Another customer, a local airline in the country, saw their order rate conversion increase by 10% through their app’s performance improvement.

For web browsing, mobile content consumption is the future and multiple vendors are now focusing on acceleration technologies to address the specific issues caused by wireless networks, which tend to have high latency and a large percentage of packet loss. With their new solution in the market, we can now count ChinaNetCenter as yet another vendor who has recently entered the mobile application acceleration market, with more to come.

ChinaNetCenter is the leading CDN provider in China with revenue of $197M in 2013. Backed by ChinaNetCenter, MileWEB was established in 2012 as the international development partner tasked with globalizing their next generation CDN services and solutions.

Aereo Had 77,596 Customers At End Of 2013: Didn’t Understand The Market

Documents filed by Aereo with the U.S. Copyright Office have finally disclosed how many subscribers they had for their streaming service and the numbers show, it simply wasn’t a service that many consumers wanted. At the end of 2013, Aereo had only 77,596 total customers. About 27,000 of those were in the NYC area, 12,000 were from Boston and 10,000 from Atlanta. For all the talk by those in the industry and members of the media of how “innovative” or “disruptive” Aereo was, the reality is that the only number that matters for a subscription business is how many customers it has. That’s it. That number determines revenue, P&L and whether or not the company survives with their offering in the market. [See: Barry Diller’s OTT Service Aereo Is Dead On Arrival]

Aereo failed because they didn’t understand the market they were in and set expectations with themselves, that were completely unrealistic. They argued that the cable companies model of bundling channels was bad, yet Aereo themselves bundled all their channels into one monthly fee as well. For all the talk by Aereo of a la carte, Aereo didn’t allow you to pay only for the channels you wanted. You had to pay a monthly fee to get channels broadcast in foreign languages, even if you couldn’t watch them. In reality, Aereo was just like the cable TV model, but with less choice, poorer video quality, less device support and fewer options overall.

The CEO of Aereo said they could sign up 350,000 subscribers in the major cities, yet the company ran out of capacity and had technical issues in NYC, when they had less than 7% of that number of customers. When management and Barry Diller are out in the public saying that between 25M-30M people wanted their service, it shows just how out of touch they were with the market. Consumers want choice, they want subscription services with a deep catalog of content to choose from. They want high-quality video, with support on all devices. Aereo didn’t have any of these with limited device support, small number of channels and streaming video quality that was on average, a third of what services like MLB.TV offer today.

Aereo raised $100M, yet couldn’t even scale the business to 100,000 subs. At 100,000 subs, that would be less than one tenth, of one percent, of the cable TV subscription market. Aereo didn’t fail because of the U.S. Supreme Court ruling, Aereo failed from day one because it was selling a service in the market that not enough consumers wanted. Aereo should be a lesson to others that just because technology allows you to bring a service to the market that does not mean the service is something consumers want, or are willing to pay for. If there is no tangible business model behind the technology you bring to the market, then how the service works does not matter. Aereo put technology before business, and that always leads to failure, when it comes to consumer services.