Content Owners Struggling To Compare One CDN To Another

I've blogged before about how most CDNs don't give out data points in the market or to customers for them to try and fairly compare one CDN to another. It's impossible to measure performance, capacity and many other aspects of a content delivery service offering from one provider to another. I got an e-mail this morning from a content owner who summed up what I hear all the time from customers:

"How do you think we should proceed looking for a new CDN and how do we get any data that will help us determine who we choose? All the CDN sales reps are blaming the other one in terms of network size, service quality, etc. and we can’t see the difference between many CDNs. Do you have research in terms of how much bandwidth, capacity, scalability, performance, etc?"

I know many of the CDNs hate these questions and say it is not fair for them to have to answer them since their capacity constantly changes and they are all making upgrades to their network footprint all the time. But why can't CDNs give some sort of metrics on capacity per format in each region of the world? Look how many CDNs say they are global when they truly aren't. Many CDNs that are in the Adobe partner program and listed on the Adobe site as supporting live Flash video don't support live. And how is it that every CDN, large and small, global and regional, has gotten an A+ ranking from Keynote on their network quality?

I am not the only one who notices this stuff. If any content delivery network thinks I am pointing out things customers don't already know, I'm not. I hear these kinds of points from customers all the time. And with more CDNs in the industry and more competition it is harder than ever for content owners to try and figure out who really does what in the space and what any one company's limitations are.

While I would propose some suggestions to the CDNs of the type of data they should talk about to fix this, it's pointless as it would then clearly show the differences between the networks. And too many of the CDNs want to be compared to all the CDNs as being on the same level playing field when in fact, they know that many times they aren't. I don't think there is anything wrong with not being as big as someone else or having as much capacity as someone else. Why would you want to tell a customer you are global, sign them up, and then have them find out the hard way that you really aren't? Why start off the relationship on the wrong foot? This happens much_too_often with customers.

CDNs should highlight what they do well, what their core strength is and what type of customers they should and more importantly should not be going after. If MLB.com were to put out an RFP tomorrow for all of their video delivery needs, I bet nearly every CDN would want to bid on that business, even though those of us in the industry know there are realistically probably only 2-3 CDNs today that could handle that level of traffic all at once.

The CDN facet of this industry has to get smarter and providers have to start evolving much faster in terms of the message they are delivering to customers. The service is already starting to be seen by many as a commodity. Now is the time to make it clear to the market and content owners what your real strength is. Looking at almost all of the CDNs websites, it is nearly impossible to find out what verticals they specialize in, what format(s) they support, what regions of the world they have delivery in (network maps don't count), what type of reporting they have, (put up a demo account on your home page), what your message is to the market, (speed, reliability and global reach don't cut it anymore), who your customers are, (case studies please) what your products are in detail (where are your product sheets?!) and for those that say they help content owners "monetize" their content, show examples of exactly what that means.

A CDN that delivers bits and who sells content delivery only with no tools or applications, that delivery service is not a monetization service. Pushing bits is not enabling content owners to monetize their content. Giving them tools or providing services to do targeted delivery, advertising, DRM and very granular reporting - those are monetization services. Simply shipping bits is not.

Too many of the CDNs are so focused on only using networking language right now that they are not delivering any real marketing message to customers. Go to the websites of the CDNs and look at the message on their home page. The majority of them, but not all, are not delivering any clear concise message with any real identity.

In my eyes, the content delivery market is going into the next big stage as we will see more growth in the next 2-3 years than we have seen in the past five years. Now is the time for CDN vendors to deliver a very clear and concise message to the industry and to customers of exactly what it is they offer and what the differences are between their company and others.

Comments

[Harold Schwartz]

I think that at least one CDN gives voluminous data about their performance. Internap Network Services has now applied the 100% SLA's to their CDN offering

[Dan Rayburn]

Having a 100% SLA does not speak to performance. Look at nearly every CDNs SLA. What is it that they are guaranteeing? That the network will be up? The server will be online? Will be available? And notice most SLAs for video delivery don't say how you are compensated with outages? They also only kick in if the customer notices the outages and are usually not pro-active SLAs, only re-active.

Anyone can guarantee anything they want, but what customers want SLAs to contain is how they are compensated and have the SLA be proactive. Most don't do that and the customer has to ask for additional language to be added to the SLA.

To date, I have never heard a single customer tell me that they pick a SLA of 100% over an SLA of 99%. Looks good on paper and to marketing departments, but it's not making customers choose one CDN over another.

[jb]

Dan - is there not methods using Gomez / Mercury or Keynote that could give statistical info on the performance? I know Gomez works well for objects.... ????

[Dan Rayburn]

I will be doing a post shortly about the problems with Gomez, the biggest of which is that every CDN you talk to tells you and wants to show you a graph that shows Gomez ranking them as the fastest. I am trying to get to the bottom of that.

[ojmd]

I totally agree, CDNs have not differentiated themselves at all. As a buyer, I simply go with the cheapest one among a pool of reasonable suppliers ("reasonable" being my gut-level opinion based on size, appearance of stability, sales rep's haircut, and other meaningless factors). To take the lack of differentiation a step further, I am sort of wondering if CDN is even worth a premium. Our CDN contract is up for renewal and I'm thinking about Amazon S3 or high-end managed hosting. Maybe it's just my technical ignorance, but I honestly can't tell if CDN is any better than those much cheaper options. I would like to think that I haven't been paying a bunch of money for nothing, but I'm sure that people who buy Armani think that as well. Any thoughts?

[Jeremy]

I also agree with everyone else here in that CDN information is very non-transparent, and to get pricing and other critical information you must go directly through each company's sales team. Most sales people are very nice but you can see there is pressure on them to sign up new clients. They are also willing to negotiate on pricing, contract terms, and other important buying decisions since competition is heavy.

I agree that the space is becoming quite commoditized and a consolidation will likely take place over the next 1-3 years, with many small players likely getting squeezed out.

I didn't choose to go with any CDN's for my video website right away as I want to scale and grow into a CDN solution. I would advice many others to do so as well if you are a startup on a small budget since the average starting price is quoted at $500/month based on a 6 month term (a $3,000 commitment)

Happy CDN hunting :)

[Jeff Geiser]

Hey Dan,

As you and I have discussed, I think this is an interesting topic. Using Keynote's Streaming Perspective service we have worked with countless companies to cost effectively evaluate CDNs and their relative performance.

As you know, the quality and availability of content delivered from a CDN varies based on a large number of variables. Our services allows companies to test the relative performance of their content (with these variables customized to their needs) and get a real-world view of how each CDN's performance compares.

By the way, we have not rated every CDN an A+.. far from it. You must be referring to the Microsoft Windows Media program which only includes a handful of CDNs and is specific to live windows media.

Looking forward to catching up soon,

Jeff Geiser
Global Director, Streaming Media
Keynote Systems, Inc.

[NikolaD]

There are substantial difference between cdn structures used by different providers. As such providing raw data like geographical presence or access point linkage is often non comparable between different CDNs. For example I'm doing some research for a smaller european company in the business. We wouldn't want to be compared to Akamai in raw presence for example. Or to the free service of youtube in pricing for that matter. There are way too many other factors that users should consider. Most importantly does the given service really work for them. Your example with monetization is a fine one. For that matter monetization is not only PPV or in-player/in-page ppc advertising, for which very few services offer any working solution at all.

You might want to take a look at our service for a different user level approach on the video delivery in the UK market: http://www.streamingvideoprovider.co.uk/

Your analysis of marketing approach is a helpful one(though there are many more points to consider) and we've taken point in it. Thanks.