Chart Shows Which Content Owners Have Direct Interconnect Deals With ISPs

With all the confusion in the media around the topic of interconnects, I thought it would be helpful to show exactly which content owners have done direct interconnects with ISPs. While some members of the media want to claim that content is exchanged on the Internet in a “secretive nature”, the fact is, it’s not a secret at all. If you know how and where to look, you can easily see what is taking place between networks. All of the information I provide in the chart below is from public information that anyone can lookup simply by knowing the networks Autonomous System number (AS) and looking at their peers. You don’t have to be a networking engineer to look at BGP relationships and see what is taking place. It’s all public info.

The search results won’t come right out and say which relationships are settlement free versus settlement-based, but you can infer from traditional business relationships and public peering policies, who’s paying whom. Not to mention, the companies who pay for these interconnect relationships will tell you which ISPs they are paying, and which one’s they aren’t, if you have a good relationship with them. Anyone involved in building out networks for a living understands how this has worked since the late 90’s. It is not a secret. Many members of the media simply don’t have any insight into these deals because many of them don’t cover infrastructure topics for a living. But just because they don’t know themselves, does not mean that these deals happen in secret.

Based on the AS lookups I did, here are the companies that have direct interconnects with various ISPs. I didn’t look at every ISP out there or every content owner, simply some of the larger ones.

Screen Shot 2014-05-20 at 8.56.01 PM

Updated 6:08pmFor those asking why Level 3 is on the list when they are not an ISP, I wanted to show that there are many forms of direct business relationships between companies for both transit and paid peering.

What you can see on the chart is that all of the major content owners have negotiated direct business relationships with multiple ISPs for their CDN. There are various business and technical reasons why they pick one ISP over another, but there are a lot of these deals in the market and have been for many, many years. Some ISPs in the U.S. have dozens of paid interconnect deals in place. Multiply that by the number of ISPs in the U.S., and the number of paid interconnect deals is probably close to 100. So for Netflix’s CEO to call their paid interconnect deal with Comcast “unprecedented,” is intentionally misleading.

Each of the X’s in my chart shows that large content providers are unafraid to connect with large ISPs, most have been doing so longer then Netflix, and none have found the need to seek intervention. Of course that does not tell you the size of the deal, how much the ISP is charging or any of the other business terms, but no one should argue or debate that these deals aren’t common, continue to happen and are how networks, especially in the U.S., have connected to one another for almost twenty years.

Part of the argument by many around interconnects is that it costs the ISPs almost nothing to turn up ports on their network, or they say that ISPs have “unlimited capacity”. Neither would be accurate. There is a direct cost to the ISP to add ports and backhaul infrastructure to bring traffic all the way into their network. For anyone to say that the costs should now shift to the ISP because it’s “cheap” to them, or “doesn’t cost them much”, it’s not a valid argument. If that’s the argument some want to use, then by the same token I could say that if I have a letter to deliver to someone’s house and the postman is already going there, then it costs them next to nothing to deliver my letter and my cost should be free. It’s simply not a logical argument.

Put it another way. What if Google, Apple, Netflix, Facebook, Ebay and Microsoft all said to the same ISP that they want unlimited peering capacity, in multiple cities in the U.S. Would anyone think that would be fair to the ISP and that they should have to spend the money to fulfill all of those requests, at no cost to the content owner? Would this entitlement give major content owners special treatment over a smaller startup competitor? I think most would agree that’s not fair, yet that is what many are suggesting should happen. And using the argument that the ISP is already charging consumers for access to the Internet is also not valid. Your connection from your ISP is about broadband Internet. It is not about the service infrastructure cost for third parties.

Historically companies like Netflix have always paid for their transit connectivity, or Internet delivery costs. The 60% of broadband customers who do not use Netflix, should not suddenly have to pay for the network transit/interconnect costs of others because Netflix feels their Internet costs should now shift to all ISP customers. Perhaps someone wants to use Hulu, Amazon or consume no video at all. Each of these companies has a cost for their service infrastructure and paid interconnect deals are not increasing that cost. These costs are staying where they have always been, included within the specific service. Saying that it is new or unprecedented is just not true.

Follow up questions can be put in the comments section below or you can email me or call me (917-523-4562) at any time.

  • Raindeer

    I’m sorry, but this bogus.

    There are a number of problems with your analysis. The first thing is the difference between peering and transit. Transit allows a company to reach the whole world. Peering reaches only the the network you’re peering with and its customers (if it has them). Almost all the ISPs you mention have extensive transit businesses. AT&T, Level 3, Sprint and Verizon are all in the Baker’s Dozen Renesys publishes as the largest Transit providers in the world. Level 3 is exclusively providing transit and will only peer with networks that reach parts of the world it doesn’t reach. Centurylink bought Qwest and Savvis and is massive in the US, just not in the top worldwide. There are not many that can claim that. Only Comcast is a bit less of a transit player. And when we analyse route propagation than it becomes clear that all you did was analyse who buys transit from whom, not who pays for peering.

    Pandora, you register it as having peering with Comcast and Level 3. However when I look it up on http://bgp.he.net/AS40428#_graph4 (use with great caution) it shows that it has a number of transit parties that it buys traffic from. Comcast and Level 3 act as transit parties. From Comcast the traffic goes on to Verizon. That clearly shows that Comcast is the transit party as there would be no reason for Comcast to allow Pandora to traverse its network for free.

    Apple is an AT&T transit customer according to http://bgp.he.net/AS40428#_graph4 (you can use the graph for AS714, but it is harder to read). I cannot see Apple buying from Verizon in this route propagation, so you may have found one there, but even then, you don’t know the contract, so it may just be that Apple does have a peering, because, you know Apple and Verizon are business partners these days, selling phones.

    Level 3 is not doing paid peering with any of the ones you suggest. Level 3 is the biggest transit network in the world. If a network wants to reach Argentina, South Africa or Myanmar, Level 3 is a safe bet as the one being able to reach the other network, because it often is their provider, or their providers provider. So why you include Level 3 in this, is unclear.

    Facebook to Comcast shows as a peering (no ongoing arrows), however we do not know the conditions. But that might be a safe bet.

    Limelight, a CDN, might pay. But maybe not because it wants to. But being a CDN today is hard. It is a murderous tough competitive business. Algorithms rule, if you don’t deliver the traffic fast enough, the algorithm of Cedexis and Conviva will choose another CDN, because many large scale sites use multiple CDNs. So traffic goes to the fastest. This gives terminating networks a lot of leverage over CDNs. Particularly if the terminating networks are very big and face little competition. So in Europe such a strategy might work less well for the terminating network. European ISPs are often smaller, they have more competition to deal with. Customers can leave to other operators. And so we see a lot of peering.

    Out of 144,000 peering agreements of 4000 networks studied by Packet Clearing House only a few hundred were paid for. That is quite a number and doesn’t resonate with your analysis.

    The main reason why networks peer, is because it saves on transit. Your argument for paid peering is therefore a strawman. Networks do not want something for free when they peer. No both peers save each other money. The best example I can give is when Google came to Kenya and traffic at the Kenya IXP rose by 600mbit/s. Transit costed at that time between 100 and 600 dollar/mbit/s/month. So Google peering saved local ISPs between 60K and 360K per month. That was money that they didn’t need to pay to get the traffic from London. That was money they could use elsewhere to upgrade their networks. It works the same everywhere. In Europe, where the markets are smaller and the competition fiercer, you see excellent peering communities.

    AMS-IX and LINX celebrate their 20th anniversary this year, they have hundreds of connected networks handling 2Tbit/s each. Many of those hundreds of networks, even well known ISPs such as Virgin and BSkyB have open peering policies and will connect to routeservers that set up automatic peering. The US is different, it has very big ISPs and less competion in local markets. That doesn’t mean the rules of the game are different. And for many ISPs in the US it is clear they do have settlement free peering. Just look at RCN and Netflix. Most of these do not have large transit providers as part of their business.

    Your analysis and knowledge of the market is severely lacking. It is wrong and it is a pity that you do not see that. It is even more a pity, because people get wrongly informed and this hurts a proper debate about the state of the market and where it is going. Unfortunately even when seasoned peering managers tweet to you, you do not acknowledge their knowledge.

    • Whatever

      You seem to be picking corner cases to justify a company position. Isn’t it the “customer” choice to pick what level of service they want from their provider? I think each of these ISPs offer a range of services from local to full transit.

    • danrayburn

      Raindeer, in response to your comment, and your recent phone call to me, you are arguing that I should be explaining what transit it. That’s not the purpose of this post. I’ve already done a whole post on that topic which you can read here:

      How Transit Works, What It Costs & Why It’s So Important
      http://blog.streamingmedia.com/2014/02/transit-works-costs-important.html

      As for Apple, the delivery requirements of Apple’s transit providers are not global. It is server-to-router in a data center, so it’s not like Apple’s transit provider is hauling stuff around the globe for them. That is why Apple is building a CDN, to serve traffic locally for performance and leverage ISP relationships to deliver traffic. That would mean Apple is probably using it exclusively for on-net.

      If you assume large CDNs have multiple transit providers, the reality is each transit provider is delivering only to a handful of designated ISP destinations. They are not sending traffic everywhere. This is called partial transit. Think about Akamai, they have lots of free peering with small and mid-size ISPs, they have some paid peering and the remainder is handled by many transit providers. They may say NTT, you deliver to yourself and to X and Y, not the whole Internet. Then say to XO deliver to yourself and A and B.

      Think about how NF used Cogent, not to deliver to the whole Internet but to congest a handful of destinations, drive rankings down for certain providers and as a peering playbook tool. Cogent was not delivering to NF’s peers, not delivering to Level 3, not to Comcast or Verizon now, or Europe where NF has lots of peering so it’s a limited routing table not the global Internet.

      You’re also asking me to disclose which companies I have spoke to and what they have specifically told me in relation to buying transit versus paying for interconnects. I’m not going to do that.

      You also make it clear in your written response that I don’t have details on who’s paying whom and say I “don’t know” what the contracts look like. How do you know that? Again, you don’t know who my sources are or what info I have. Insisting that I must share that info with you is not relevant. But I’m not the only one who speaks to them, others know as well where money is exchanging hands.

      You say Limelight may pay, “but maybe not because it wants to”. This post isn’t about who wants to pay versus who doesn’t, that’s not what the focus of this post is about.

      Giving me examples of how it works in Europe, Africa or some other part of the world isn’t relevant for what I am looking at. I’m writing and talking about how it works in the U.S., which I agree, is different than how it works pretty much everywhere else in the world.

      And yes, when you tweet me 15 times in one day, I’m not going to send you 15 responses. I replied once, but I don’t have conversations on Twitter, that’s what I use email for.

      I did add a comment to the post to explain that while Level 3 is not an ISP, I listed them to show that there are different “direct business relationships”, which is what the post talks to.

      As for your argument on the phone that I am responsible for a website other than my own, taking something from my post and getting it wrong, I spend a lot of time to try and make sure that anyone who uses or publishes anything from my post does so correctly. But I can’t control a third party website.

    • Big E

      Lighten up dutch. Dan is giving a point of view, and he’s covering a lot of different areas. Of course he’s never going to be 100% accurate, and neither are you. Just look at your argument on “CDNs being ruled by Algorithms, Cedexis and Conviva”. In this respect, you are flat out wrong, and have no idea what you are talking about. It tells me you haven’t worked for a CDN. For your information, Cedexis and Conviva are probably about 1% of the CDN ecosystem. So drink a beer and chill out, and Dan – keep doing what you do best.

  • http://www.ivpcapital.com/blog Michael Elling

    The table is a series of apple and oranges. Level3 is not the same as edge access providers (who happen to have their own transnational transit networks). It is a core transit provider like Cogent. Likewise Akamai and Limelight are not content or app providers/aggregators. They are CDNs that either own their own transit or buy from the competitive core transit providers and then cache content (mostly ads delivered) at or near the edge.

    What you should be asking is why is the transit or CDN market becoming non-competitive? I say this because only in a non-competitive market does an app provider/aggregator feel compelled to vertically integrate and build their own delivery networks.

    As for your cost argument, none of the content or app providers particularly care about latency; even Netflix. But they do care about transit costs and they are in the best position to decide on layer 1-3 tradeoffs than the balkanized edge access providers, none of whom own more than 40% of any given market.

    As I’ve said elsewhere, this is not an issue about current supply/demand, rather where should the WAN/MAN demarc be in a world of 4K video, 2-way HD collaboration, seamless mobile BB, and IoT. All of these need cloud economics moved to the edge and the edge access providers are merely trying to prevent that from happening. Been going on for 100 years in one fashion or another. The internet was an accidental result of one of these; namely flat-rate expanded calling areas by the Baby Bells.

  • Mbps

    There is a lot of precedent for content providers paying transit providers and CDNs to get their traffic to the last mile of the Internet. What is too common these days is ISPs intentionally congesting their links with transit providers like Cogent so the Internet does not work, making an uncongested link to these ISPs captive end users unavailable—creating a slow lane. And what is unprecedented is some content owners, like Netflix, being forced to pay these ISPs for a fast lane to avoid the congestion the ISPs themselves are causing.

    As I understand the scenario, Netflix must deliver its traffic all the way to these ISPs local markets—so they are not buying any “transit” service from these ISPs—Netflix itself is providing that function—then paying the ISP a fee for no service whatsoever, but rather, a fee simply for access to the ISP’s customer.
    A customer, by the way, which the ISP has already sold the right to receive all of the lawful content on the Internet. And as to “expense”—as evidenced by the fact that Netflix’ congestion problems were instantly solved the minute they agreed to start paying Comcast— adequate network capacity already existed on Comcast’s network (so there was no added cost), and adding an additional interconnection port to alleviate congested links costs around 5-10k, NRC.

    • anon_coward

      cogent should have just bought some more ports instead of sending all that traffic over a free link with agreed traffic ratios. and the ports aren’t that expensive. and comcast had no issue with it

      instead they are losing a huge customer and setting a precedent of ISP’s peering with edge providers

  • Mark Able Jones

    With net neutrality, it makes financial sense for ISPs to peer at no cost because it’s the most efficient method of delivering the data.

    Without net neutrality, ISPs will refuse peering, force congestion, and make money of the inefficiency by selling prioritization.

  • Reed

    We live in capitalistic economy. If you use 32% of all internet bandwidth in the US, maybe you should start paying for some of that usage that has allowed your executives to all make millions and stop whining trying to get everyone else to pay your bills!

    • James Hutton

      Yeah, but the customer that is requesting the days already paid and so has the content producer. This is about a third charge to a company that has already been paid for the traffic. It is not like the content producer is pushing the data the customer is pulling it. A lot of the concern is that many who use Comcast cannot use the free market option to switch providers when their isp makes decisions and policies they don’t agree with.

  • capella

    “Peering at no cost” may be the ultimate win-win for two partners doing business, but is not the economic reality. There are financial inequalities in real-world relationships. Claiming that intelligent companies will refuse peering unless forced to for free under the banner of “net neutrality”, seems to ascribe nefarious motives that I don’t understand.

  • Justin W.

    Dan: Can you explain what it is about Comcast and Verizon’s architecture where they need to recover funds by charging content companies for peering? Cox/Charter/RCN are doing it for free, and also pass millions of homes. Thank you.