How Transit Works, What It Costs & Why It’s So Important

One infrastructure service that’s gotten a lot of coverage in the media lately is transit, with many using the term incorrectly or defining it as something it’s not. I thought it might be helpful to explain what transit it, the different types of transit services sold, a list of providers who sell it, what it costs and why it is so important to the Internet. There are a lot of pieces that make up the Internet including products like wholesale, transit, wavelengths, backhaul and others which all share the same underlying optical transport infrastructure, which is the foundation for all Internet and IP services. Many of these terms are used interchangeably, but they shouldn’t be as they all provide a very different function in the market.

In its simplest definition, transit is a “network that passes traffic between networks in addition to carrying traffic for its own hosts”. The Internet is made up of a collection of networks, and in order to get traffic from one end user to another, all service providers, hosting providers and ISP networks need to have an interconnection mechanism. These interconnections, which allow the sharing of traffic, can be either direct between two networks or indirect via one or more other networks that agree to take the traffic. Many of these network connections are indirect as most providers don’t have a global network footprint and as a result, the traffic will be sent through several different interconnections to reach the end user.

The commercial interconnect relationships that allow networks to directly and indirectly connect are referred to as peering and transit relationships. While both those terms are often used interchangeably, they aren’t the same thing and they are many flavors of each. Peering is when two or more networks interconnect directly with each other to exchange traffic. While many think peering is “free” to both networks, that’s almost never the case. Like transit, there are many types of peering both public and private, and paid and settlement free. Peering is between two networks whereas transit allows you to connect to multiple networks.

Transit is where one network agrees to carry traffic that flows between another network and all other networks connected to it. No single provider in the market connects directly to all the other networks on the Internet, so any network that provides transit will deliver part of their traffic indirectly through multiple other transit networks. Transit providers’ routers lets other networks carry traffic to the network that has bought the transit and get a fee for that service. It sounds complicated, but really all the transit provider is doing is allowing multiple networks to exchange traffic with one another.

Some have written that transit allows two networks to exchange “bandwidth”, but that’s not accurate. Transit allows providers to exchange traffic, but bandwidth and traffic are not the same things. When it comes to how transit is sold, companies can buy full transit, partial transit, select routes, on-net routes, etc. and ISPs will create the service and pricing around the customer request. Transit deals vary greatly, in size, type, price and performance and are not a one-size-fits-all model. Many transit deals are alike, but transit relationships also vary greatly based on the region of the world you are buying transit in.

There are a lot of transit providers in the market, but many get confused as some companies just sell transit, while others sell a wider portfolio of products. For example, one of Cogent’s core products is selling transit, while others like Level 3 sell transit, but also VPN, CDN, WAN optimization and a host of other managed services. Many transit providers also only sell access in specific regions of the world, while others sell in multiple continents. Combined, there are lots of transit providers all over the world, of all sizes. While not a complete list, some of the more well known transit providers are:

  • AT&T
  • CenturyLink
  • Cogent
  • GTT
  • Hurricane Electric
  • KPN
  • Level 3
  • NTT Communications
  • Sprint
  • Tata Communications
  • Telefonica
  • TeliaSonera
  • Verizon
  • XO Communications

A lot of mainstream outlets talk about transit, but never seem to mention what it costs. While transit prices are all over the map based on location and quality of what is being bought, today, most transit in the U.S. costs less than $1 per Mbps, for large volume deals. In Australia, I’ve seen it as high as $150 per Mbps. Japan can easily be $25 per Mbps, but it all depends on the volume being bought. Most companies who buy transit, including Netflix, buy from multiple providers, at different price points and most important, with different SLAs. So many of those who buy transit from multiple providers distribute traffic, across multiple transit providers, in real-time, based on performance metrics. I’ve compiled some of the most common prices I have seen in the market, from those I speak to who buy transit. Again, there are many variables that determine the price, but here are the most common monthly rates I see in the U.S., with commits:

  • 10Mbps $7.00
  • 50Mbps $4.00
  • 150Mbps $2.00
  • 300Mbps $1.00
  • 600Mbps $0.80
  • 1500Mbps – 1.5Gbps $0.65
  • 3000Mbps – 3Gbps $0.50

While there has been a lot of talk about Netflix delivering content inside the last mile, via Open Connect or commercial interconnect relationships, it’s important to remember that 18 months ago, CDNs were still accounting for 40% of the overall traffic volume flowing into ISP networks. I’d have to check what that number is today, but it’s still going to be high as most content owners today use third-party CDN service providers, they don’t try to build out their own CDN as Netflix and a few others have done. This is where transit comes in and allows all of these CDNs to connect to all the different ISPs, so that video gets to end-users.

Transit is so important because without it, the Internet would not work. We’d have a bunch of closed networks that don’t connect with one another and traffic would not make it to end users. From a business standpoint, there are many backbone and transit providers to choose from in a highly competitive market, which all CDNs and some larger content owners work with and gain price reductions every year. Transit pricing has and continues to get cheaper every quarter, and it is expected it will decline in price once again this year.

If you have questions on the transit prices I have shown, please contact me. There are many variables but I am happy to share with you the data I have collected on transit pricing, free of charge.

  • aswath

    Can you clarify the price. Is it for a month? Since you quote the price for Mbps, what is being charged is bandwidth and not traffic?

    • danrayburn

      Sorry, meant to say that, it’s monthly.

      • Brady

        Sorry if I’m the only one still confused… So, for a huge source of traffic such as Netflix, they pay $0.50 per Megabyte moved, billed each month? So hypothetically, if they moved 1 TB in one month, they would pay roughly $500,000 that month in transit costs (assuming 1 TB = 1000 GB, not 1024)?

        • danrayburn

          Please don’t think of Netflix when you look at this pricing. This is not the price Netflix would pay. The prices I listed are for “common prices”, Netflix is not a common customer.

          You are correct on the math, this is pricing per Mbps sustained, not per GB delivered.

          • beforewepost

            I too want make sure I understand your pricing chart.

            Say I start an ISP and buy the 3 gbps pipe. I guarantee my customers 10 mpbs each which means I have at most 300 customers. Does each customer’s 10 mpbs chunk costs me $5 each month, i.e., my monthly bandwidth bill would be $1500 for the 3 gbps pipe?

          • Roy Stuart

            With some slightly changes between carriers thus with the price matrix above, yes. We bought ethernet transport and for a full 1GigE we pay just below $1,000/month.

          • kevin freels

            Must be nice. The best price I can get in my area for 500M is $3300 per month.

          • In South Africa, 200Mbps Global IP transit would cost me $3200,00 per month with a 24-36 months contract at a major IXP in a city like Johannesburg, Cape Town or Durban. Add the cost to deliver it to your area through a backhaul provider.

          • Ewing Fox

            See, that is where the magic of ISP bandwidth allocation takes over – The economics of oversubscription come into play here – What defines how many customers you can support isn’t how much bandwidth you promise your customers – what defines the number of Mb/s sold is the utilization rate – often considered to be calculated as a subscription ratio. A fitness gym can be built to comfortably fit 20-30 individuals working out at a time – but they have hundreds of customers – how can they do that? Because most customers rarely if ever, utilize their paid membership. In this case, their subcription ratio would be 10:1. Datacenters utilize netflow metrics and other QOS indicators to monitor customer traffic over time, aggrigate multiple customers, and forcast increases in customer utilization based on market trends, technology changes, and other factors. This data is then considered during forecasting sessions where the network team develops a maximum oversubcription rate that they feel the current infrastructure can support. What the author of this fantastic article doesn’t discuss (appropriately) is the time involved with getting these relationships locked in – often taking over 90 days to secure transit, and even longer for last mile circuits. This means that networking teams need to be very careful to maximize profit when considering the actual bandwidth costs (which need to factor datanceter costs, power costs, staffing, licensing, switch, router, traffic shapers, QOS monitoring, Netflow monitoring, etc) vs. reoccurring revenue from the services they believe they can safely deliver from a fairly static, unforgiving and complicated network of providers and availability!

            The unspoken fact that the profit margin between actual cost for wholesalers (including the support costs) and the retail prices the market will support is found purely in how aggressively a company is willing to oversubcribe this resource. To do this safely, while maintaining SLA’s and happy customers, requires careful monitoring, a conservative approach to the math and metrics, and a willingness on the network operations team’s part to push back against the desire for radical oversubscription that ultimately backfires and costs a provider customers!

  • “Netflix delivering content inside the last mile”? Isn’t the last mile closed? Isn’t this the problem? Aren’t the internet access providers trying to push the demarc farther away from the last mile demarc deeply into the WAN core so as to protect their middle mile and switching/routing from arbitrage? Won’t a competitive interconnect model in the last mile introduce marginal costs, become generative in terms of 2-way HD solutions, enable 4K to evolve quickly and in general disrupt the business models of the vertically integrated access providers who rely on (high) average costs and pricing models?

    • Frank Bulk

      If you’re stating that ISPs don’t open up peering to the last router facing the customer, that’s typically correct. But that reason is not to “protect the middle mile and switching/routing from arbitrage”, but part of normal traffic engineering. Can Netflix afford to peer with the provider in Bellingham, WA? There’s probably not enough traffic to justify the costs. But it might make sense for Netflix to peer with that provider in Seattle. The largest MSOs and RBOCs typically peer with CDNs and the like in several key cities.

      • Layer 1 interconnect at the LAN/MAN interface (the premise or last router as you call it) would imply that a service (content/app) had enough revenue and/or necessary security requirement or bandwidth guarantee to justify it.

        Revenue justification could come from a centrally procured high capacity telepresence solution (say 20mbs CIR, upgraded to 50mbs CIR for 4K within a decade) for work, medicine, education, civic group, etc… or an internet of things set of solutions. Or it could come from an aggregator of apps and content, like Facebook or Google. Who knows what the model looks like, but the revenue potential is infinite as opposed to today’s high-priced walled garden subscription packages.

        A security requirement might necessitate a hard layer 2 interface (box at the premise) as opposed to a software VPN over the access providers layer 2. Alternatively there could be both hard and soft asynchronous layer 1-2 solutions to make it hard for physical intercept of 2-way sessions. Throw in a backup or insurance option if the application is considered mission critical beyond 2-3 9’s.

        But since traffic is increasingly being accessed through portable or mobile devices and will increase geometrically with the internet of things there might be a wide array het-net approaches that require interconnect in the premise, at the curb, at the pedestal, in the remote serving unit, in the CO, or in the head-end. All of these to handle cross-border interworking seamlessly.

        So when we add up all 3 (high-capacity, security, mobility), there is plenty of demand/revenue for multiple last mile wired/wireless networks to rapidly evolve.

        Historically there are plenty of last mile and premise interconnect solutions: dial-1 for voice, wifi, fax, computer dial-up, (all of which scaling in the 1980s paved the way for the web to explode in the US over the internet in the 1990s) etc… where open interconnect has generated the greatest usage and possible revenue ecosystems. The most recent market driven example was Steve Jobs forcing AT&T to afford equal access for his iOS application ecosystems to wifi offload. Without the latter, undoubtedly the iPhone would have been a marginal success and the smartphone ecosystem would be a fraction of what it became.

        Today’s incumbent communications model of a vertically integrated, silo-ed, duopoly supplemented by a relatively narrowband wireless oligopoly cannot address simultaneous and conflicting forces of supply depreciation (capex/opex) and infinitely segmenting and growing demand. Nor is the current IP model good at sending price signals that clear supply/demand north-south and east-west in the informational stack, so we don’t get rapid and concerted upgrades of technology at the edge. Both models need to continue to evolve, taking the best, not worst parts of each.

  • Vaughan Read

    According to a global survey carried out by the International Telecommunications Union and published in its 2013 trends report, 99% of peering agreements are settlement-free

  • Very well written article Dan. For anyone wondering what are some of the things that you need to determine to finalize pricing are deciding between a BGP or static routing configuration, whether to run multiple BGP sessions on one port or not, IPv6 native or dual stack, DDOS and other protections, link aggregation, and more.

  • Muang San

    I just want to know if I brought Transit, do I need to pay Prefix advertisement fee?