Netflix’s Streaming Quality Is Based On Business Decisions by Netflix & ISPs, Not Net Neutrality

Over the past two days, there’s been some news articles suggesting that some ISPs, like Verizon, may be purposely throttling or slowing Netflix’s video content over their last mile network, because of the recent Net Neutrality rulings. While that sounds like a great headline and something to get people all worked up over, that’s not what’s really taking place. The reality is that business decisions that Netflix and ISPs make regarding quality of service determine how good Netflix or any other content looks when streamed to consumers. Both Netflix and ISPs are constantly having to add capacity to their networks, of all kinds, and make decisions on how much money they want to spend versus the level of quality they want to offer.

To make it simpler, if Netflix or the ISP can deliver content right now and the video starts up within three seconds and has SD quality, they have to decide if it is worth spending the additional money to make it startup within one second and have HD quality. From a business decision, it does not help any ISPs business to improve the quality past a certain point. Each ISP decides what that point is and the quality that is good enough, versus how much they would have to spend to improve it. These decisions that Netflix and ISPs make determine how much transit, peering and other technical resources they deploy and what impact it has on their bottom line. While the technical pieces like servers, transit, peering etc. is what makes streaming services work, it’s the business decisions made behind the scenes that determines the user experience, not Net Neutrality.

From a technical standpoint, it’s also extremely complicated how Netflix’s content is delivered as they have a total of three different CDNs, all delivering content in different ways. Some content is delivered inside last mile networks via Netflix’s Open Connect program, some is delivered via third party CDNs like Level 3 and Limelight Networks and other content is delivered via servers Netflix controls outside the last mile. When it comes to ISPs, no two are identical in footprint and number of subscribers so there are many variables. Netflix buys transit from many different providers, which come at different price points and with different SLAs.

None of the big ISPs buy much in the way of transit and those that do, buy it primarily for minor destinations they can’t get to through peering. The majority of U.S. originated traffic comes through peering points into the big ISP networks and ISPs who choose to, can make those runs congested for many hours a day. The whole process of getting Netflix content, or any video for that matter at scale, to consumers is very complex. Many of those who write about Netflix streaming don’t understand all of the pieces involved, how they all tie together and are quick to point fingers as things like Net Neutrality, which isn’t the case. There are also a lot of issues on the consumer side, inside the home, that affect the quality of video being delivered where many times, it’s not the ISPs fault for the poor experience.

I had an issue over the weekend where Netflix would not stop buffering on my Xbox 360, but once I used the Roku, streaming was perfect. After spending time to diagnose the problem, it turns out my Xbox 360 was having NAT problems with my router and even though it had a static IP, it had to be flushed out and setup again. In a lot of these types of cases when there is a problem, most consumers don’t know how to do a trace route, assign a static IP, log into their router and really diagnose the problem. ISPs can and do have issues, but I’m willing to bet that more than 50% of the time the poor QoS is due to the device, WiFi or local network issue inside the home.

Some are suggesting that because Verizon, Comcast or other ISPs have been dropping in their ranking via Netflix’s “ISP Speed Index Results“, it’s evidence that these ISPs are purposely making Netflix streaming from their network look bad. That makes no sense at all and would only hurt the ISPs business. If it looked bad enough and impacted enough customers, many would leave. Some can’t as they only have one ISP in their area, but many have choices. Also, keep in mind that the ISP ranking that Netflix provides is NOT comparing apples-to-apples nor does it even say what exactly it is defining. As usual, no one seems to question the data that many of these companies present to the market. Netflix does have a lot of very detailed performance data and intelligent video player that makes real-time delivery decisions based on that data, but Netflix doesn’t share those metrics or their methodology of how they say one provider is better than another.

Google Fiber is at the top of Netflix’s ISP Speed Index list, yet has the smaller footprint of any ISP in the top 10 of Netflix’s rankings. While Google refuses to reveal subscriber counts or uptake numbers for Google Fiber, research reports predict it will be another two years before they have 3 million subs. So even if they have 1 million subs today, it’s easier and cheaper for them to make the Netflix experience on their network the best it can be, compared to Comcast that has 20 million Internet subscribers. You can’t compare the two fairly.

In addition, what exactly are Netflix’s ISP results measuring? At the bottom of the Netflix page, it says the results “reflect the average performance”, but then they don’t define what methodology is being used to define performance. The only thing they show is the “Average Speed in Mbps”, but what exactly does that mean? That’s the speed at which the streaming is being delivered, but that measurement alone doesn’t show startup time, buffering, or a whole host of other factors. Where is Netflix methodology of exactly what is being measured and shown in their ISP charts? Netflix was very smart when they launched the “ISP Speed Index Results” as it puts the pressure on ISPs to be seen in a good light from their customers. It also means that in most cases, instead of the customer complaining to Netflix when they get bad quality video, they will instead complain to their ISP, but that may or may not always be fair. Some ISPs simply don’t appropriately add capacity between their network and all of their peers, while other times it’s Netflix, not the ISP who has made decisions that impact quality.

While there is this idea that Netflix has moved most of their video delivery to inside the last mile, by working with ISPs, the reality is that in the U.S., Netflix is still delivering a lot of their own streaming, via third party CDNs Level 3 and Limelight Networks as well as Netflix’s own caches. From what I have seen, only Cablevision, Suddenlink and Google Fiber have allowed Netflix to put caches inside their networks so many times the quality issue a consumer is having might not be the ISPs fault. It could be the CDNs, the connection between the CDN and the ISP or a whole host of other factors. It’s true that some ISPs won’t add more capacity when they need it, but they have made the decision that it does not make sense from a business standpoint, based on the return. At some time, the ISP has to decide the trade off between quality and how much money they spend upgrading their network. So while no ISP wants to come out on record and say it, many see Netflix as a threat and don’t want to give a competitor any advantage in the market.

It’s easy to use Net Neutrality as an excuse for why there are quality issues with streaming, but that’s not accurate. Delivering video over the web has inherent flaws and it’s not like traditional broadcast distribution that scales much easier. It’s one of the main reasons why pay TV won’t be replaced by Internet streaming at scale. It’s too expensive to support so many eyeballs while also providing the best quality possible. Case in point. If all we wanted was to have better video quality, why doesn’t Netflix encode their videos at a higher bitrate, say 8-9Mbps and deliver all of their content that way? Reason is that the ISPs would have trouble delivering it with good quality, which is the whole reason why Netflix is trying to get ISPs to join Open Connect and allow Netflix to put servers inside the ISPs network. While that sounds like a simple and free solution Netflix is providing to ISPs, the reality is far from that.

This is the problem that Netflix is currently facing with most of the major ISPs in the U.S. who won’t join Open Connect, which has made Netflix’s current multi-tiered approach to video delivery complicated. Even though Netflix is not causing a lot of people to cancel their cable, many ISPs sell pay TV services, have their own OTT services and see Netflix as a competitor. Some ISPs have also told me they would rather deploy a solution like transparent caching which would help the ISP with caching all types of video content, not just Netflix’s. Netflix has to make some decisions about how they work with ISPs, what role third party CDNs will have delivering their content and how much of a network they want to own and operate on their own. The good news is that I think all of these things can be worked out over time, amongst Netflix, ISPs, CDNs and transit providers, but it’s going to require Netflix to change their current approach. I’ll have more on that in a follow up post.

  • No Nonsense

    You sir are an idiot or an ISP plant. This is a pretty basic monopolistic issue. The ISPs deliver internet AND they compete with Netflix via their own video streaming offerings which have been an abysmal failure.

    • danrayburn

      “ISP Plant”? Sounds like something you need to water.

      You say the ISPs compete with Netflix. In my post I said they “see Netflix as a competitor”. So don’t know what your argument is.

    • Nonsense

      Actually it is pretty interesting. I pay Verizon to give me broadband service and I pay Netflix to give me Netflix service. Verizon is delivery a good quality broadband product to almost every destination and service on the Internet. Netflix is the one that is having a problem and is making poor decisions on the best way to delivery their service to multiple ISPs

      When I try an alternative ISP, I get similar Netflix results. When I try a different streaming video provider (even competitors to VZ), I get better results. The “Redbox argument” is a red herring.

      This is simple logic.. The common denominator here is Netflix and not ISPs.

      • danrayburn

        It’s both, it is Netflix but it’s also ISPs who don’t want to spend the money to improve the end result to the customer.

        • No Nonsense

          “ISPs who don’t want to spend the money to improve the end result to the customer.”

          Obviously. Bingo we have a winner!

          “Verizon could always harm Netflix (and other) traffic if it chose to do so simply by failing to upgrade peering infrastructure in a timely manner. ”

          “Yet some ISPs have improved their Netflix speeds even as Verizon and Comcast falter. Google Fiber, Cox, and Cablevision in particular have been impressive. The comparisons to better performing ISPs place Verizon and Comcast in a bad light.”

          http://arstechnica.com/information-technology/2014/02/netflix-performance-on-verizon-and-comcast-has-been-dropping-for-months/

      • uniquename72

        And yet Netflix on my Cox internet is faster than ever — as is my friend’s, who is lucky enough to have Google Fiber — so no, the common denominator is not Netflix.

        Reading isn’t that hard.

        • Nonsense

          And Amazon Prime, Hulu, Vudo, HBO Go, and every other service is great on my Verizon…

          Perhaps Netflix likes Cox

          Depends on who you want to believe, and I expect Netflix wants special network privs over others and is not above negatively impacting their customers to get it.

  • http://zsolt.takacs.cc oker

    Completely misunderstanding net neutrality. It is not about peering/routing, it’s about packet prioritization. All it means is you cant prioritize packets from different parties differently. If the ISP has OpenConnect that just means it’s cheaper for them to serve netflix content than other streaming vendors’ content. If they have bigger traffic from Hulu than Netflix net neutrality means they cant throttle it to serve more cheaper netflix traffic instead to less customers, they have to divide the bandwith by the ratio of their clients streaming from the different vendors, to be neutral towards the vendors, and in the end their customers.

  • sam1331

    I don’t understand why some people act as if when Verizon provides poor Netflix service as long as it doesn’t directly violate “Net Neutrality” it’s okay. Verizon is advertising a certain internet speed. If that speed is not possible 99% of the time “in the real world,” meaning from Netflix, Youtube, Amazon, etc., it is clear that they are lying to their customers. If their peering does not allow them to deliver advertised speeds, they should be sued and fined until they begin advertising actual speeds or improve their peering to allow those speeds. There are laws in many different areas of commerce to protect the consumer from false advertising that specifically lay out what types of claims are allowed and what types are not. There are more ways than “throttling” to purposely slow Netflix (and video internet competitors) speeds. The FCC should really be testing for “internet” speeds rather than “connection to Verizon server” speeds when they do their yearly tests, and punish ISPs that lie to their customers.

    • SamKnows

      Actually all the FCC measured data says that Verizon is delivering over 100% of advertized speed. What is not working is Netflix’s commitment to their customers of delivering a good product and it is happening across many ISPs. Netflix is the one not investing in quality network delivery. Lowest cost transit gets you lowest quality delivery. Other video services don’t seem to have the same problem and work well.

      • sam1331

        My point is that the FCC data is flawed. It measures something that has no relation to the real world. I can do a Speedtest all day long on Time Warner Cable telling me I get 20 Mbps. But no matter where I go on the internet, I see much slower speeds. You can blame the individual services. But it seems unlikely that every service on the internet is providing poor service. It is much more likely TWC is not actually offering those speeds.

        • SamKnows

          FCC and Speedtests are good measures of the ISPs.

          The reason you can’t get 20 Mbps of application X is
          1) It is being throttled at the source (server load, etc)
          2) It is using a crappy transit ISP
          3) It needs less than 20 Mbps

          If every service on the Internet is poor… I agree this is your ISP

          If one is poor… It is the server you are using

          If a few are poor and others are great. They cheaped out on a common transit provider (e.g. Cogent)

          • Fanfoot

            Sorry, but I gotta call it out the minute says that Speedtest is a good test of your internet speed. I’m writing this in a hotel room where the Wi-Fi here is stressed right now to the point its having trouble rendering pages, and no I’m not talking about video. I’m talking about something like the text-heavy techmeme.com. Yet if I run speedtest it tells me my internet performance is great.

            You’ve heard that people sometimes cheat on benchmarks right? How Samsung maybe tweaked their phones to run faster when they noticed a benchmark was running? How GPU companies tweaked their drivers time and time again to get better scores on benchmarks alone? You’ve got any tech history at all? Now tell me again how a benchmark used to gauge network performance can’t be prioritized with a straight face.

            And that’s without even mentioning that Speedtest.net discards the slowest 30% of your test results (you know, those delays that really bother you because they are unusual), etc etc. Its a fine quick and dirty speed test of maybe the maximum you might see out of your cable modem. It does NOT represent the throughput you’ll get even from an uncongested server with a good peering arrangement sending you real data.

  • Tom Stuveboul

    Dan does a good job of explaining just how complicated this is and he’s right, it has nothing to do with net neutrality. This is simply the business of how the Internet works.

    Quality issues are Netflix’s fault, but it’s also the ISP’s fault as well. ISP’s aren’t throttling traffic like some suggest, but rather not upgrading capacity between their network and their peers. They don’t want to spend the money on more transit. So while it’s not technically “throttling” what the ISP’s are doing, the result is the same, poor quality. The ISP’s know what they are doing, it’s intentional.

    • Tim1942

      Now that Netflix moved to their own CDN, and increased to Super HD, and also grew a lot in the 4th quarter, do they have enough capacity?

    • SamKnows

      I would agree with you if there wasn’t a large number of peers they had to congest. It would be so many that this was impacting many services (which tells us that other paths exist).

      Sounds to me more like Netflix thinks they can move ungodly amounts of traffic around willy nilly to create disruption. If you peer with them, they will stop. Until then.. you will be blamed for their changes.

    • Skippy9250

      It is the ISP’s responsibility to make sure they are providing the consumers of their product with the quality they are expecting. So if they need to upgrade capacity at certain nodes, then they should upgrade the capacity because that is what they need to do to keep their consumers happy. All of them, (Comcast, Verizon, AT&T, etc) make billions in profit every quarter, they can afford to upgrade the capacity of their nodes that are congested.

      If our infrastructure was actually competitive we wouldn’t have this problem. ISP’s would upgrade as needed to maintain the satisfaction of their consumer base or they would go elsewhere. However, being that a vast majority of every broadband consumer has a 1 choice of cable or DSL there is nothing they can do to “punish” the ISP’s lack of effort in serving them.

      Add on to that above the fact that you have content companies (companies that even Netflix is a competitor to) that have even less of an incentive to make their ISP product deliver quality Netflix service. If people can get the content they want at a quality of service they want, then there is less reason to have pay TV. Which is just another clear vision to the fact that ISP’s are simple dumbpipes. They could providing nothing more than bandwidth and a public IP address to their user base and they could enjoy all of the internet.

      Bottom line is that ISP’s have a much bigger stake in crippling the quality of Netflix and have virtually 0 cost (consumer backlash) in doing it while having lots to gain. Netflix crippling their service comes at great cost to them (subscribers).

  • Fanfoot

    Wow, this is really rambling.

    I’m not sure what your point is exactly.

    Do we know that Verizon is definitely throttling Netflix? No, no we don’t. We have data that suggests something funny is going on but no actual proof. Of course its not clear that its possible for us to get proof without the complete and total cooperation of Netflix and Verizon so that may be as close as we come. We’ll see.

    However, your attempt to say its really Netflix fault and not Verizon’s fault (actually you don’t quite say that, which is sort of odd actually given the title of the article) isn’t really justified either. You provide no proof of any sort one way or the other. In fact you even suggest that ISP’s have the right to do whatever they want.

    Seriously, the competition argument? In the US? What percentage of people have a real alternative? I mean how many locations does Verizon service now? 6 Million? And 1 Million or so of those are DSL? So really 5 Million actual customers who could choose something else? And there are what, some 200 Million US homes? So there’s real choice in the US for 2.5% of them? The rest have the option to what a) use their only cable provider b) use DSL and get low speed internet with low caps c) use their cell phone and blow through their caps within minutes d) get satellite with its wonderful latency? This is the competitive market you are highlighting?

    Yeesh. Call me when you figure out your own position please.

  • Tom Z

    At this point it doesn’t really matter to me any more who is to blame, Netflix or FIOS on my end. For at least 4 months now I can’t get any thing more than 235kbps on any video any part of the day. Connecting to my office via VPN a few miles down the road, I get 3000kpbs every time all day long…….

    I am paying Verizon and Netflix and all I get is 235kbps, does it really matter to me who’s fault it is. NOPE. I just want it to work better than 235kbps, am I asking for too much after paying $140 for both each months!!!!!!!!

  • bronxlcsw

    Well i just bought a Vizio 70 inch LED and its absolutely stunning picture looks like blue ray quality when i play an HD movie using the smart tv Amazon channel. But when i play the same movie through netflix its choppy looking and lags even though its supposed to be so called super hd quality.

  • Adolfo Rosas

    First of all, I have to say this is a clear and direct to the point article. Congrats for that Dan.

    I essentially agree on your view and it is clear to me that people have a tendency to name net neutrality too frequently when they face internet behaviour and facts that they do not like.
    Judging by the comments of some readers I see the great confusión that there is about ‘net neutrality’ and the role of the ISPs. It is a very sensitive matter. Even in the deepest Telco offices it is difficult not to find people that do not understand which is the Telco business.
    Telcos were used to sell a voice service, and times when this service had ‘bad availability’ have been long forgotten (but they existed and they come back sometimes for very short periods like in new years eve and other ‘synchronizing events’). An internet Access is essentially a shared resource in the same sense as it is the voice Exchange (PBX). But for the internet access the ‘condition of being shared’ is by far more impacting as for a voice call usually 1 or 2 exchanges (company-to company) suffice, but for a internet packet many more may happen. If you realize that it IS actually impossible to grant enough capacity in voice exchanges to 100% of registered users, (though this rarely has an impact on everydays users), why does anyone have a problem in realizing that is many times more difficult to grant anything to a single internet Access?.

    The internet is NOT an end-to-end service. Internet experience will rarely be confined to the responsibility of one single company, or two…or ten.
    If you were talking about a walled garden inside internet and you never get out of it you may start blaming the ones that built the garden and its interfaces all the way up to you… but this product is not commonly found in the market for internet access. When some ISP comes to you and offers to sell you Internet access for some bucks a month he cannot promise to give you a walled garden experience. The ISP notices there is great responsibility in selling a service in which he is just one of the pieces and there are many others that have an influence.

    Net neutrality is an elusive concept that has nothing to do with the shared condition of the internet access. It is impossible to share the access fairly, end to end, applying technical means so every registered user gets its fair quota.

    So if we realize that the neutrality concept is impossible to apply (strictly) how can we then complain and claim for neutrality?. In my view only really clear abuses like some unjustified aggressive throttling that sometimes in the past happened can be claimed, but even for these cases there is tremendous difficulty to demonstrate the throttling was unjustified and aggressive.
    Regards.
    Adolfo M. Rosas

  • Michael Scott

    The article says, “From what I have seen, only Cablevision, Suddenlink and Google Fiber have allowed Netflix to put caches inside their networks”. My provider, Cox Communications, is an Open Connect “partner”; I always get the highest quality from Netflix. Even more impressive if no inside-the-network caching “storage appliances” are involved.

  • http://www.a3mediallc.com A3MediaLLC.

    Oh and just one more thought….People say, “Why would a ISP mess up their users/costomers experience? That would cause them to chose a different provider and they’d lose money”…In my case that was true but I’m an exception with multiple carriers in a small area that is highly populated….If they are the only game in town they know well …you can always go with dialup…snicker……
    What I think reality is and what the mass “they” have been shilling in the rumor mill is …..Ohhh you don’t like the user experience at our base level…well you john q customer and Joe Plumber business can upgrade to our wonderful higher speed speed price at triple your current price…And thus “they” ISP’s are begginning our slow walk as consumers to them making 2 pipes of access to World Wide Web content….the slow shoddy service or the premium unfiltered access…Guess why all these people are harping on the net neutrality thing huh? Well that and the competing services thing too…

  • http://www.a3mediallc.com A3MediaLLC.

    WOW and speaking of sensorship…I made a great argument didn’t curse, didn’t provoke fighting and my post gets deleted because hey I mentioned how comcast has throttle down user accounts in the past…..What a shill of an article.

    • danrayburn

      Nothing got “deleted”. All three of your posts are here. Before you start “assuming” something got deleted, mybe you should first check with me that it actually went through, wasn’t flagged as spam, or didn’t get blocked due to my language filtering.

      My phone number and email are listed for anyone to easily contact me if they have a problem commenting, but I guess that’s too much work for you. It’s more dramatic to infer something else must be taking place.

  • http://www.a3mediallc.com A3MediaLLC.

    Here is the re-post of the comment I left that got sesnored…Luckily someone was talking about it on Linkedin and I had posted there as well as had it still in notepad.

    This is clearly the ISP’s…sorry if you disagree….To say that an ISP would never do this would not hold up to their history in the actual real world…I little reminder does anyone remember the throttle down of accounts because of audio and video file sharing???? No? Well it was a proven FACT that if you were constantly uploading/downloading audio and video files(even if you did it legally) Comcast would throttle your account speed down….Seems like some are back to their old tricks…Starting the same exact way…well no one could prove really that they were doing it until someone people did and some people came out to whistle blow on it as well. MSNBC, CNN had a ton of articles back in the day.
    I’m talking as one who has been on at least cable modem since 1996(orig via comcast).
    Hey speeds were great for those first few lean years and yes once more and more people started to get on of course expected slow downs but here is the thing…You remember me saying legally because I know there will be someone going to comment well you doing illegal stuff…I was originally an Audio Recording Engineer and also a musician…I used to upload and down load audio files wav, aiff, mp3 all the time day in day out for clients, my own personally stuff creating with music partner ect…Guess what started happening ohhh I’d connect to my server…get pretty good 200kb/s upload…within 10 seconds it would throttle down to 34kb/s….ohhhh maybe it was your server…nope when I’d connect from work say from Sony Music or any later on from my first live streaming company we had a Verizon T1 would get up to 450kb/s + transfer up to my server…My singer later got Verizon fios and would get excellent connections…..then months and years went by with people complaining…and guess what ohhh big articles popping up how Comcast was throttling ultra user accounts transferring a/v…Later my roomates and I did switch to verizon fios but there were issues with that too with them adding stupid services to raise billing aka antivirus, game server access, when we never asked for these things….Had to unfortunately switch back to comcast…Luckily though I live in NJ and when I moved when I got married I was able to have choice to go with Optimum/Cablevision..I’ve had great service and haven’t had real tv for 2.5years now all wife and I use is Roku and also hey ready for this….you can get free dvd rentals from the library. Optimim/Cablevision solid IMO.

    I truly feel sorry for those that have no real competition choice for high speed access especially now with Comcast looking to do another large merger.

    Oh only real Netflix flakeouts ever experienced is when DC area gets hits with storms and think that they have large cdn farm that area? I can’t confirm this but thought what I read once why they had real outages.

    So if I were a betting man do I think that some ISP’s are playing dirty pool? I’d take that bet any day of the week and win everytime.

  • http://www.a3mediallc.com A3MediaLLC.

    So that’s twice now that you’ve deleted the post…Maybe the next time Joel asks me if I am going to get a booth at StreamingMedia East and or for any advertising request to your Publications I should mention how infantile you blog post moderators are in attempts to sensor real world discussions.
    What’s funny is you leave up posts by people calling you an idiot and a plant but delete real world discussion that is for the most point on target with the discussion and what has been done in the past.

    Anthony Sellitto
    President
    A3 Media LLC.

  • michiganman

    I’ve had to deal with Comcast customer service and they are lucky they are in another country. If they were in this country then they would think twice about disconnecting after an hour on hold, sending you to a dead line or shuffling you back and forth between service agents. The cable companies have monopoly contracts and will not compete with each other in the same areas. I have no problem what so ever believing that they would interfere with a competing business provider. I just “chatted” with Netflix support and got no satisfaction, but I actually think their native language was English and they could have a conversation without going by a script of their computer. I used to get HD on PC and thoroughly enjoyed Netflix. Now I can’t get anything in HD at all no matter what settings I use. All the other services Hulu, You-Tube, etc. I can stream 1080 with no problem. I don’t even think I’m getting 480 with Netflix anymore. I think the price is right for Netflix, but I would pay another $2 a month for HD. I think I may have to wait until house of cards comes out on Blue Ray…because I can’t stand the terrible pic I am getting with a 20 Mbps download and 12 ping.

  • michiganman

    The just announced deal that Comcast is going to acquire Time Warner will give them too much power and this kind of thing is just the beginning.

    • http://www.geo-tel.com/ GeoTel

      Absolutely. They will monopolize the landscape and it will greatly penalize the consumer.

  • Bob

    Allowing inter-carrier interconnects (eg peering) to become congested is an excellent way to debilitate competing service offerings without technically violating net-neutrality.

    Traffic-balance rules for settlement-free peering were established in the old days when long-haul providers bore the pain of accepting data at remote locations and delivering it to their own customers (and recursively). Traffic-balance ensured that peering long-haul providers shared the pain (at least approximately).

    Times have changed. The traffic flow patterns on the Internet are not the same today, as they were in 1995. For example:
    o Today, thanks to intelligent server-selection algorithms, content is often sourced from servers in the SAME CITY as the end-user who requested it.
    o Last-mile networks are for the most part asymmetric in nature (providing much much greater downstream capacity than upstream capacity; due to technical constraints of the physical media and/or MAC protocol). Such asymmetric-technology last-mile networks (eg cable-modem) will NEVER be able to achieve traffic-balance; there will ALWAYS be more data coming into them than going back up towards peers.

    It is disingenuous for the operators of last-mile networks to insist on traffic-balance. It provides a (false) moral foundation upon which they erect a financial framework intended to either a) discourage OTT (and other types of competing service), and/or b) extort money from the content supplier( by making them or their ISP pay settlement-fees to the last-mile operator).

    Customers of last-mile ISPs should be able to expect to download content from anywhere on the Internet at reasonable speeds. They are paying for more than last-mile service; they are funding the transit-provider(s) under contact to their last-mile ISP. End-users should not have to worry about squabbles between various long-haul and last-mile networks. Alas, that is far from true.

    If end-users lived in an area with aggressive competition in the last-mile market place, then competitive pressure could force last-mile operators to provide good service. (It is worth recalling that cellular service sucked big-time until the PCS auctions brought four or more competitors into the market place. Even two competent viable 800Mhz competitors were not enough to engender meaningful competition to the benefit of the consumer!) Many markets in the US have only a single genuine high-speed last-mile ISP; a few have two providers. Time and time again, consumer surveys report that ISPs are one of the most hated businesses in the US. Doesn’t seem like competition is working.

    How bad is the ISP market?
    When a content supplier (or their CDN) delivers requested content ALL THE WAY to the city of the end-user, the last-mile ISP won’t accept that data and deliver it to their own customer (at the user’s contracted data rate, or even a substantial fraction of that rate).

    I live in LA and have 500Mbps FTTH. There is an MLAB3 NDT speed-test server on Cogent here in LA. In the middle of the night, I can download from that server at 500+Mbps. (FTTH is GOOD STUFF!) But at 9PM, I see only 522 KILO bps. A factor of ~1000 reduction in download speed. The last-mile network measures 500+Mbps 24 hours a day (from speed test severs on ISPs that don’t have congested peering with my last-mile ISP). The owner of the last-mile ISP is fighting with Cogent, refusing to modernize their concept of peering. Their own customers are suffering. But since there is really no competition in the FTTH market (and cable-modem ISPs are not any less greedy in their peering negotiations), we users have NO ALTERNATIVE. We cannot use market competition to force change on the last-mile ISPs because there aren’t enough competitors in the market place.

    The business practices of ISPs today are the same as the old days of 800Mhz cellular, where bills were high and coverage sucked. As both cellular and last-mile ISP each appear to be undergoing market consolidation, it seems less and less likely that consumers will have an opportunity to make market competition work for them. If government does not become engaged in some manner (even by building empty conduit that ISPs can rent) then consumers are doomed.

    In the case I cited earlier: LITERALLY, my last-mile ISP won’t carry data from one side of LA to the other. That’s how bad it is. And it is going to get worse.

    • Vlab

      That goes two ways. A CDN intentionally delivering traffic over an ISP that knowingly cannot handle the massive load is an excellent way to cause disruption and leverage in negotiations. It is also an excellent way to impact Mlabs measurements and other service using the same path (vs many others that exist)

      The transit market is very healthy and prices go down every quarter. There is no reason anyone has to buy direct as long as they spread the load properly to avoid creating problems (see every other CDN).

      The transit market also keeps the paid-peering market honest.

  • michiganman

    Amazing 2 days after the Comcast/Netflix deal was announced almost all my video on Netflix is now once more available in HD, except obviously that content which was not in HD to begin with.