Excerpts from The Internet Peering Playbook: Connecting to the Core of the Internet
From the ashes of @Home, how the Cable Companies emerged as leaders utilizing "Access Power Peering"
The Internet peering ecosystem continues to evolve out of its NSFNET roots as a market force driven ecosystem. New players emerge in the ecosystem, assume and adjust their positioning, and grow their influence with new and advanced network technologies. This chapter highlights the significant and some would argue fundamental changes that are taking place in the U.S. Peering Ecosystem during this new millenium.
The following six evolutions demonstrate how the players react to the changes in the environment. We will first highlight the evolutions that occurred during the economic downturn in the year 2001.
The cable companies were forced to in-source the broadband Internet services that @Home had provided until its bankruptcy in 2001. The cable companies were given only 30 days’ notice to establish high-capacity transit relationships and launch their own Internet services. They entered the ecosystem as independent broadband access network providers as shown graphically in Figure 10-1.
Figure 10-1. The Cable Company model.
The cable company player model is similar to that for Tier 2 ISPs but is a little different in that the cable companies are regional and don’t tend to compete in overlapping areas, and they tend to pull enormous volumes of traffic.
When the cable companies attached to their new upstream ISPs, each cable company experienced the same thing: Internet Transit circuits were congested almost immediately. Each of the cable companies ordered upgrades to these circuits, and almost immediately the Internet Transit connections were again congested!
What was going on here?
After some analysis, the cable companies found that about 40% of all cable company traffic was peer-to-peer traffic; traffic was ultimately destined to other cable company networks. I had been working with the cable companies on the benefits of peering, and as a result, many of them built into multiple regional IXPs and started peering openly with each other and with the other Tier 2 ISPs in the ecosystem.
These new players and their peering relationships are shown graphically in Figure 10-2.
Figure 10-2. The cable companies emerge as independent players.
One might ask, why is there a separate species of player; isn’t the cable company model identical to the Tier 2 ISP model? Why is this evolution significant?
This evolution represented a significant change in the U.S. Internet Peering Ecosystem because:
It was also noted that the peer-to-peer traffic volume grew immediately after the cable companies peered with each other. Peering caused Kazaa to prefer to fetch files across the now peered network path. The Kazaa selection protocol at the time used latency to determine which Kazaa file sharer was “more local”, and automatically selected that file sharer. The result was that when cable companies peered with each other, there was an immediate 20% growth in Kazaa-originated peering traffic volume.
Notes from the field.
The Grandma Story.
A grandmother in Australia invited her grandkids over for the holidays. In Australia, end users are charged for Internet access on a per-megabytes-downloaded basis. Grandma just received a computer and the grandkids helped her set it up. The grandkids were excited to demonstrate their expertise, and wanted to hear the latest Britney Spears album on grandma’s new computer. So, they installed some peer-to-peer software and started the download.
The download did not complete, they never got to hear the album, and the kids forgot about it. By default peer-to-peer software shares all files it downloads, and soon grandma’s computer became the preferred source for the Britney Spears album for the entire continent! Grandma‘s bill from Telstra was thousands as opposed to the usual $25 she paid per month. The issue was compounded since the pricing point in Australia was twice as high if you were a net source of content as opposed to a net sink of traffic.
So grandma called Telstra and said its billing machine was broken. Telstra reversed the charges and told her to turn off her computer until someone could uninstall the peer-to-peer software. So grandma thanked them, hung up the phone, and turned off the monitor to the computer. Why not—that works with the TV set. The next month the bill was double; again she disputed it, but Telstra didn’t reverse the charges and the story became public, so I could write about it. In the U.S. where broadband is typically a flat fee, there is no such disincentive to use all the bandwidth available.
Around the same time, a small group of Large-Scale Network-Savvy Content Providers emerged as large volume peers.
Definition: A Large-Scale Network-Savvy Content Provider (LSNSCP) is a content provider that sees networking as strategic enough to build a backbone and peer.
Most of the LSNSCPs peer at least bi-coastally, and some peer more broadly across the country. The dominant three motivations to peer are similar to the Tier 2 ISP motivations to peer:
Large-scale content players have always purchased transit to deliver their content to the end users on the Internet. Therefore, their role and behavior in the Peering Ecosystem is similar to the Tier 2 ISPs with the exceptions that Content Players:
The model for the Network-Savvy Large-Scale Content Provider is shown in Figure 10-3.
Figure 10-3. The Large-Scale Network-Savvy Content Provider model.
These players still purchase transit (generally from the Tier1 ISPs), but they supplement it by peering openly with anyone and everyone.
Examples of these Large-Scale Content Players are Yahoo!, Google, Microsoft, Amazon, Walmart.com, Apple, Electronic Arts, and Sony Online.
This evolution represents a significant change to the Peering Ecosystem because:
We can see a graphic depiction of this evolution below in Figure 10-4.
Figure 10-4. The Large-Scale Network-Savvy Content Providers peer openly.
The peering of the Network Savvy Large Scale Content Companies with the Cable Companies put the most popular content on the Internet directly onto the same network as the broadband eyeballs. The end result was a major disruption in the Internet Ecosystem, resulting in network performance improvements and significant cost savings from peering!
Notes from the field.
We Expect Customers to Peer When Rational
I spent some time speaking with the Tier 1 ISPs at the time, asking about the massive migration of their customer traffic. Their response was that they expected it would happen.”We expect our customers will do that which is rational.”
And the middle of the Internet Peering Ecosystem got a little bit fatter as shown in Figure 10-5.
Figure 10-5 Content peers with eyeballs. The Tier 2 ISPs and the cable companies peer with the content companies that entered the peering ecosystem.
This represents a significant dynamic shift in the peering ecosystem since the Tier 1 ISPs are being cut out of a high volume traffic exchange loop. The hierarchy of the Basic Internet Peering Ecosystem has evolved into a flatter mesh, with the Tier 1 transit providers required only as the route of last resort. Traffic destined to locations too far away or too expensive to reach will ultimately need the services of an international transit provider. At this stage, the Tier 1 transit providers have lost their grip on the U.S. Internet Peering Ecosystem.
We have learned that Internet traffic volume is a key determinant as to whether peering makes sense financially. Video represents a proportionately larger amount of traffic when compared against e-mail, web browsing, and other traditional request-response messages. Amplifying the importance of video traffic is the fact that not only is it a larger amount of traffic, but it is also an increasingly popular type of traffic across the Internet. As it turns out, CDNs are really good at distributing video to the edge for offloading to the last mile consumers.
The CDN player (Figure 10-6) is similar to the Tier 2 ISP but instead of handling a steady stream of content, they distribute any “web objects” to caches at the edge. Today the service is priced the same as Internet Transit, so for modeling purposes we will make the simplifying assumption that the CDN is essentially providing transit - just a much better performing one due to the caching at their edge.
Notes from the field.
CDNs Pushing Massive Traffic volumes
Around 2006, as YouTube was starting to gain popularity and started using CDN services for the most popular most viral content. The community started noticing the CDN guys were deploying massive peering gear. At the time, most were peering at 1Gbps, and perhaps an occasional 10Gbps connection. I was traveling the peering speaking circuit with the CDN guys who were deploying 10G and occasionally multiple 10G connections to the peering infrastructure. To some CDNs, videos are just another web object to be distributed at the edge of their network. These objects however are very large, so the wakes set off by movement by the CDNs were starting to be felt in a big way. The CDNs, as very large volume open peers, quickly became significant players in the Internet Peering Ecosystem.
It may have been YouTube that activated the video revolution around 2006, but there was an entire underlying Video Internet Ecosystem that conspired to enable it starting back as far back as 2002.
Consider one slice of Internet Video category is User-Generated Content. To enable content creation side, small easy-to-use video cameras came onto the market. Home video editing software (like iMovie) was inexpensive or bundled with the sale of new computers. Next in the supply chain, video distribution became free and easy thanks to YouTube, a service that leveraged the inexpensive transit and CDN services. These services leveraged volume pricing with ISPs that interconnected with last mile providers that upgraded the last mile infrastructure (to DOCSIS-3 for example). All of this would not have mattered if it wasn’t for the high-speed wireless plug-and-play networks deployed in every home today. Innovations along each link of the supply chain were required for end-to-end video distribution to work well. Once these services matured, Internet backbones quickly became dominated by video.
Cisco estimates that by 2013, about 80% of all Internet traffic will be video, leading me to coin the phrase, the “Video Internet.” Friends in Japan tell me the Internet in Japan is already 80% video, while others in the U.S. say conservatively 40%-50% of all U.S. traffic is video in 2011. While this “Video Internet” term is still gaining acceptance, I believe it more accurately reflects the dominant traffic type across the Internet today.
It was the innovations across the entire supply chain that activated the Video Internet, a system that is now positioned to service the video distribution needs of the planet.
This all sets the stage for the Access Power Peering - the first time in history that a customer has successfully forced a transit provider to pay for access to its eyeballs!
DrPeering.net ©2014 | Privacy Policy | Terms and Conditions | About Us | Contact Us
The 2014 Internet Peering Playbook is now available on the iPad at the Apple Store and for the Amazon Kindle. The Kindle, ePub and PDF form are also perpetually updated on the DrPeering DropBox share.
