At a Glance
Section I - Connecting to the Edge of the Internet
Section II - Connecting to the Core of the Internet
Sectoion III - The Global Internet Peering Ecosystem
Section IV - The Tricks of the Trade: The Playbooks
Table of Contents
Context Is Important
Introduction to Internet Transit
Internet Transit Price Declines
Implementation Model for the Internet Transit Service
Observations about Internet Transit
Edge "Tricks of the Trade"
Transit Tactic 1 – Optimal Internet Transit
Transit Tactic 2 – Gaming the 95th Percentile.
Transit Tactic 3 – Multi-Homing
Transit Tactic 4 – Renegotiate Multi-Year Contracts.
Transit Tactic 5 – Play the Market
Transit Tactic 6 – Resell Transit.
Transit Tactic 7 – Secret Sauce Transit.
Transit Tactic 8 – Build into Cheap Transit Internet Region..
Transit Tactic 9 – Internet Transit Troughs.
Transit Tactic 10 – Capture Content and Access Customers.
Transit Tactic 11 – Short-Term Transit.
Transit Tactic 12 – Deploy Trial Gear into Operational Environment.
Three Key Points about Internet Peering
The Top Five Motivations to Peer.
The Internet Peering Process..
The IXP Peering Cost Model
A Paid Peering Aside.
The Cost of Peering.
Calculating the Cost of Peering
The Peering vs. Transit Economics.
Tracking Peering Effectiveness.
Chapter 6 – Selecting an IXP
1 – Telecommunications Access.
2 – Deployment Process.
3 – ISP Current Presences.
4 – Operations Support
5 – Business Alignment
6 – Cost
7 – Credibility
8 – Exchange Population.
9 – Existing vs. Emerging Exchange..
10 – Regional Route Strength..
One Final Note
Chapter 7 – Public vs. Private Peering
The Top 3 Reasons Public Peering Is Better Than Private Peering.
The Top 5 Reasons Private Peering Is Better Than Public Peering.
Hybrid Approach (Public + Private Peering).
Chapter 8 – The 20th Century Internet Peering Ecosystem
The First Peering..
NSFNET-Era Internet Peering, 1987–1994.
NSFNET Transition Ecosystem, 1992–1996
The Early Commercial Internet Model
Post-NSFNET Internet in the Growing-Pains Era, 1997–1998.
Carrier-Neutral IXPs Replace NAPs.
Chapter 9 – The Global Internet Peering Ecosystem.
The Global Internet Peering Ecosystem
The Internet Peering Ecosystem.
The Tier 2 ISP.
The Content Provider. 1
The Base Internet Peering Ecosystem.
Evolution #2 – Large-Scale Network-Savvy Content Providers Peer.
Evolution #3 – Cable Companies Peer Directly with Large-Scale Network-Savvy Content Providers
Evolution #4 – CDNs Dominate Traffic Volume.
Evolution #5 – The Video Internet Is Activated
Evolution #6 – Access Power Peering.
Chapter 11 – The ISP Peering Playbook.
The Graphical Representation of the Plays..
The Internet Service Provider Peering Playbook
Tactic 1. The Extended Direct Approach.
Tactic 2. Internet Transit with Peering Migration.
Tactic 3. The End-Run Maneuver
Tactic 4. Bundle Internet Transit with Peering..
Tactic 5. Buy Transit from Restrictive Tier 1 ISP
Tactic 6. Transitional Paid Peering..
Tactic 7. Partial-Route Internet Transit..
Tactic 8. Play Chicken.
Tactic 9. Traffic Manipulation.
Tactic 10. Bluff Issues.
Tactic 11. Be Open. Loudly..
Tactic 12. Be Everywhere.
Tactic 13. Get Traffic
Tactic 14. Friendship-Based Peering.
Tactic 15. Spam Peering Requests.
Tactic 16. The Honey Approach – Be Sweet
Tactic 17. Purchase a Well-Peered ISP.
Tactic 18. Bait-and-Switch..
Tactic 19. False Peering Outage.
Tactic 20. Leverage Broader Business Arrangement.
Tactic 21. Overseas Power Play
Tactic 22. Construct Peering from Transit
Defensive Peering Tactics.
Tactic 23. Defensive Tactic - Do Not Peer in Your Home Market
Tactic 24. Defensive Tactic - Protect Peering Policy
Tactic 25. Defensive Tactic - Change & Escalate Peering Prerequisites.
Tactic 26. MILD Defensive Tactic
Tactic 27. Congest Transit Pipes.
Peering Tactics That Don’t Work
Chapter 12 – Taxonomy of Data Centers.
The Internet Data Center Model..
The Basic Data Center Model
Data Center Operations
The Networked Data Center..
Carrier Data Center
Internet Data Center.
The European IXP Model
The U.S. IXP Model
Chapter 13 – The IXP Playbook..
The Value of an IXP.
A Simple Calculation of the Financial Value of an IXP..
The IXP Playbook Tactics.
Tactics That Manipulate the Value of the IXP.
Tactic 1. Bluff the Size of the Population
Tactic 2. Build a Network Umbilical for Later IXP Migration..
Tactic 3. The Group Buy-In
Tactic 4. Buy-In Key Players.
Tactic 5. Divide and Conquer
Tactic 6. Beachhead Verticals and Niche Markets
Tactic 7. Extend the Dominant IXP
Tactic 8. Prevent Rogue IXPs
Tactic 9. Swim with the Fishes..
Tactic 10. Bundling.
Tactic 11. Build and Maintain Population Stickiness..
Tactic 12. Strengthen Peering Population..
Tactic 13. Demonstrate Leadership
Tactic 14. Purchase Data Center During Downturns..
Tactic 15. Drop Peering Cost.
Tactic 16. Price on Value.
Chapter 14 – The Future
Appendix – The Peering Simulation Game.
The Origins of the Peering Simulation Game
In a Nutshell.
Introduction and Definitions
The Game Board.
Strategies Seen in the Game.
How This Simulation Is Different from Peering Reality..
About DrPeering Press
Preface – Note to the Reader
This is an exciting time to be working on the Internet. Today hundreds of millions of users are connected to this global network, using it as part of their daily workflow. Many of the most innovative applications reach viral popularity literally overnight, and multi-millionaires are being made every day.
Many of these emerging services grow to require better than commodity Internet services purchased at the edge of the Internet. To continue the wave of mass adoption, these services require flawless performance at massive scale, achievable only by connecting directly to the core of the Internet. Massive volumes of Internet traffic are exchanged at the core of the Internet in a sort of open Internet transit marketplace. This area is where the largest Internet Service Providers (ISPs) in the world interconnect, where the largest content providers interconnect with the largest regional ISPs, where the content delivery networks offload their traffic directly onto the broadband networks. The information and strategies described in this book will enable the next generation of Internet services companies to connect to and leverage the performance benefits at the core of the Internet as well.
This book is the assimilation of thousands of discussions with hundreds of the smartest peering coordinators in the world. I spent over a decade on the road, travelled to almost every continent, racking up over 500,000 air miles and spending over $500,000 in travel budget. I attended every Internet Operations conference so I could understand how Internet interconnection worked and document what I learned. I documented what I learned, citing the sources when allowed, and then walked others through the white paper to see if I got it right. After about 100 walkthroughs, I had a white paper that documented a particular aspect of Internet interconnection. Over 10 years I went through this process to produce 12 peering white papers. This base research is the raw source material for this book.
The information contained in this book is valuable. As a consultant, I help executive teams build effective peering and transit strategies. I also advise investors on the core Internet industries (ISPs, carriers, colocation centers, IXPs, CDNs, etc.). For these engagements I am paid quite a bit, and about 70% of the information I provide during these consulting engagements is information contained within this book. The rest of the information is provided on-site as the teams develop their strategies and investment theses. On-site consulting costs a lot of money, but to these teams, collectively, their time is worth a lot more money. They prefer a two-day on-site consulting engagement where they learn together and develop a strategy based on the information I collected during the last couple of decades. Internet interconnection and colocation is a highly specialized area of expertise, and there are only a few of us that can deliver this type of on-site consulting. This book provides this proven valuable information in a much more cost-effective package.
My hope and expectation is that this material will help the next generation of peering coordinators, network engineers, network architects, and business leaders understand and leverage the power of peering at the core of the Internet.
About this Book
This book is not about technology. There are many great books about the protocols, hardware, and algorithms. One can look at the protocols, the hardware, and the algorithms, but that doesn't tell the story about how the Internet works at the core.
This book is about the core of the Internet, the relationships between the companies that operate the central infrastructure, the "interconnection machinery" that makes the Internet work. It describes the business relationships that simultaneously demonstrate symbiosis and grudging interdependence. This book models the Internet as a Global Internet Peering Ecosystem, and describes the players, their power position, and the corresponding motivations and, indeed, predictable behavior.
This book includes everything you need to know about the Global Internet Peering Ecosystem to make the critical decision: when and why does your company need to connect to the core of the Internet? Some parts of this book will be of interest to you immediately, and some parts will be of interest to you later on. All of it is essential for business people, executives, and technical staff focused on the Internet.
Together, we will take an outward-in view of the Internet.
Section I Connecting to the Edge of the Internet
Section II Connecting to the Core of the Internet
Section III The Global Internet Peering Ecosystem
Section IV The Tricks of the Trade: The Playbooks
We will start by looking at how most companies attach to the edge of the Internet, and learn about the industry terms. We then dive into the core of the Internet, where the largest ISPs and content companies in the world peer their networks. Then we take a 30,000-foot view on the Global Internet Peering Ecosystem, and recognize the commonalities across the globe. With this holistic view of the Global Internet Peering Ecosystem, we will finish by examining the "Tricks of the Trade"—the playbooks collected from some of the smartest peering coordinators and Internet Exchange Point Operators in the world. These playbooks document how ISPs obtain peering where they otherwise would not have been able to. The IXP playbook documents how the smartest IXPs build, grow, maintain, and attack or defend their critical mass of peering participants.
This book takes the reader beyond "The Internet is a network of networks" and documents the internal workings of Internet interconnection and the strategic nature of these interconnections. This information is essential for any Internet services company building out at scale.
This book is for anyone who wants to learn how the core of the Internet works.
Internet Service Providers will benefit from understanding the methods and clever maneuvers surrounding interconnection used around the globe. This book highlights some of the issues and clever tactics that seasoned peering professionals see after years in the field.
Internet Exchange Point Operators (IXPs) will find this book helpful for building a peering ecosystem in their facilities. They will also better understand their customers’ mindset and motivations. This book presents the financial value proposition that Internet Exchange Point Operators provide to customers. The business case for peering at an Internet Exchange Point is clearly explained. Most important to IXPs, this book documents the tactics that the smartest IXPs in the world have used to build and grow, attack and defend a critical mass or peers. This material has been used as training material for IXP staff as well as for their customers and investor base.
Hardware vendors have found this material of practical value for better understanding their customers’ deployment environment, their applications, and their motivations. While this book was not written explicitly for this audience, the larger network equipment manufacturers are now becoming keenly interested in this material.
Students have used the previously released white papers in their classes for years. This book, rewritten from this earlier material, provides a good starting point for understanding the Internet core. The book provides a pragmatic industry view on interconnections that are essential for identifying research projects that are relevant.
Investors in colocation, data centers, IXPs, or large-scale Internet services will find this book to be an invaluable resource in understanding how the Internet actually works at the core.
This book is an assimilation of thousands of discussions with some of the brightest peering coordinators in the world between 1994 and 2011. A subset of these people is listed in the acknowledgements. Many more of them provided valuable data and insights but asked to remain anonymous. What you hold in your hands represents over fifteen years of primary research.
Notes from the field.
Stories and Anecdotes to Make a Point
Throughout this book I include stories and anecdotes that illustrate a particular point. I will format them as I do here to set them apart from the generalizations made in the book. When I consult with clients and when I present peering workshops, I have found that these stories help bring to life the points presented in the material.
Much of this information was previously documented in my peering white papers and remains freely available on the DrPeering website. This research has been used at universities around the world. The research has been updated and the text rewritten and edited for clarity and readability.
In The Grand Design, Stephen Hawkings says that a model is a good model if it is "1) elegant, 2) contains few arbitrary or adjustable elements, 3) agrees with and explains all existing observations, and 4) makes detailed predictions about future observations that can prove or disprove the model if they are not borne out”.
While the modeling of the Internet core as an ecosystem is not of the scale or importance of his work, I believe the Global Internet Peering Ecosystem model is a good model. The model is 1) simple, 2) contains few variables, 3) agrees with the observations made around the world, and 4) provides detailed predictions on the behavior of the parties involved at the Internet core.
The Global Internet Peering Ecosystem has many characteristics of a living ecosystem. There are individual identifiable species that hold power positions within the ecosystem, complete with motivations and predictable behaviors consistent with their position. The ecosystem itself has morphed over time to adjust to the stimulus placed upon it by the customers and their applications at the edge. The scope of this Internet ecosystem is global, so the stakes are very high, and information has been hard to gather. This book uncovers perhaps the most important topics on the Internet today—the evolution of the Global Internet Peering Ecosystem, and are we going in the right direction?
Palo Alto, California
Tell me what you think
All of this work is based on extensive primary research in the field. The previous white papers were discussed with the population, and comments, refinements, and adjustments were made as a result of the feedback. Through this stepwise refinement process, the research more or less accurately reflected the views of the population, or at least the population studied.
Feedback from readers will be a natural part of the process for this book as well. If you have any comments regarding how we could better improve the quality of this book, please send an e-mail to feedback@DrPeering.net and be sure to include the book title and ISBN number in the e-mail subject field.
Thank you for your feedback and suggestions.