European vs. U.S. Internet Exchange Points

Abstract

Internet Exchange Points are just ethernet switches, aren't they?

This article spotlights a few of the key differences between U.S. and European IXes.


Introduction

Definition: An Internet (IX) is a layer 2 service to facilitate the interconnection between Internet Service Providers and Large Scale Network Savvy Content Providers.

 

These interconnects are either Peering or Transit interconnections, but the mission of most IXes is the facilitate Internet Peering wthin the Internet Region.

Definition: Internet Peering is the business relationship whereby companies reciprocally provide access to each others’ customers.

This definition applies equally to Internet Service Providers (ISPs), Content Distribution Networks (CDNs), and Large Scale Network Savvy Content Providers.

To illustrate peering, considerthe figure below showing a much simplified Internet; an Internet with only three ISPs: WestNet, MidNet, and EastNet.

Peering Diagram

WestNet is an ISP with green customers, MidNet is an ISP with blue customers, and EastNet is an ISP with red customers.

WestNet is in a Peering relationship with MidNet whereby WestNet learns how to reach MidNet's blue customers, and MidNet reciprocally learns how to reach WestNet's green customers.

EastNet is in a Peering relationship with MidNet whereby EastNet learns how to reach MidNet's blue customers, and MidNet reciprocally learns how to reach EastNet's red customers.

Peering is typically a free arrangement, with each side deriving about the same value from the reciprocally arrangement. If there is not equal value, sometime one party of the other pushes for a Paid Peering relationship.

Important as well is that peering is not a transitive relationship. WestNet peering with MidNet and EastNet peering with MidNet does not mean EastNet customers can reach WestNet customers. WestNet only knows how to get to blue and green customers, and EastNet knows how to reach only blue and red customers. The fact that they both peer with MidNet is inconsequential; peering is a non-transitive relationship.

Peering can be Public or Private, and both are supported at most European and U.S.-based Internet Exchange points.

How are European Internet Exchange Points different from U.S. based Internet Exchange Points? The models are a bit different.

European IXes and U.S. IXes tend to differ in:

Other than that, they are identical. The technology and the business rationale is essentially the same; facilitate the interconnection of ISPs.

Over the years, there are a few things that ISPs have found surprising as they extend their networks from Europe into the U.S. These differences between U.S. and European Internet Exchange (IX) points can be traced back to an evolutionary split that took place around 15 years ago. The European model for IXes is largely based on the successful model initiated by the London Internet Exchange (LINX), while the modern U.S. IX model is largely based on the carrier-neutral IX model initiated by the Palo Alto Internet Exchange (PAIX).

We will sketch the most notable distinctions graphically in this article.

The European Internet Exchange Point Model

In Europe, as shown in the figure below, Internet Exchange points have a handful of distinguishing features when compared with their U.S. cousins.

 

 

U.S. vs European IXes

The European Peering community would point with pride at the European IXes saying that they are both carrier AND colocation neutral. This means that the IXes in Europe are generally available at multiple colocation facilities operated by different companies, none of which is the IX operator. ISPs can then decide which colocation operator provides the facilities services needed, and can connect to the IX whichever site they choose.

European IXes don't usually have customers, they have members. They are often formed as a not-for-profit associations (44%) founded by members that are the ISPs that first connect to the fabric and seed its operation. The other common form of European IX organization is that of a service offered by an academic or regional development organization (42%). In either case, the focus is on providing mutual value, sharing the costs, and having some degree of community oversight over what they see as their organization. Even when they are not formal associations (e.g. NetNod), they tend to operate as if they were. Many European IXes also have activities outside the mere operation of the IX, including political, legal, and community outreach and research activities. Results are generally shared with the broader community.

Soapbox. This association structure for IXes has proven to be both a blessing and a curse. It helps get the IX launched and to critical mass; since the founding association members share in the benefits and the costs equally they all can exert some pressure to get additional ISPs into their IX. However, the 'same pricing for all structure' also limits the use of sales promotions and subsidies which a commercial IX could use to strategically grow its customer base. This is especially important during the early stages before an IX reaches critical mass, and the value of peering there is less than the cost of peering there.

Exceptions: There are three or so commercial IX operations in Europe. Netnod in Sweden is actually run as a nto-for-profit trust, but tends to act as a European membership organization.

Pricing in Europe for IX services tends to approximate a cost-based pricing model. This is reflective of the not-for-profit nature of the operation; if the not-for-profit makes too much excess profit its not-for-profit status is in jeopardy. The pricing decline then makes it easier for other to justify joining, increasing the economies of scale and allowing the operator to decrease the prices further. European IXes are distinct from the U.S. counterparts also in that they cooperate. There is a long history of European IXes sharing operations notes, growth trends, additional service offerings, pricing information, traffic statistics and customer status updates at public Internet operations fora like RIPE, APRICOT, and NANOG. Now let's compare the U.S. IXes across the same dimensions.

 

 

The U.S. Internet Exchange Point Model

Internet Exchange Points in the U.S. are largely for-profit commercial operations run by the colocation companies that house them.

US IX Model

This has several implications.

First, for-profit commercial IX companies generally have the ability to do strategic differential pricing. This means a U.S. IX can offer free colo and free switch ports for IXPs that will help lure in other ISPs into the building. For example, when Equinix Ashburn was launched, and there were no tenants, WorldCom was incented to come in with stock warrants in the IX. In Los Angeles the Any2 IX was free for tenants in 1 Wilshire, and in Miami the NOTA IX offered free ports to all tenants. The ability to incent the right ISPs into the IX is a tool that the U.S. IXes have that the European IXes generally do not have.

Second, since they are for-profit, the pricing in the U.S. for IX services tends to approach the price the market will bear, as opposed to the European prices which tend to approach the cost of operations.

Third, in the U.S. you don't tend to see many IXes interconnect with other IXes. The whole question of which IX benefits more comes into play as discussed in the previous Ask Dr Peering article on How Equinix beat MAE-East. Ultimately, these IXes compete aggressively on the colocation and IX fronts and, unless both sides think the deal is unfairly balanced in their favor, the inter-IX deals don't go forward. Especially within the same building that houses multiple IXes, there is much tension surrounding inter-building cross connects between different IXes customers.

Fourth, the focus within commercial IXes tends to be more towards revenue generating activities. While there is much outreach done here, there is a commercial ultimately revenue generating motivation behind most activities.

There are no members of commercial IXes - peers are customers. Customers pay, they don't vote. Decisions are made by the commercial IX operator, not by the population. Next, a soft point. You tend to see that U.S. IXes tend to compete with one another more than cooperate. The exception to this is when the customer base insists that the different IXes work together. This occurred for example when each U.S. IX started running their own peering forums. The customer base, faced with constant travel to so many peering events, insisted that the IXes pool resources to run a single group event which became the Global Peering Forum. It has turned into a singular must-attend event for U.S. peering coordinators that were otherwise splintered across many different events, so everyone won with this evolution.

Finally, U.S. IX public peering traffic volume is much lower than Europe - tens of Gbps vs hundreds of Gbps on each peering platform. Also, in the U.S. there is much more private peering than public peering. As DrPeering has documented in the past, there are valid arguments on both sides of the Public v Private Peering debate. The U.S. IXes are essentially venture capital-backed purpose-built secure 100,000 sq. ft. meet-me rooms facilitating thousands of private peering connections.

These are the differences that folks point to the most often. I've held back some of the more controversial points surrounding contract negotiations and terms, quality of colo and SLAs.

-- DrPeering

Thanks to Stephen Wilcox, Colin Corbett and John Souter for helping edit these articles, and to ianai and ras for their observations as well.


Presentation Materials

 

  1. Under Development for Peering Workshops

Mentions

Ask DrPeering Blog

Related Documents

  1. Internet Service Providers and Peering
  2. A Business Case for Peering
  3. About the White Paper Process
  4. The Art of Peering - The Peering Playbook
  5. The Art of Peering - The IX Playbook
  6. Chief Technical Liaison
  7. Ecosystems: 95th Percentile Measurement for Internet Transit
  8. Ecosystems: Asia Pacific Peering Guidebook
  9. Ecosystems: Evolution of the U.S. Peering Ecosystem
  10. Emerging Video Internet Ecosystems
  11. European vs US Internet Exchange Points
  12. Internet DataCenter Build vs Buy Decision
  13. Internet Service Providers and Peering
  14. Internet Transit Pricing Historical and Projections
  15. Modeling the value of an Internet Exchange Point
  16. NANOG History
  17. Peering: Motivations to Peer
  18. A Study of 28 Peering Policies
  19. Peering Policies
  20. Peering Simulation Game
  21. Peering: Top 10 Ways to Contact Peering Coordinators
  22. Peering: Top 10 Reasons NOT to peer
  23. Public vs Private Peering - the Great Debate
  24. The Folly of Peering Ratios
  25. Top 9 IX Selection Criteria
  26. Video Internet - The Next Wave of Massive Disruption to the U.S. Peering Ecosystem

 

Index of other white papers on peering

About the Author

Who is William B. Norton?

Executive Director, DrPeering International

Chief Strategy Officer, International Internet Exchange (IIX)

WIlliam B. Norton is the author of The Internet Peering Playbook: Connecting to the Core of the Internet, a highly sought after public speaker, and an international recognized expert on Internet Peering. He is currently employed as the Chief Strategy Officer and VP of Business Development for IIX, a peering solutions provider. He also maintains his position as Executive Director for DrPeering.net, a leading Internet Peering portal. With over twenty years of experience in the Internet operations arena, Mr. Norton focuses his attention on sharing his knowledge with the broader community in the form of presentations, Internet white papers, and most recently, in book form.

From 1998-2008, Mr. Norton’s title was Co-Founder and Chief Technical Liaison for Equinix, a global Internet data center and colocation provider. From startup to IPO and until 2008 when the market cap for Equinix was $3.6B, Mr. Norton spent 90% of his time working closely with the peering coordinator community. As an established thought leader, his focus was on building a critical mass of carriers, ISPs and content providers. At the same time, he documented the core values that Internet Peering provides, specifically, the Peering Break-Even Point and Estimating the Value of an Internet Exchange.

To this end, he created the white paper process, identifying interesting and important Internet Peering operations topics, and documenting what he learned from the peering community. He published and presented his research white papers in over 100 international operations and research forums. These activities helped establish the relationships necessary to attract the set of Tier 1 ISPs, Tier 2 ISPs, Cable Companies, and Content Providers necessary for a healthy Internet Exchange Point ecosystem.

Mr. Norton developed the first business plan for the North American Network Operator's Group (NANOG), the Operations forum for the North American Internet. He was chair of NANOG from May 1995 to June 1998 and was elected to the first NANOG Steering Committee post-NANOG revolution.

William B. Norton received his Computer Science degree from State University of New York Potsdam in 1986 and his MBA from the Michigan Business School in 1998.

Read his monthly newsletter: http://Ask.DrPeering.net or e-mail: wbn (at) TheCoreOfTheInter (dot) net

Click here for Industry Leadership and a reference list of public speaking engagements and here for a complete list of authored documents

About the White Papers...

The Peering White Papers are based on conversations with hundreds of Peering Coordinators and have gone through a validation process involving walking through the papers with hundreds of Peering Coordinators in public and private sessions.

While the price points cited in these papers are volatile and therefore out-of-date almost immediately, the definitions, the concepts and the logic remains valid.

If you have questions or comments, or would like a walk through any of the paper, please feel free to send email to consultants at DrPeering dot net

Please provide us with feedback on this white paper. Did you find it helpful? Were there errors or suggestions? Please tell us what you think using the form below.

 

Contact us by calling +1.650-614-5135 or sending e-mail to info (at) DrPeering.net

 

 

Back To DrPeering Home

DrPeering.net ©2014 | Privacy Policy | Terms and Conditions | About Us | Contact Us

The 2014 Internet Peering Playbook is now available on the iPad at the Apple Store and for the Amazon Kindle. The Kindle, ePub and PDF form are also perpetually updated on the DrPeering DropBox share.

Sponsors