Internet Data Center : Build vs Buy Decision

Abstract

Internet Data centers provide a secure, high-capacity environment for communications companies and content providers to locate and host their Internet equipment. These centers provide the foundation for a variety of business models and Internet operations. The build versus buy decision for Internet Data centers is an important and multifaceted one; the issues span technical, financial, and business domains. The expertise involved in building and operating a highly secure and network-focused facility are non-trivial. Even the low-end data center requires both an immediate capital outlay and a recurring operations cost. The construction project itself requires months of planning and execution. Like most things, the first time one endeavors into a new area, potentially costly mistakes are made. Economies of scale can be realized and exploited only in large data centers. These issues must be contrasted with the desire for control over design and operation of the data center.

This paper explores some of the tradeoffs between building a data center and outsourcing the data center to a third party, aka leasing space. The scope of this research is limited to the common “core infrastructure”, that is, the systems that all Internet data centers require regardless of business model. We are focusing this research on the decision rule: when does it make sense to build your own data center and when does it make sense to outsource it?

The author has toured over seventy Internet data centers around the world and combines this experience with interviews of industry experts to highlight what they believe are the most significant tradeoffs. This paper presents a few summary graphs and tables that provide a current perspective on this decision. In the appendix is a vast list of on-line Internet Data enter Resources.

Webmaster’s Note: EARLY DRAFT RESEARCH. This research is based on dozens of conversations with data center folks from the industry. While this research went through the process of review, it was never released because no consensus could be reached on the definition of a Data Center for build vs buy comparison, or on even the basic requirements of the subsystems. The goal of this unreleased paper was to be able to say something like “If you need more than maybe 70 racks, you may save money by building and operating it yourself, otherwise, you should lease and I can prove it makes more sense to do so”, but this turned out to be a more difficult task as the interviewees could not agree on what one would have to build.

This research started when capital was readily available and an increasing number of Internet Service Providers started building their own data centers when it seemed clear that leasing would have been the better option. Many initiatives were led by Internet engineers that seemed convinced that they had the “right” way to build an Internet Data Center, the “right” tray layout, the “right” configuration for power, etc. and nothing else would do. However, financially, it didn’t seem reasonable, which begged the question, at what point would it be financially prudent to build your own data center?

From this graph we see that if we assume that both the build and buy decisions are at a scale where the average cost per square foot of $400/sq foot, the break even point is about 60 racks. That is, if the unit costs are identical and the lease price given to you is $1500/rack/month, then you should be indifferent between building your own data center and purchasing those services when you require 60 racks of conditioned operations space. Given the assumptions below, you will prefer to outsource if you expect to use less than 60 racks, and consider building your own if you require more than 60 racks.

Assuming these best case scenario, the question comes down to core competency; what business do you want to be in and where do you want to build your expertise. Typically, companies that build their own Internet Data Centers don’t build one, but build at least two facilities for redundancy. The load on an organization for building two data centers is greater than twice the burden, due to the need to interconnect the two, synch systems, etc. To continue along this line, companies that rationally build two facilities often build many more (order ten or more) for resale and build the staff to do so. Building a department of real estate, architecture, design, engineering, etc. fundamentally shift the focus of all but the very large organizations.

The other key issue that comes to play is the learning curve. All companies make mistakes and overlook underlying assumptions when building the first few data centers. Common snafus include real estate selection that significantly delays the installation and activation of fiber. Trenching and lighting fiber takes time and energy from companies that are already heavily oversubscribed.

Other mistakes overcome by the learning curve include issues with real estate. Multi-floor telco hotel landlords may inaccurately characterize floor loading capabilities; if not checked, the issue comes up during permitting, and can yield substantial delay while adding support or substantially reduce the rack density. Landlords have been known to change the rules regarding the interconnection among the tenants; in one case the landlord changed the rules regarding the use of the risers (conduits through which inter-floor cabling is done) requiring the use of a shared “meet me” room, at additional cost. These issues, unknown to the first time builders, will ad delay and cost to the first few Internet Data Centers. These and other snags are inevitable.

Key assumptions include:

1.Both: $400/sq foot build out costs for conditioned space

2.Buy: retail price $1,500/rack used for comparison,

3.Buy: No discount for buying more or larger cages

4.Build: Building Real Estate comes in 10,000 sq foot increments

5.Both: Conditioned space includes infrastructure (power, HVAC, Security systems, etc.)

6.Both: Does not include network bandwidth, hosting equipment, managed services, etc.

7.Build: Financing of build costs is available,

8.Build: Adequate building is available, and build time (time to market) is not an issue

9.Build: Sufficient network capacity (transport and transit) is available and not an issue

10.Both: Facility population is not an issue – ignoring collocation benefits and don’t expect to buy or sell services at the facility.

CLICKABLE INTERNET DATA CENTER MODEL

Summary Graph: Build vs. Buy

All Internet Data centers include some form of the following core systems:

1.Real Estate: Places to put customer equipment and support infrastructure

2.Infrastructure: Hardware and Software Systems required for data center operation including Power, Heating Cooling / Air Conditioning (HVAC), Security, etc. and monitoring equipment for the infrastructure.

3.Facilities Operations: Activities to ensure the infrastructure is operating properly.

4.Network: Support for connectivity into/out of data center (This includes both a sustainable business model and infrastructure support).

Modeling the Internet Data Center

Conclusion

So what does the conditioned real estate cost function look like? If we remove extreme location expenses from the equation (they can be easily added back to the financial models later) and assume that both facilities are of same scale and cost ($400/sq ft), and assume that the network access can be brought in, we end up with a graph like the typical buy vs. lease graph:

On top of this Internet Data Center model, Internet Telecommunications companies (telephone companies, Internet Service Providers, Content Distributors, Content Hosting companies, etc.) operate their network and systems equipment in configurations suitable for their business models. In this way (see figure below) the company (or companies) can add value and build reliable dependent systems within the conditioned environment.

The reason that the model is evolved in this way was to facilitate an apples to apples comparisons for the build vs. buy decision. Since so many Internet Data Center companies are bundling varying professional services and requiring the use of multiple of those services, it would be difficult to compare in a generic fashion their offerings against a build decision. Since all of these business models require the core Internet Data Center model as described above, we can isolate the core “rings” of the model for the build vs. buy comparison.

We will next describe these core systems required for both build and buy scenarios - click on the graphic for details on each ring of functionality.

Services on top of the Internet Data Center Model

We will present the Internet Data Center as a sequence of resources applied to real estate. As one adds infrastructure, management services, etc. the value of the new infrastructure increases. Real estate becomes conditioned power facilities with the addition of redundant power backup systems, HVAC systems sufficient to cool electronic equipment, etc. Add access control and security guards escorts, and the service is more valuable. Add multiple network transport providers (providing circuits to the rest of the world), and the value in further increased. Add multiple Internet Service Providers to provide Internet access, and the value of the real estate is much higher. The value grows as the rings expand outwards.

When companies build their Internet Data Centers, they do so with the expectation of building something on top of the code Data Center Operations model. Here is where the greatest division of facilities occurs - we’ll call them purpose-built data centers. A single owner computer center rests large computing platforms within the data center and acquires enough network capacity to service their needs; there is less need to access control, escorts, etc. than a multi-tenant shared data center facility. An Internet Exchange Point builds a business around luring enough ISPs into the building to peer with each other ; they need good security, access controls, multiple carriers and proximity to fiber, and a critical mass of other ISPs in the building to make it work. ISPs sometimes build their own data centers for their customers, with the goal of keeping that traffic on their networks while opening up their operations to multiple competing managed services companies, or hosting companies. All of these different purposes lead to different data center models specifically taylored to the business models.

Let’s look at the commonalities, since we wish to determine when it financially makes sense to build your own purpose-built data center versus lease space from someone else and build on top of that.