More about companies in this post: Amazon - Apple - eBay - Facebook - Google - Microsoft - Yahoo

Mega Data Centers

We haven't updated this post in a while, so thought it would be a good opportunity to look at new mega-facilities. The latest addition is the Linkedin facility in Oregon USA.

Mega-datacenters feature technical and operational innovation that has overturned previous notions of limits on energy efficiency. But does the focus on efficiency obscure an acceleration in ICT energy and water consumption and in CO2e production? While mega-data center operators are claiming exceptional power usage effectiveness (PUE) ratios, we're still talking about facilities that consume 20-200 MW apiece and emit a considerable carbon footprint unless powered by renewable energy. 2014 statistics that suggest the mega-datacenter boom is causing a decrease in the total number of facilities. The displaced facilities are presumably smaller and less efficient, so the media-datacenter boom is likely slowing the growth of total ICT resource consumption.

Mega data centers:
- modular containers that can hold 1000+ servers, along with their power and cooling
- ultra-dense server and cabinet packaging
- replacement of tradition chiller cooling with lower-energy technologies like water-based evaporative cooling or air-side (free air) cooling
- new power schemes ranging from demand-based dynamic voltage/frequency scaling to high-voltage DC power distribution to proprietary UPS technology.

It is unclear how or when these technologies will migrate beyond the mega data center niche. Some, like sophisticated trade-offs between running hot racks and equipment failure, only work in data centers with homogeneous ICT gear and cloud-type operations that allow for hot-swapping overheated units with service outages. On the other had, air-side cooling - use of the cooler air which surrounds most of the world's data centers most hours - is an widely underutilized option.

All this doesn't mean that smaller data centers can't achieve impressive PUE numbers, too. Interxion's Stockholm data center uses seawater cooling among other tactics to obtain a 1.09 PUE.

Company Facility PUE Status
 eBay Jordan, UT-USA 1.4 Target set 2010.05
 Microsoft Dublin, Ireland 1.25 2011.04
 Microsoft Chicago, IL-USA 1.22 target set 2009.06
 Microsoft Quincy, WA-USA 1.15-1.2 2011.04
 Google all 1.12 Trailing 12-month avg. 2013.04
 Google best 1.08 Trailing 12-month avg. 2013.04
 NCAR Cheyenne, WY-USA <1.1 Target set early 2013
 Yahoo Lockport, NY-USA 1.08 target set 2010.09
 Facebook Prineville, OR-USA 1.08 2011.11
 LinkedIn Infomart Hillsboro, OR-USA 1.06 2016.12

Mega-Datacenter PUE

A November 2014 report from IDC suggests that mega-datacenter are capturing an increasing share of ICT growth. "IDC expects the total number of datacenters (all types) deployed worldwide will peak at 8.6 million in 2017 and then begin to decline slowly. This shift will be triggered by a decline in internal datacenter server rooms starting in 2016 and internal server closets starting in 2017...Despite a decline in the number of datacenters, total worldwide datacenter space will continue to increase, growing from 1.58 billion square feet in 2013 to 1.94 billion square feet in 2018...The most significant development in datacenter construction is the growing importance of service provider mega datacenters, which are the primary server location for large collocation and cloud service providers. By 2018, these mega datacenters will account for the vast majority (72.6%) of all service provider datacenter construction in terms of space while also accounting for 44.6% of all new high-end datacenter space around the world (up from 19.3% in 2013)."

Mega data centers can be the focus of tax and regulatory controversies.


Update: 2012.05.21
Apple's Maiden data center promises to use 100% renewable energy and has many efficiency features, but the company is not publishing a target PUE.


Update: 2011.05.12
eBay describes the features of its Topaz datacenter in Utah: "…everything is running at 400V. This means we lose an entire level of transformers and deliver 230V to the servers. That’s a 2% efficient gain through the entire electrical system…a 400,000 gallon cistern which collects rain water and will be used as a our primary cooling source…a water side economizer, which allows us to use the outside air to cool the data center for more than half the year instead of running expensive chillers…technology that will dynamically match the power used by the pump and fan motors to the cooling loads ensuring we only consume the energy needed to support the compute load…fully contained hot aisle design that isolates the heated air from the cold air…closely coupled cooling units (in-row) to add additional capacity where it is needed…we can put anything anywhere and still ensure it meets its optimum cooling efficiency even with a mixed workload. We can support racks that are less than 1,000 watts to dense racks that generate more than 30,000 watts of heat (more that an industrial pizza oven)." Topaz received LEED Gold status in November 2010.


Update: 2011.11.17
Facebook reports that its Prineville, OR facility it achieving a PUE of 1.06 - 1.1 in actual operation, with a mode reading of 1.08. The full load test at commissioning was 1.07.


Update 2013.05,01
More immediate than future targets are Google's quarterly online reports of "the PUE results from all Google-designed data centers with an IT load of at least 5MW and time-in-operation of at least 6 months. We've put the latest numbers in the table above. Google sets the standard in mega-data center transparency. The company also says, "we're currently using renewable energy to power over 30% of our operations."


Update: 2016.12.01
The LinkedIn data center in Hillsboro OR reports the most efficient PUE on our list. An interview with facility staff in Data Center Frontier reveals innovative cooling emerges as a key efficeincy driver. "[Because the] rear door heat exchanger technology [allows] transferring the heat generated by the high density compute nodes to water closer to the source, we are able to leverage the naturally cool ambient Oregon air nearly 220 days a year to reject that heat, as opposed to using energy to create cool water." " using rear door heat exchangers, we drove containment to the rack level. This approach gave us an unprecedented level of control in managing the cooling resource to exactly where and when needed. The improved control allowed us to raise the delivered water temperature. With a cold air objective of 75 degrees, the chilled water supply can be as warm as 67 degrees. Producing warmer chilled water saves energy year round and expands the number of ‘free cooling’ days. There are additional benefits to warmer water, namely eliminating condensation risk. For any acceptable range of relative humidity, the pipes will be above the dew point. Less insulation on pipes lowers construction and maintenance costs as well."

The LinkedIn data center was awarded an Uptime Institute Efficient IT Award. The Linkedin facility is a partnership with STACK INFRASTRUCTURE, formerly known as Infomart.


Update: 2011.05.03
Microsoft reports, “The company’s Dublin facility maintains a PUE of 1.25.…Microsoft’s new, fully modular datacenter in Quincy takes the best practices gained from research and development further and maintains a PUE of 1.15-1.2, while reducing the typical datacenter construction time of two years by half."

Update: 2009.09.25
More on the Google/Belgium and Microsoft/Dublin mega data centers, which are both now online.

Update: 2009.06.29
Microsoft announced that its Dublin, Ireland and Chicago, USA data centers will come online in July. Current/potential critical power ratings are 5.4/22 MW and 30/60 MW, respectively. Microsoft is aiming for a "PUE yearly average calculated at 1.22".

National Center for Atmospheric Research (NCAR)

Update: 2013.05.02

Moved to separate post due to the significant discussion about this facility and the additional information received.


Update 2010.09.20
Yahoo's Lockport, NY "Chicken Coop" is aiming for a PUE of 1.08.