Interest in energy-efficient computing has sparked a Vertatique discussion of energy-sensitive benchmarks beginning in 2007. We originally saw pages per kilowatt hour, "MIPS / Watt" or "Flops / Watt" and Peter Kooge's "performance / joule". More have since been proposed, including one from focused on CO2e, but none have emerged as definitive.

"Flops" denotes "floating point operation per second", a measure of computer performance. An exaflops machine would be capable of a quintillion floating point operations per second. 'Exascale computing' is the quest for machines that can execute 1 exaflop or more. These benchmarks are critical as designers contemplate the DARPA* challenge of an exaflops supercomputer. Kooge writes in IEEE Spectrum:

The biggest obstacle…by far is power. A modern supercomputer usually consumes between 4 and 6 megawatts—enough electricity to supply something like 5000 homes. Researchers at the University of Illinois at Urbana-Champaign's National Center for Supercomputing Applications, IBM, and the Great Lakes Consortium for Petascale Computation are now constructing a supercomputer called Blue Waters. In operation, this machine is going to consume 15 MW—more actually, if you figure in what's needed for the cooling system. And all that's for 10 petaflops—two orders of magnitude less than DARPA's exaflops goal.
If you tried to achieve an exaflops-class supercomputer by simply scaling Blue Waters up 100 times, it would take 1.5 gigawatts of power to run it, more than 0.1 percent of the total U.S. power grid. You'd need a good-size nuclear power plant next door. That would be absurd, of course, which is why DARPA asked…how to limit the appetite of such a computer to a measly 20 MW and its size to 500 conventional server racks.

Kooge goes on to explain the challenges inherent in this goal and suggest some approaches. The article also reminds how far we have come, comparing a 1997-vintage supercomputer of 1.8 teraflops consuming 800,000 watts of electricity with a contemporary game console delivering the same performance with less than 200 watts.

CNET, reporting on a November 2011 talk by Nvidia CEO Jen-Hsun Huang, writes "the industry target he set was to achieve one exaflop of computations with 20 megawatts by 2019. In this year's list of the Top500 supercomputers, the K supercomputer from Japan topped the 10 petaflops mark and consumes 12 megawatts of power." Huang's target can be expressed as 50 petaflops/MW. K is only 0.83 petaflops/MW or 1.7% of Huang's target. Note that Huang's target echos DARPA's, but also sets a date. Contrast this with Kooge's assessment that "…it might not be possible anytime in the foreseeable future."

IBM has used DARPA fundes to develop a very fast, very low power optical communications link aimed at exascale computing. An IBM researcher sees "reaching the exascale mark around 2020."

The Green500 ranks supercomputers according to megaflops-per-watt and not on total performance. IBM's BlueGene/Q tops the November 2011 rankings at 2026.48 megaflops/W or ~2 petaflops/MW. The site observes of the current leaders,"…the ten greenest supercomputers in the world continue to follow one of two trends: (1) aggregating many low-power processors like IBM BlueGene/Q and (2) using energy-efficient accelerators, typically from the gaming/graphics market…to complement the commodity CPUs…" (More on supercomputer energy efficiency.)

Japan began looking in 2010 at three performance-based ICT energy efficiency metrics:
• Servers: Watt/MTOPS (Millions of Theoretical Operations Per Second)
• Storage: Watt/Gbyte
• Networking : Watt/Gbps

IBM referenced one of Japan's metrics the same year when it reported, "The six System x servers announced in 2010 for which comparison models existed provide reductions in watts/MTOPS…of 87 to 98 percent over the previous generation server." Cisco uses the networking metric to compare the performance of its CSR-3 telecom router to competitive products and conclude, "Power Reduction Per Gbps 60%"(Slide 14).

In the IBM source cited above, the company describes its point-of-sale (POS) terminal a little differently: "…the SurePOS™ 700…delivers a 56 percent reduction in the maximum power consumption of the system per composite theoretical performance (CTP)…It also provides a 98 percent reduction in power use when the system is idle." CTP is usually based on MTOPS.

Some storage vendors are looking at capacity-per-watt measurements, the inverse of Japan's watts-per-capacity. This is reminiscent of the conflicting distance-per-fuel (miles/gallon) versus fuel-per-distance (liters/100k) metrics for automobile efficiency.

Cloud applications provider Salesforce.com advocates a "carbon per transaction" metric. The company has already put its metric to use: "…the average amount of carbon produced per transaction dropped from .10 grams in 2010 to .08 grams in 2011[1], a decrease of 20%!" The Salesforce article has links to a relevant white paper.

* DARPA is the US Department of Defense's Advanced Projects Research Agency, the R&D funding powerhouse that, among other things, sowed the seeds of the Internet.