Idea Transcript
ENERGY SAVINGS FROM ENERGY STAR-QUALIFIED SERVERS
A recent set of tests demonstrate that replacing an older server with a new ENERGY STAR-qualified model and modern operating system will save energy and deliver more processing power in the bargain
CONTENTS ACKNOWLEDGEMENTS................................................................................................................................................................ 1 OVERVIEW....................................................................................................................................................................................... 2 ENERGY STAR SPECIFICATION.................................................................................................................................................... 3 GOALS AND LIMITATIONS OF THE STUDY................................................................................................................................ 3 TEST METHODOLOGY.................................................................................................................................................................... 3 TEST ENVIRONMENT..................................................................................................................................................................... 4 WORKLOADS................................................................................................................................................................................... 5 Baseline Workload..................................................................................................................................................................... 5 Web Fundamentals.................................................................................................................................................................... 5 FSCT.............................................................................................................................................................................................. 5 RESULTS........................................................................................................................................................................................... 5 Baseline Workload . .................................................................................................................................................................. 5 Web Fundamentals.................................................................................................................................................................... 8 FSCT.............................................................................................................................................................................................. 10 WHY NEWER HARDWARE AND OPERATING SYSTEMS ARE MORE ENERGY-EFFICIENT............................................... 12 Hardware..................................................................................................................................................................................... 12 The Operating System............................................................................................................................................................... 12 CONCLUSIONS................................................................................................................................................................................ 14 Expected Annual Savings......................................................................................................................................................... 14 Savings Comparisons, Avoided Carbon Emissions............................................................................................................... 14
Etuscide mporent ecatem et labo. Et viderum a volorep udaepe quam imolorem aut quiscianis alis elendandis verum sinvele sedio. Ut omniend anderum enihit lam, ut que qui dolut ut entionsedis del mi, id et, in cust, sumquaeperum dolori quam lit faccuptae illorrore mos in cum es sum venisciis dios di desto core pa ditis mos qui de verum faceatusciis dolor maion eatempo saeserum harchit et as eum ea quam et vendi id quas re consequam ipsus as dolupta turionsenime omnihil luptaque quae volo inimos dollupt ionsequi dellita tendamet et ea voloratem quasperem voloria volum quam, apientus sitas vent, natia con con ex eum ut anda audi omnis ex eicienditati te plautempor aut minis et lant at. Vid modis sitis nonsequi de doluptatum faccus, sitiae distium non re poratem faccullo quiscias adipiendam laccae volorepudam qui volorepudae volorro que dem repro ommolor asit aut is di to doluptu ribust maiore, nimincit que volestrumet odi in cum ad etur? Qui que dolorit, solessum res dolorem dolo ma nonsend itaqui dolum dolore erument. Ficturissus. Optis dolupta ecerit expel il id et dello et videllam restem que officto rporatem nosam et qui vit acculpa cum res vercipieni dolorum ditint rerit experum ex ex ex eum unt ut optasit offic temquidissit minimol uptur? Ad expe sita nos earit optaque sitiis miliquae quia nis aceria pratio. Nam nime sandi doles quaepelia dere simus et, ut eosam, tem volenis mint aceptat omnima sero de venetusant, et, ommod es magnatium nobissit officit volorest aut quas atecum ut explici aecere natem nimi, volorereprae dios dios inciat et offic tectem faceri untur, voluptaquas unt hiliatquo et mossimpor seque is magni de et aut eaquos autem adigent ommoluptas et eic temporume ped ut dolesed et latium volut et laborep tatio. Est qui volora doluptaspere plibusd aectum as ex est ipsusandus cuptat. Edi con et ent laborio ipsam voloratem remqui temqui berunt etur? Edi blaciur si cuptatumet officae vollab ilitiae pediamendit et que qui doluptiae nulpa consequia con cum alicill iquassi consequ aturio. Ehenim sam, omnihil et es aut volupie nduciis utatia quas esedit etur modi cus maximus esequiatium evenis eaquos aut dellis volorro vitiunt quaestem ilignihic torrumque voluptat plab il ipsaecus dolori offic tem alit moloreprate culpa si rerit vid ex exces aut ametum, alitat. Gendes et aut facidiae consequost estioss intibere susae. Abo. Quas maxim etur? Caborrupta es estiae conecto volupit, int explit acepudae et atem ex estrumq uatiumquia ventiandam quam, ne maximoloria quam et alia essinis sandit, si omnis est aut archicia nos rest, eost, conecus minctiossus ulla nisse pratur, quo corae lab ium repe pos sus saerferem doluptame re reperia excerna tiassim vent doloreptae reriore pellamuscium quat ommo consed molorerepro odiam volorro omnit, simpore ndanditat. Velibus dolectum aliquid elignimet voloreiur, con postem eic te num, quiani nonsequi alit velloreped enimaximendi optate imporpo repero magni duscide ribeatem velignit min ni dolupta tiunt, ipsam fuga. Et harciist rectore, nus doluptur sit, consequ isquas aut eveliquae del imus volut et volupiducia doloreste eost am ut qui dolupta eatquodist, sequam sintur? Occum quiditiatem dolutem cupta sequiant. Mus imint laborit eaquod most rernatur seque quatent ionseque parchit lamusan dipsam, core nis dolupta
ENERGY SAVINGS from
ENERGY STAR-QUALIFIED SERVERS Prepared by The Cadmus Group, Inc. for U.S. EPA ENERGY STAR® August 2010
ACKNOWLEDGEMENTS A number of individuals made important contributions to this study. They include (in alphabetical order by organization) Greg Davis and Scott Faasse from HP; Mark Aggar, Sean McGrane, Dan Reger, Bryan Weinstein, Bruce Worthington, and Qi Zhang from Microsoft; Steve Ryan from the US EPA ENERGY STAR program, Mike Walker and Eric Butterfield from Beacon Consultants Network Inc. (an ENERGY STAR technical support contractor), Robert Huang from The Cadmus Group (an ENERGY STAR technical support contractor), and Tom Bolioli from Terra Novum (an ENERGY STAR technical support contractor).
1
OVERVIEW Data centers use a lot of energy — nearly 3% of the electricity consumed in the United States, according to an EPA report to Congress1. Because computer servers are at the core of data centers — and because the heat they generate drives air conditioning costs — they are a prime target for energy-savings measures. Deploying more energy-efficient servers is a very effective strategy for reducing energy consumption in the data center. In tests conducted for this study, a newer ENERGY STAR-qualified server running a modern operating system consistently used less power to deliver substantially better performance, compared to an older non-qualified model running an older operating system.
1
See http://www.energystar.gov/index.cfm?c=prod_development.server_efficiency_study
2
ENERGY STAR SPECIFICATION In May 2009, ENERGY STAR released its first energy efficiency specification for computer servers. To earn the ENERGY STAR, servers must offer the following features: ««Efficient
power supplies that limit power conversion losses and generate less waste heat, which reduces the need for excess cooling where they are housed;
««Improved
power quality, which provides buildingwide energy efficiency benefits;
««Capabilities
to measure real-time power use, processor utilization, and air inlet temperature, which improves manageability and lowers total cost of ownership;
««Advanced
power management features and efficient components to save energy across various utilization levels, including idle;
««A
Power and Performance Data Sheet for purchasers that standardizes key information on energy performance, features, and other capabilities.
GOALS AND LIMITATIONS OF THE STUDY Today’s servers deliver far more computing power than models introduced just three to four years ago. ENERGY STAR-qualified servers, however, provide that additional computing performance using roughly 30% less energy, according to EPA estimates. In late 2009, EPA wanted to validate its original savings estimates by measuring power consumed under various types of workloads for two similar servers: one ENERGY STARqualified, the other not. The goal was to realistically measure how much electricity a new ENERGY STARqualified model would save in a real-world operating environment, compared to a typical three- to four-yearold server.
3
Microsoft graciously offered to host a metering study at its Windows Server Performance Lab in Redmond, Washington, and HP kindly donated server equipment for the tests. Representatives from EPA, Microsoft, and HP alike participated in the testing, from the initial operating system installation process through the collection of test results. Before we describe that operating environment and our test methodology, it is important to note that a host of variables influence how much energy a server consumes: server hardware, server software, percentage of CPU utilization, input/output, and the amount of storage access a given workload requires. That said, it would have been too expensive and timeconsuming to conduct a study that looked at all of the possible hardware, software, and workload variables. As a result, the test team selected a typical threeyear-old server and a comparable new ENERGY STAR-qualified server that might be considered as a reasonable replacement. This scenario was intended to mimic the type of decision faced by IT administrators who are trying to save energy in their data center. The team then set out to document energy consumption for both servers over a wide range of workloads.
TEST METHODOLOGY The new ENERGY STAR-qualified server was the HP ProLiant DL360 G6, using an out-of-the-box configuration with a fresh operating system (OS) installation (Windows Server 2008 R2). We compared this to an older HP ProLiant DL360 G5 running Windows Server 2003 Service Pack 2, which was not ENERGY STAR-qualified. The G5 was also set up with the out-of-the-box configuration and a fresh OS installation. Table 1 contains detailed specifications for the server hardware provided by HP.
TABLE 1: SERVER SPECIFICATIONS SERVER
HP PROLIANT DL360 G5 (“OLD”)
HP PROLIANT DL360 G6 (“NEW” ENERGY STAR-QUALIFIED)
OS
Windows Server 2003 SP2
Windows Server 2008 R2
Default Power Management
HP Dynamic Power Saving Mode
HP Dynamic Power Saving Mode2 - OS Default Balanced Power Policy
Hardware Available June, 2006 to Public
March, 2009
Processor(s)
(2) Intel Xeon Dual-Core 5160 Processors (3.00 GHz)
(2) Intel Xeon Quad-Core X5560 Processors (2.80 GHz, HT Enabled, Turbo Disabled by OS3)
Cache Memory
4MB Level 2 cache
8MB Level 3 cache
Memory
32 GB (8 x 4 GB) PC2-5300 Fully Buffered DIMMs (DDR2-667)
32 GB (16 x 2 GB) PC3-10600R DIMMs (DDR3-1333)
Network Controller
Embedded Dual NC373i Multifunction Gigabit NICs
(2) HP NC382i Dual Port Multifunction Gigabit NICs
Storage Controller
HP Smart Array P400 Controller with 512MB BBWC, Smart Array P800 controller
HP Smart Array P410i Controller with 512MB BBWC, Smart Array P800 controller
Internal Drive
(2) 146GB SAS Disk drives
(2) 146GB SAS Disk drives
Optical Drive
IDE DVD-ROM/CDRW combo
Slim SATA DVD RW drive
Form Factor
Rack (1U)
Rack (1U)
Power Supply
(1) Hot Plug Fan and Power Supply (Not Rated)
(1) 750W Hot Plug Power Supplies (80+ Gold certified)
Both servers were delivered by HP to the Microsoft Server Performance Lab and were racked “as-is” – no special tuning was performed.
TEST ENVIRONMENT The test environment was as follows: ««Microsoft
Windows Server Performance Lab (climate-controlled server room, non-isolated hot/ cold aisles);
««Standard
rack (filled with active servers in hot/cold isle configuration with no containment);
««1
Gigabit Ethernet and 10 Gigabit Ethernet (fibre channel) network cards;
««Instek
GPM-8212R AC power meter (with RS232 communications cable)4;
««Re-purposed
servers acting as client machines (in separate rack);
««Controller
running the workloads (that is, controlling the clients) and interfacing with the power meter.
««SAS
arrays as external storage (for the Web Fundamentals and FSCT workloads);
2
Processor Clocking Control (PCC). For additional information, see section entitled “Why Newer Hardware and Operating Systems are More Energy-Efficient”.
3
Turbo mode is disabled in Windows Server 2008 R2 balanced mode for the X5560 processor, but is enabled in balanced mode for newer Intel processors.
4
The manufacturer’s data sheet claims that the accuracy of Watt readings (at 23°C±5°C) is ± 0.2% of reading and ± 0.2% of range.
4
WORKLOADS We selected three workloads for our tests: ««An
industry-standard power and performance workload (run as a baseline test)
««Web ««File
Fundamentals
Server Capacity Tool (FSCT)
BASELINE WORKLOAD The baseline workload is an industry-standard, CPU intensive benchmark used to compare power and performance among different servers. It measures power consumption for servers at different performance levels — from 100 percent to idle in 10 percent segments — over a set period of time. The graduated workload reflects the fact that processing loads and power consumption vary substantially over the course of days or weeks. WEB FUNDAMENTALS Web Fundamentals “Full Mix” is a web server workload based on Microsoft.com usage patterns and Microsoft IT proxy server traffic. Using the Web Capacity Analysis Tool5 (WCAT 6.1) load generator, a set of clients initiated by the controller generate HTTP requests against the target web server. The workload consists of requests for a combination of dynamic ASP. NET pages and static files, some of which hit the file cache. This test exercises the CPU, memory, disk, and network, and is a good workload for performance and scalability testing. A limitation of this workload is that it consists mostly of static file hosting and ASP Server Side Includes (SSIs) in order to exercise the server side scripting engine. There is no server side scripting beyond those includes. FSCT The File Server Capacity Tool6 (FSCT) is a capacity planning tool for Common Internet File System (CIFS), Microsoft Server Message Block (SMB), and SMB2 file servers. The tool is also useful for identifying performance bottlenecks for a file server workload. FSCT results include the maximum number of users
5
for a file server configuration and throughput for that configuration. This benchmark performs a lot of hard disk access and is very I/O intensive; it is generally unable to significantly stress the CPU and memory before saturating the network and/or disk I/O. For this particular test, it was necessary to install an additional 10 GB NIC and a higher performing RAID controller in order to stress the G6 system. For consistency’s sake, these hardware items were added to both the G5 and G6.
RESULTS Under both workloads and the baseline benchmark, the ENERGY STAR-qualified server, in combination with Windows Server 2008 R2, provided significantly lower energy consumption when performing the same number of operations as the previous-generation hardware and Windows Server release. Additionally, the ENERGY STARqualified server consumed substantially less power overall across all target loads in the Web Fundamentals and baseline tests. BASELINE WORKLOAD Our results show significant across-the-board lower power consumption at various loads on the ENERGY STARqualified ProLiant G6 server with Windows Server 2008 R2. On average, the G6 with R2 consumed 26% less power than the ProLiant G5 while handling the same target load. The savings were larger for lower load levels. The tables and graphs below detail the number of transactions per second and the average power consumption at each of the 10 target load levels tested. On average, the ENERGY STAR-qualified G6 server with R2 delivered performance-to-power ratios 271% higher than the non-qualified G5. Power-to-performance ratio is the ratio of useful work (transactions per second) performed per unit of power (watts) consumed by the system. The G6 server with R2 delivered consistently lower power usage over the older G5—as much as 36% less at the 10% target load level and 54% less at idle. At the 50% target load level, the G6 consumed 24% less power than the G5.
5
Available at http://www.iis.net/community/default.aspx?tabid=34&g=6&i=1467.
6
Available at http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=b20db7f1-15fd-40ae-9f3a-514968c65643.
TABLE 2: DATA FROM BASELINE WORKLOAD G6 Performance G6 Avg. (Transactions/ Power Second) (Watts)
G6 Power Efficiency Difference Difference (Performance/ In Power In Power Watts) Consumed7 Efficiency8
Load Level
G5 Performance
100%
147,566
346
427
420,092
307
1369
11%
221%
90%
133,887
337
397
380,126
288
1321
15%
233%
80%
119,010
Baseline Workload: Workload: G6 G6 Baseline 1600 1400 1400 1200 1200 1000 1000 800 800 600 600 400 400 200 0
284% 295% 312% 332% 271%
54% 36% 31% 29% 27% 24% 22% 20% 17% 15% 11%
332% 312% 295% 284% 279% 263% 252% 244% 233% 221%
26%
271%
1200 1000 800 600 400 200 0
Baseline Workload: G6 400 350 300 250 200 150 100 50 0
Load Level
Expressed as a percentage of G5 Average Power (Watts)
8
Expressed as a percentage of G5 Power Efficiency (Performance/Watts) 400 400 450 400 350 350 350 300 300 300 250 250 250 200 200 200 er (Watts)
7
r Efficiency mance/watts)
Average Power (Watts)
er (Watts)
279%
Load Level
Power Efficiency (Performance/Watts) Power Efficiency (Performance/Watts) Power Efficiency (Performance Average Power (Watts) Average Power (Watts) Average Power (Watts)
er Efficiency (Performance/Watts) Power Efficiency (Performance/Watts)
Efficiency ance/watts)
Difference Difference in Power in Power Consumed Efficiency
Load Level
Load Level
rage Power (Watts)
1400 Power Efficiency (Performance/Watts)
(Performance/Watts)
400 350 300 250 200 150 100 50 0
Power (Watts)
400 350 300 250 200 150 100 50 0
263%
1600 1400 1200 1000 800
1600 1400 1200 1000 800
350 300 250 200
r (Watts)
Load Level
450 400 350 300 250 200 150 100 50 0
252%
FIGURE 2: BASELINE WORKLOAD – G6 POWER AND POWER EFFICIENCY AT LOAD LEVEL
Baseline Workload: G5
Power (Watts)
Power Efficiency (Performance/Watts)
seline Workload: G5
244%
Power (Watts) (Watts) Power
FIGURE 1: BASELINE WORKLOAD – G5 POWER AND POWER EFFICIENCY AT LOAD LEVEL
r Efficiency mance/watts)
Power 70% 105,509 Performance Power Efficienc y rage Average 60% 90,402 Target Actual wer (Perfor ssj_ops Power Load Load 73,434 atts) mance/ 50% (Watts) Active 60,175 256 Idle 0 40%0 0 256 262 10%57.9 10.10% 15,143 262 30% 45,230 267 20%113 20.20% 30,248 267 273 30%166 30.30% 45,230 20% 30,248 273 281 40%214 40.30% 60,175 281 10% 73,434 15,143 291 291 50%253 49.20% 304 60%297 60.50% 90,402 304 0% 0 316 70%334 70.60% ###### 316 (Active Idle) 326 80%365 79.70% ###### 326 337 90%397 89.60% ###### 337 346100%427 98.80% ###### 346 252 ∑ssj_ops / ∑power = er =
wer
326 365 338,164 270 1255 17% G6 G6 Power Power Power 316 Performance 334 297,926 253 1177 Power 20% Performance Power Efficienc Efficienc Efficienc y304 y 236 yDifference Average 297 254,409 1078 Average 22% Difference Target Actual Target Actual (Perfor (Perfor ssj_ops Power (Perfor ssj_ops Power in Power in Power Load Load Load Load Consumed Efficiency mance/ mance/ 291 253 210,826 959 (Watts) 24% (Watts) mance/ 220 Active Active 2810 214 0 822 0 27% 0 Idle 0 169,079 119 Idle 0206 0 11954% 57.910% 9.90% 41,820 167 10%250 9.90% 41,820 16736% 250 332% 273 166 127,026 194 655 29% 11320% 20.20% 85,272 183 20%466 20.20% 85,272 18331% 466 312% 166 30% 30.00% ###### 194 30% 655 30.00% ###### 194 29% 655 267 113 85,272 183 466 31% 295% 21440% 40.00% ###### 206 40%822 40.00% ###### 20627% 822 284% 262 57.9 167 ###### 250 36% 25350% 49.80% ###### 41,820 220 50%959 49.80% 22024% 959 279% 29760% 60.10% ###### 236 60% 1,078 60.10% ###### 23622%1,078 263% 256 0 0 119 0 54% 33470% 70.40% ###### 253 70% 1,177 70.40% ###### 25320%1,177 252% 36580% 79.90% ###### 270 80% 1,255 79.90% ###### 27017%1,255 244% 26% 39790% 89.80% ###### 288 90% 1,321 89.80% ######Averages: 28815% 1,321 233% 427 100% 99.30% ###### 307100% 1,369 99.30% ###### 30711%1,369 221% 252 952 ∑ssj_ops / ∑power = 26% 952271% ∑ssj_ops / ∑power =
Power Efficiency Power Efficiency (Performance/Watts)
G5
G5 Power Efficiency (Performance/ Watts)
G5 Avg. Power (Watts)
6
Load Level
G6
G5
FIGURE 3: BASELINE WORKLOAD – POWER COMPARISON AT LOAD LEVEL Baseline Workload: Power at Load Level
Baseline Workload: Throughput Load LevelPower at Throughpu BaselineatWorkload: 450,000
Throughput Throughput Power (Watts) (Transactions/Second) (Transactions/Second)
Power Efficiency (Performance/Watts) Average Power (Watts)
Power Efficiency (performance/watts)
350 1600 400 1400 300 350 1200 Power Efficiency at Throughput Baseline Workload: 250 300 1000 200 1600 250 800 1400 150 200 600 1200 100 150 400 1000 100 50 200 800 50 0 0 600 0 0 90000 180000 270000 360000 450000 400 200 0
Throughput
0
100,000
400 Baseline Workload: Throughput at Load Level 400,000 350,000 450,000 300,000 400,000 250,000 350,000 200,000 300,000 150,000 250,000 100,000 200,000 50,000 150,000 0 100,000
350 300 250 200 150 100 50 0
50,000 0
0
200,000
G5
G6
FIGURE 5: BASELINE WORKLOAD – POWER COMPARISON AT THROUGHPUT LEVEL
G6
1,600
Power Efficiency Power(Performance/Watts) Efficiency (Performance/Watts)
Baseline Workload: Power Efficiency at Load Level 1,400
400 350 300 250 200 150 100 50 0
1,200 1,600 1,000 1,400 800 1,200 600 1,000 400 800 200 600 0 400 200 0
0
100,000
200,000
300,000
400,000
500,000
Load Level
Throughput (Transactions/Second)
G5
G5
Load Level
G6
G5
G6
G6
FIGURE 7: BASELINE WORKLOAD – POWER EFFICIENCY COMPARISON AT THROUGHPUT LEVEL
Power Efficiency (Performance/Watts)
Baseline Workload: Power Efficiency at Throughput 1600 1400 1200 1000 800 600 400 200 0
400 350 Baseline Workload: 300 400 250 350 200 300 150 250 100 200 50 150 0 100 50 0
Power at Load Level
Load Level
G5 Load Level G6 100,000
200,000
300,000
Throughput (Transactions/Second)
G5
7
Average Power (Watts) Average Power (Watts)
Baseline Workload: Power at Load Level
0
G6
400,000
G5
FIGURE 6: BASELINE WORKLOAD – POWER EFFICIENCY COMPARISON LOAD LEVEL Baseline Workload: PowerAT Efficiency at Load Level
Baseline Workload: Power at Throughput
500,000 G5
300,000
Throughput (Transactions/Second)
G5 Load Level G6
G5 Throughput AverageG6 Power (Watts) (Transactions/Second)
Average Power (Watts)
100,000
Load Level
Load Level
200,000 Power 300,000 400,000 500,000 Efficiency (Performance/Watts)
G5
G6
FIGURE 4: BASELINE WORKLOAD – THROUGHPUT COMPARISON AT LOAD LEVEL
Average Power (Watts)
G5
(Transactions/Second)
G6
G6
400,000
WEB FUNDAMENTALS Over the 10 target load levels tested, the ENERGY STAR-qualified ProLiant G6 with Windows Server 2008 R2 used an average of 28% less power than the non-qualified ProLiant G5 server. In addition, the G6’s performance-to-power ratio was, on average, 330% higher than the non-qualified G5. The G6 with R2 consistently consumed less power than the G5 across all target loads. At 10% target load, it consumed 33% less power than the G5; at 50% target load, the G6 used 25% less power than the G5. TABLE 3: DATA FROM WEB FUNDAMENTALS WORKLOAD G5 Performance G5 Avg. (Responses/ Power Second) (Watts)
Load Level
n avg pwr
0
8
264
265
18%
291%
318
62
62,918
253
249
20%
300%
312
56
55,910
236
237
24%
320%
292
53
48,921
221
221
24%
320%
277
48
41,922
211
199
24%
319%
274
40
34,923
205
170
25%
326%
271
32
27,936
197
142
27%
338%
268
25
20,935
188
111
30%
352%
264
17
13,962
182
77
31%
363%
261
8
6,985
175
40
33%
375%
256
0
0
119
0
54%
-
Power
330%
FIGURE 9: WEB FUNDAMENTALS WORKLOAD – G6 POWER AND POWER EFFICIENCY AT LOAD LEVEL
(Perfor
3
70%
48,921
6
80%
55,910
2
90%
8
100%
Power (Watts)
119 175
/Watts )
0 40
182
77
188
111
197
142
205
170
211
199
221 Load Level
236
Web G5 WebFundamentals: Fundamentals: Web Fundamentals: G6 G6 350 300 250 200 150 100 50 0
350 300 250 200 150 100 50 0
300 80 70 250 60 200 50 40 150 30 100 20 50 10 0
250 200 150 100 50 0
300 250 200 150 100 50 0
221 Load Level
Load Level Load Level
237
62,918 253 249 Power Efficiency Power(Performance/Watts) Efficiency (Performance/Watts) 69,978 264 265 Average Power Average (Watts) Power (Watts)
9
300 350 300 250 250 200 200 150 150 100 100 50 50 00
300
Average Power (Watts)
Target Responses/Sec 80 80 Load ond 70 70 0 600% 60 10% 50 50 6,985 40 40 13,962 20% 30 30 20,935 30% 20 20 40% 27,936 10 10 50% 34,923 0 0 60% 41,922
Average Average Power Power (Watts) (Watts)
mance Average Web Fundamentals: Web Fundamentals: G5 G5
Power Efficiency (Performance/Watts)
2
69,978
Power Efficiency Power Efficiency (Performance/Watts) (Performance/Watts)
5
68
Average Power (Watts)
7
324
Average Power (Watts)
8
Difference in Power Efficiency10
FIGURE 8: WEB FUNDAMENTALS WORKLOAD – G5 Efficien POWER AND POWER EFFICIENCY AT LOAD LEVEL cy
Power Efficiency (Performance/Watts)
0
G6 Power Efficiency Difference (Performance/ in Power Watts) Consumed9
G6
r n
r e s
G6 Performance G6 Avg. (Responses/ Power Second) (Watts)
Averages: 28%
Power Efficiency (Performance/Watts)
0 0 6 5 2 9 8 7 5 1 9
perf to pwr 21,959 100% change change 90% 19,752 0.54 0.36 80% 3.32 17,552 0.31 70% 3.12 15,360 0.29 60% 2.95 13,163 0.27 2.84 50% 10,969 0.24 2.79 0.22 40% 2.63 8,777 0.20 30% 2.52 6,584 0.17 20% 2.44 4,390 0.15 2.33 10% 2,196 0.11 2.21 0.26 0% 2.71 0
G5 Power Efficiency (Performance/ Watts)
Load Level
Power Efficiency (Performance/Watts) Power Efficiency Power (Performance/Watts) Efficiency (Performance/Watts) Average Power (Watts) Average Power Average (Watts) Power (Watts)
Expressed as a percentage of G5 Average PowerG5 (Watts)
10
Expressed as a percentage of G5 Power Efficiency (Performance/Watts)
U CPU zati CPU Utilizati (6 Utilization Throughput/W on (6 Throughput/W Throughput/W
Operations /
CPU Utilizati on (6 Throughput/W
8
G6
FIGURE 10: WEB FUNDAMENTALS WORKLOAD – POWER COMPARISON AT LOAD LEVEL Web Fundamentals: Power at Load Level 350
Power Efficiency (Performance/Watts) Average Power (Watts)
Differenc Web 300Fundamentals: Power Efficiency at Throughput e in 250 300 Power Difference 200 250 Consume in Power 150 d Efficiency 200 100 150 54% 5033% 375% 1000 31% 363% 50 30%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 352% 0 27% 338% Load Level 25% 326% 0 20,000 40,000 60,000 80,000 G5 G6 24% 319%Throughput (Requests/Second) 24% 320% G5 G6 24% 320% 20% 300% 18% 291% FIGURE FUNDAMENTALS WORKLOAD – 28%12: WEB 330%
form ce to wer atio 0 40 77 111 142 170 199 221 237 249 265
POWER COMPARISON AT THROUGHPUT LEVEL
265
THROUGHPUT COMPARISON AT LOAD LEVEL
Web Fundamentals: Throughput at Load Level 80,000 Web Fundamentals: Throughput at Power Load Level Web Fundamentals: at Throughput 70,000 80,000 350 60,000 70,000 300 50,000 60,000 40,000 250 50,000 30,000 200 40,000 20,000 150 30,000 10,000 100 20,000 0 10,000 50 30% 40% 50% 60% 70% 80% 90% 100% 0 0% 10% 20% 0 Load Level 0% 10% 20% 40% 50% 60% 70% 80% 90% 60,000 100% 0 30% 20,000 40,000 G5Load LevelG6Throughput (Requests/Second) G5
G6
G5
G6
FIGURE 13: WEB FUNDAMENTALS WORKLOAD – POWER EFFICIENCY COMPARISON AT LOAD LEVEL
Power Efficiency Power Efficiency (Performance/Watts) (Performance/Watts)
Average Power (Watts)
264
250300
350 300 250 200 150 100 50 0
200250 150200 100150 50100 0 50 0 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0
20,000
40,000
60,000
80,000
0% 10% 20% 30% 40% Load Level50% 60% 70% 80% 90% 100%
Throughput (Requests/Second)
Load Level
G5
G6
G6
G5
FIGURE 14: WEB FUNDAMENTALS WORKLOAD – POWER EFFICIENCY COMPARISON AT THROUGHPUT LEVEL Web Fundamentals: Power Efficiency at Throughput 300
G6
Web Fundamentals: Power at Load Level Web Fundamentals: Power at Load Level
350 300350 250300
Average Power (Watts) Average Power (Watts)
G5
Power Efficiency (Performance/Watts)
237221 249237 265249
Web Fundamentals: Power Efficiency at Load Level Web Fundamentals: Power Efficiency at Load Level 300
Web Fundamentals: Power at Throughput
200250 150200 100150 50100
250 200 150
0 50
0 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
100
Load40% Level50% 60% 70% 80% 90% 100% 0% 10% 20% 30%
50
Load Level
G5
0 0
20,000
40,000
60,000
Throughput (Requests/Second)
G5
9
236221 253236 264253
Average Power (Watts)
G5
80% 312292 5653 80% 55910 70% 17552 15360 48921 Throughput70% (Requests/Second) 90% 19752 318 62 90% 62918 80% 17552 312 56 80% 55910 100% 324318 6862 100% 69978 G5 G6 90% 21959 19752 90% 62918 100% 21959 324 68 100% 69978 FIGURE 11: WEB FUNDAMENTALS WORKLOAD –
Throughput (Requests/Second) Throughput (Requests/Second)
Load Level
G6
80,000
G5
G6 G6
80,000
FSCT In this workload, the ProLiant G5 was able to reliably serve 2800 users with a load of 256 operations per second before the server reached its limits. The ENERGY STAR-qualified ProLiant G6 with R2 was able to more than triple this with 9443 users and 881 operations per second before reaching its limits. Despite this tripling of performance the G6 with R2 was able to serve 881 operations per second using less energy than the idle power of the G5. Note that the G6’s CPU was barely stressed during this test. This was a function of the nature of the workload (as mentioned previously), additional processor cores, and hyper threading.
TABLE 4: DATA FROM FSCT WORKLOAD – G5
Users
Overload11
Throughput (Operations / Second)
Average Power
CPU Utilization (4 Logical Processors)
Power Efficiency (Throughput/ Watts)
560
0%
51
276
3.70%
0.18
1,120
0%
102
279.7
10.30%
0.36
1,680
0%
154
284.2
16.40%
0.54
2,240
0%
205
295.4
31.30%
0.69
2,800
1%
256
317.8
62.00%
0.81
All higher levels had excessive overload
TABLE 5: DATA FROM FSCT WORKLOAD – G6
Users
Overload
Throughput (Operations / Second)
Average Power
CPU Utilization (16 Logical Processors)
Power Efficiency (Throughput/Watts)
560
0%
51
175
0.70%
0.29
1,547
0%
142
185
1.50%
0.77
2,534
0%
232
190.1
2.70%
1.22
3,521
0%
323
193.8
4.00%
1.67
4,508
0%
414
196.7
5.50%
2.1
5,495
0%
504
198.7
7.20%
2.54
6,482
0%
595
200.1
8.60%
2.97
7,469
0%
688
201.8
9.70%
3.41
8,456
0%
784
203.9
12.70%
3.85
9,443
1%
881
209.6
17.30%
4.2
All higher levels had excessive overload
11
Overload is a condition in which a system can’t handle requests fast enough and starts dropping them. The number represents the percentage of requests with no valid response.
10
600
800
G5
G6 G5
1000
Average Power (Watts) Average Power (Watts)
Average Power (Watts)
0
G6
G5
CPU Average FSCT: Power at(Operatio Throughput Utilizatio Throughp ns / Power 203.9 FSCT: Power at Throughput Users Overload Second) (watts) n (4 LPs) ut/Watt 350 200.1 560 0% 51 276 3.70% 0.18 300 1120 0% 102 279.7 10.30% 0.36 196.7 250 1680 0% 154 284.2 16.40% 0.54 200 0% 205 295.4 31.30% 0.69 190.1 2240 150 2800 1% 256 317.8 62.00% 0.81 100 175 50 G6 0 200 400 600 800 1000 0 200 400 Throughp 600 800 1000 Throughput 0 200 400 600 800 1000 ut Throughput (Operations/Second) CPU / Second) (Operatio Average (Operations Throughput (Operations/Second) Average Power (watts) Throughput G5 ns /G6 Power Utilizatio Throughp G5 (watts) G6 n (16 LPs) ut/Watt Users Overload Second) 560
0%
51
175
0.70%
0.29
1547
0%
142
185
1.50%
0.77
2534
0%
232
190.1
2.70%
1.22
3521
0%
323
193.8
4.00%
1.67
FIGURE 4508 17: FSCT WORKLOAD NUMBER OF 0% 414 – 196.7 5.50% USERS COMPARISON AT504 THROUGHPUT LEVEL 5495 0% 198.7 7.20% 6482
0%
595
200.1
8.60%
7469 Number 0% of Users 688 at201.8 9.70% FSCT: Throughput
Number of Users
10000
2.1 2.54
784
203.9
12.70%
3.85
9443
1%
881
209.6
17.30%
4.2
8000 6000 4000 2000 600
800
70%
50% 284.2 40%
50%
30% 279.7 20% 276 10%
0
60% 40% 30% 20% 10% 0 0%
0
200
G5
Average Power (Watts)
Average CPU Utilization
40% 30% 20% 10% 0% 600
11
G6
600
800
1000
800
G6
FSCT: Power at Throughput
50%
G5
400
Throughput (Operations/Second)
60%
400
Throughput (Operations / Second) G6 G5 G6
4.5 4 3.5 3 2.5 2 1.5 1 0.5 0
G6
Throughput (Operations/Second)
G5
FSCT: Power Efficiency at Throughput
1000
70%
200
Throughput (Operations/Second)
FIGURE 18: FSCT WORKLOAD – POWER EFFICIENCY COMPARISON AT THROUGHPUT LEVEL
FSCT: CPU Utilization at Throughput
0
50 100 150 200 250 300 200 400 600 800 1000 Throughput 0 Throughput 200 (Operations/Second) 400 600 800
Average Power (watts)
Throughput (Operations/Second)
G5
G6
FSCT: CPU Utilization at Throughput FSCT: CPU Utilization at Throughput
317.8 70% 295.4 60%
0%
0 400
1000
FIGURE 16: FSCT WORKLOAD – G5 CPU UTILIZATION G6 COMPARISON AT THROUGHPUT LEVEL FSCT G5 Power at Throughput
3.41
0%
200
1000 800
2.97
8456
0
200 400 600 800 0 Throughput 200 400 600 (Operations/Second) Throughput (Operations/Second)
G5 15: FSCT WORKLOAD – POWER FIGURE Throughp FSCT G6 Power at Throughput COMPARISON AT THROUGHPUT LEVEL ut 350 300 250 200 150 100 50 0
0
Average CPU Utilization
400
Average CPU Utilization Average Power (Watts)
200
Throughput (Operations/Second) Throughput (Operations/Second)
Power Efficiency (Performance/ Watts)
0
1000
350 300 250 200 150 100 50 0 0
200
400
600
Throughput (Operations/Second)
G5
G6
800
1000
1000
WHY NEWER HARDWARE AND OPERATING SYSTEMS ARE MORE ENERGY-EFFICIENT
efficiency over previous generations of servers, according to the company.
HARDWARE
Most ProLiant servers also include the HP Power Regulator Dynamic Power Savings mode feature, which automatically optimizes processor power consumption based on server activity. Power Regulator is implemented in the system firmware and directly monitors the instruction load of the server processor(s) to determine the level of system activity. Power Regulator uses this information to continuously adjust the performance states, or p-states, of the processor(s) to match processor power consumption to the current workload without noticeably impacting overall system performance.
HP cites a number of reasons why its latest servers offer improved power efficiency over previous generations: common slot power supplies that are redundant, better DC voltage regulators, Intel Xeon 5500 processors that consume less power, and less power needed to operate cooling fans. HP ProLiant G6 servers make use of the company’s common slot power supply design that can be used interchangeably across multiple platforms. According to HP, these are more efficient at all power loads than previous generations of HP power supplies. Common slot power supplies allow planners to select a power supply that will operate close to its maximum efficiency for the planned server power load. For example, a 750-watt power supply would be the optimal choice for a server that has an average power load of 350 watts, since it would be 92% efficient at that load, according to the company. A 1200-watt power supply installed in that same server configuration would only operate at 88% efficiency. Although in this test a single power supply was used, redundant power supplies increase reliability. However, in the past this could result in lower power efficiency. For example, in G5 servers, both redundant power supplies are online simultaneously; this lowers the amount of power drawn from each supply, but has the potential to decrease the power efficiency of each one. A feature of the High Efficiency Mode (HEM) option on DL G6 servers is that one of the redundant power supplies can be kept in a standby state; this increases efficiency by allowing the remaining supply to support the full power load. The additional power supply is brought online only if the primary supply fails. The improved DC voltage regulators in the G6 servers convert the 12-volt DC from the power supply into the 5-volt, 3-volt, and other feeds used by the various system components. This results in an 8-point gain in DC power
12
Many ProLiant G6 servers use the Intel Xeon 5500 series processors, which have Intel’s Intelligent Power Technology. This is a set of features that can be used to lower power consumption of the processor and related subsystems when they are not fully utilized.
The power management mode used in our G6 tests combines the capabilities of HP Power Regulator with the Balanced Power Policy in Windows Server 2008 R2. This is achieved using an interface that Microsoft and HP jointly created called Processor Clocking Control (PCC). The operating system calculates the future performance requirements of each of the processor cores on the system and communicates these requirements to the DL 360 G6 using the PCC interface. HP Power Regulator manages the power controls on the processors and other components on the system to deliver the requested performance level for each core. PCC enables the hardware and software to work together to delivery optimal power efficiency for the workload running on the server.12 Additionally, ProLiant G6 servers can use up to 32 sensors to map the temperature profile inside the server. Instead of using a fixed fan speed curve, a proprietary feedback algorithm adjusts individual fan speeds to maintain specific temperatures. This prevents overcooling and lowers the overall power consumption of the fans. THE OPERATING SYSTEM Windows Server 2008 R2 offers a number of power control features as well as an optional “Enhanced
Details of the PCC interface can be found at http://www.acpica.org/documentation/related_documents.php.
12
Power Management” qualification program for servers. For starters, it includes three in-box power policies: Power Saver , Balanced (default) and High Performance. The default Balanced policy continuously alters the power states of the processors in the system in response to the utilization level of the workload. This ensures that processor power usage maps to the needs of the workload, with minimal impact to workload performance. R2 achieves additional power savings by combining processor power state control with a feature that consolidates work onto a smaller number of processor cores when workload utilization is low. This feature is referred to as Core Parking. Processor cores that aren’t doing any work are placed into a deep sleep state. This feature effectively scales the number of processor cores in active use. Other features such as Timer Coalescing and Intelligent Timer Tick Distribution (or Tick Skipping), extend the time that processor cores stay in deep sleep states by avoiding waking cores unnecessarily. The balanced power policy delivers power efficiency out of the box. For workloads that prioritize lowest latency and highest performance levels over power efficiency, the High Performance power policy can be used. Although not featured in this study, these power efficiency improvements apply to Hyper-V, offering a significant reduction in platform interrupt activity and enabling power savings and greater scalability for virtual machines (VMs). As described above, support for the new power management interface called Processor Clocking Control (PCC) was introduced in Windows Server 2008 R2. The operating system and platform use the PCC interface to coordinate on power management. Windows Server 2008 R2 uses the PCC interface to pass future processor performance requirements to the hardware, as a percentage of maximum frequency. The hardware is in direct control of the processor power states in this mode of operation and is responsible for delivering the requested performance. This enables both the OS and the platform to innovate and add value in their respective domains which results in improved power efficiency for the server. This is the power management mode used for the G6 testing detailed in this paper and is the default configuration for new G6 servers with Windows Server 2008 R2.
13
Although not leveraged in this study, Windows Server 2008 R2 supports the new Power Meter and Budget firmware (ACPI) interface that is included in the ACPI 4.0 specification. The interface can be used by Windows Server to discover power monitoring and budgeting hardware on the platform and to access power consumption and power budget information. Windows Server 2008 R2 exposes power information to remote management software using WMI (Windows Management Interface), which adheres to the DMTF Power Supply Profile v1.01. This interface can be used by developers to build software that can remotely access power meter and budget information and modify Windows Power Policy across groups of servers. System Center Operations Manager 2007 R2 uses this interface to provide centralized power management. The Windows Server 2008 R2 server hardware logo program includes an optional Additional Qualification (AQ) called “Enhanced Power Management”. This qualification indicates that a server supports the following features: ««Power
metering and budgeting hardware
««Power
Meter and Budget firmware (ACPI) interface
««Enabling
Windows power management
Hardware with this AQ will take full advantage of the power management features in Windows Server 2008 R2, and will natively support the new SCOM 2007 R2 power management features. The HP DL360 G6 server referenced in this paper is qualified for the Enhanced Power Management AQ. Although not leveraged in this study, remote power metering capabilities are required for ENERGY STAR compliance and provide datacenter administrators with a valuable window into the power consumption and cooling trends of servers in situ.
CONCLUSIONS Without doubt, server performance has increased over the past 3 years. However, better performance does not account for all -- or even the bulk of -- the energy efficiency improvements we documented. Instead, server hardware and software makers -- including HP, Intel, and Microsoft -- have worked just as hard to reduce platform power consumption, especially at lower utilization levels. These tests suggest they have had much success. EXPECTED ANNUAL SAVINGS Our findings imply that, at the average US commercial rate for electricity of 10 cents per kilowatt hour (kWh), the energy savings from a single ENERGY STAR-qualified server could range from $60 (at 50% utilization) to $120 (at idle) annually, or $240-$480 over the useful life of a server (4 years). In addition to using less energy themselves, ENERGY STAR-qualified servers substantially reduce cooling loads in data centers. A general rule of thumb suggests that one watt saved by a server has the added benefit of saving one to two watts of cooling power. This yields a total savings of between $480 and $1,440 over the useful lifetime of a server. It’s important to note that these power savings come with a substantial increase in performance—at 50% utilization, for example, the newer, more energy-efficient server handles over three times the workload, thereby reducing the number of systems needed to support the same load. SAVINGS COMPARISONS, AVOIDED CARBON EMISSIONS Because saving energy lowers demand on the nation’s power grid, it results in the generation of less electricity and thus prevents pollution, too. Our data suggests that a single ENERGY STAR-qualified server saves enough electricity to avert nearly ½ to 1 ton of carbon dioxide emissions, based on the assumptions stated above. Accounting for cooling savings makes it a total of 1 to 3 tons of carbon dioxide abated.
14