Accelerating business innovation;
a Technology Strategy Board programme
Energy efficiency and research computing
RUGIT Away Day, 24th Jan 2008
Dave Berry
Deputy Director, NeSC &
Technology Lead, Grid Computing Now!
[email protected]
www.gridcomputingnow.org
Contents
Accelerating business innovation;
a Technology Strategy Board programme
Background and goals
Desktop Grids for High Throughput Computing
Data centres and HPC centres
Ways forward
www.gridcomputingnow.org
GCN! Aims
Accelerating business innovation;
a Technology Strategy Board programme
Accelerate the benefits to the UK
economy of adopting modern
computing technologies, i.e.:
The creation of scalable, secure,
efficient ICT infrastructures,
For delivering IT services, linked to
business processes through a
service oriented architecture,
While achieving greater utilisation with
reduced energy consumption and
reduced costs
www.gridcomputingnow.org
Knowledge Transfer Network
Accelerating business innovation;
a Technology Strategy Board programme
Part of the Technology Strategy Board’s Innovation
Programme
Run by Intellect, NeSC and CNR
Activities
Web platform; user case studies; events; webinars; active
sector and regional programme
Communities of Practice
Green IT (including MBE KTN, BCS, …)
Transport Modelling, Grids in Health, Public sector IT, …
www.gridcomputingnow.org
Aims for this session
Accelerating business innovation;
a Technology Strategy Board programme
To expand our community of practice
Stimulate discussion
Share knowledge
Form relationships
Suggest a “best practices”
document
Input to policy (Unis, RCs,
government)
Want to learn as much as inform
www.gridcomputingnow.org
Desktop Grids
Accelerating business innovation;
a Technology Strategy Board programme
High Throughput Computing
Many experimental scientists are more interested in
number of jobs/month rather than instantaneous
computing power
One approach: use “Spare” Cycles
No need for air conditioning, etc.
But don’t we want to switch off machines at night?
Claim: desktop grids can give computing power for less
cost and less electricity
www.gridcomputingnow.org
Cardiff slides from James Osborne,
High-Throughput Computing Week
Central Manager
NeSC, 27-30 November 2008
master, collector, negotiator
Execute Nodes
1600 Workstations
Submit Nodes
30 Workstations
master, schedd, shadow
master, startd, starter
Based on a P4 3GHz PC with 512MB RAM
Power Consumption
Watts Consum ed
150
160
140
112
Watts
120
100
100
80
60
40
20
0
0
5
Off
Hibernate
Standby
0
Idle
Machine State
Office
Condor
Based on a P4 3GHz PC with 512MB RAM
Economic Viability
•
•
•
•
•
Makes sound financial sense
Hibernate saves £60 per year
Condor = £30 per year (max)
Dedicated = £150 per year
Condor is 5 times cheaper
Saving of Hibernate = Cost of 100W Electricity (Idle State) for 16 Hours out of 24
Cost of Condor = Cost of 150W Electricity (Condor State) – Cost of 100W Electricity (Idle State)
Cost of Dedicated = Cost of 150W Electricity (Condor State) + Cost of 100W Electricity (Air Con)
Based on a P4 3GHz PC with 512MB RAM
Environmental Impact
•
•
•
•
•
Makes sound environmental sense
Hibernate saves 650Kg CO2 per year
Condor = 325Kg CO2 per year (max)
Dedicated = 1,625Kg CO2 per year
Condor is 5 times greener
Saving of Hibernate = Cost of 100W Electricity (Idle State) for 16 Hours out of 24
Cost of Condor = Cost of 150W Electricity (Condor State) – Cost of 100W Electricity (Idle State)
Cost of Dedicated = Cost of 150W Electricity (Condor State) + Cost of 100W Electricity (Air Con)
Based on 10,000 P4 3GHz PCs with 512MB RAM
Across Campus
• Makes sound financial sense
– Hibernate would save £600,000 per year
• Hibernate 16 out of 24 hours
• Makes sound environmental sense
– Hibernate would save 6,500T CO2 per year
– Rainforest required = 52Km2
– Rainforest required = 40% area of Cardiff
Saving of Hibernate = Cost of 100W Electricity (Idle State) for 16 Hours out of 24
Cost of Condor = Cost of 150W Electricity (Condor State) – Cost of 100W Electricity (Idle State)
Cost of Dedicated = Cost of 150W Electricity (Condor State) + Cost of 100W Electricity (Air Con)
The best of both worlds?
Accelerating business innovation;
a Technology Strategy Board programme
What if we could power-off idle machines and wake them
up when Condor has jobs to run?
JISC Low Carbon ICT project
http://projects.oucs.ox.ac.uk/lowcarbonict/
Develop an institution-wide wake-on-LAN service
Monitor energy consumption across the University
Write and implement a communications strategy
Towards Low Carbon ICT conference
Oxford, 19th March 2008
www.gridcomputingnow.org
Data Centres and HPC Centres
Accelerating business innovation;
a Technology Strategy Board programme
Need: 2xProcessing
Capacity per annum;
Target: 60% Energy
Reduction from 1990
levels by 2050
www.gridcomputingnow.org
The scale of the problem
Accelerating business innovation;
a Technology Strategy Board programme
Power consumption by data
centres:
Estimated 1.5% of UK national
electricity generation
Peak consumption of 8GW
Estimated to rise from 46 TWH
in 2006 to 93 TWH in 2020
Comparable with consumption
by refrigeration…
… or greenhouse gases emitted
by aviation
www.gridcomputingnow.org
Technology Power Loss Chain:
Fossil Fuel – CPU Used
Fossil Fuel
35%
Heat Exhausted
65%
Electricity Generated
2.5 2.5
% %
95%
Data Centre
Transmission Losses
Transformer Losses
Data Centre
Cooling Losses
Equipment
40%
25%
35%
Power Infrastructure
IT Equipment
Network Equipment
65%
Servers
20%
15%
Storage Equipment
Servers
CPU
30%
45%
25%
Power Supply
Other Components
CPU
Idle Time Power
CPU Utilisation
20%
80%
CPU Load Power
0.5% of Fossil Fuel Energy
Slide from BCS DCSG
Policies and measurement
Accelerating business innovation;
a Technology Strategy Board programme
BCS DCSG model
Open source model of energy-efficiency for data centres
and servers
The Green Grid
Vendor consortium
EU code of conduct on data centres
Development of baseline measures
Voluntary contribution of data from subscribing
organisations
www.gridcomputingnow.org
Press Examples
Accelerating business innovation;
a Technology Strategy Board programme
Ultraspeed DC-based system in East London
Claims 30% power saving from use of DC
Extra 10% saving from diskless servers
http://pcworld.about.com/od/recyclin1/Data-center-claimspower-cuts.htm
Plan for green data farm in Lockerbie
Using renewable energy sources
Waste heat used to heat new “eco village”
http://www.redwasp.co.uk/newsitem.asp?id=280
www.gridcomputingnow.org
GCN! Webinar
Accelerating business innovation;
a Technology Strategy Board programme
The Business Case and Methods for the Green Data
Centre
Recording available on the web
Zahl Limbuwala
Chair, BCS Data Centre Specialist Group
Motivation and an introduction to the BCS model
Kate Craig Wood
Managing Director, Memset Ltd.
Practical steps to running a “carbon-neutral” data centre
www.gridcomputingnow.org
Finance and administration
Accelerating business innovation;
a Technology Strategy Board programme
Who sees the power bills?
Who calculates the total cost of ownership?
Does this affect purchasing decisions?
Trade-off: cost of reliability vs. cost of downtime
Don’t overspecify
Staff requirements
Specialised HPC kit may need specialised staff
Other kit may not
www.gridcomputingnow.org
Power engineering
Accelerating business innovation;
a Technology Strategy Board programme
Location
Near power stations
Or local generation (CHP?)
AC or DC?
Provisioning levels
Nameplate provisioning is inefficient
Cables
Oversize for lower resistance (can halve losses)
Route through cool underfloor area
www.gridcomputingnow.org
Example benchmarks
Dell 1U server power usage
2006 SC1425 Dual
Xeon
Idle
Max load
Label
2007 860 Single dual
core Xeon
0
100
300
200
400
500
Watts
Slide from Kate Craig-Wood
Graded UPS usage

Latest switch-mode based generation vastly more
efficient

96-96% vs. ~90%

Most efficient when fully loaded (98%)

Don't run at half-capacity

40KVA steps, rather than typical 500KVA steps
Slide from Kate Craig-Wood
Cooling
Accelerating business innovation;
a Technology Strategy Board programme
What target temperature?
Water or air?
Store heated water externally?
External air or recycled air?
Savings from fresh air cooling of 72% (Source: BCS)
Modular cooling
Cabinets or rooms?
Modelling and controlling heat flow
Waste heat
How to sell or reuse?
www.gridcomputingnow.org
Fresh Air Cooling
Accelerating business innovation;
a Technology Strategy Board programme
www.gridcomputingnow.org
State of the art?
Accelerating business innovation;
a Technology Strategy Board programme
“By careful design, matching the specification of the
plant to the needs of the machine(s) we have
brought the summer time cooling overhead down
from 60 - 65% (very typical of most computer rooms)
to 25 - 30%.
Further, by using freecool this drops to 7% in the
winter.”
www.gridcomputingnow.org
System utilisation
Accelerating business innovation;
a Technology Strategy Board programme
Server virtualisation
Load balancing
Note – this is typically done already for compute clusters
May still apply to other university systems
Choice of equipment
Multi-core, power management, etc.
Diskless servers
www.gridcomputingnow.org
Ways forward
Accelerating business innovation;
a Technology Strategy Board programme
Best practice document(s)
Perhaps proven + experimental?
Shared facilities
Shared data centres?
Shared compute clusters (a la NGS)?
Training for IT staff?
Case studies?
Input to policy (Unis, RCs, government)
www.gridcomputingnow.org
Benchmarks and Labelling
Power Report for <VENDOR> <DEVICE> <MODEL>
Under <BENCHMARK>
Processor:
Memory:
Disks:
2 x 2.8GHz Quad Core
4 x 2GB 667MHz
2 x 146GB SAS
IO Cards:
IO Cards:
PSU:
2 x 10W PCI-E 10GBE
2 x 18W PCI-E FC
2 x 600W
Power (Watts)
600
500
400
300
200
100
Load (%)
Sleep
Idle 0
Load
Sleep
Idle
Power
25
190
10
20
30
40
50
60
70
80
90 100 Peak
PSU
10
20
30
40
50
60
70
80
90
PSU
0
100 Peak
243 267 286 304 320 334 347 360 371 381 390
440
600
BCS Data Centre Specialist Group
Slide from BCS DCSG
More references
Accelerating business innovation;
a Technology Strategy Board programme
GCN! Webinar
http://tinyurl.com/2gtslj
BCS Data Centre Specialist Group
http://tinyurl.com/2dyy5t
EU Code of Conduct
http://tinyurl.com/2drxoh
HTC week
http://www.nesc.ac.uk/action/esi/contribution.cfm?Title=831
Technology Strategy Board
http://www.berr.gov.uk/innovation/technologystrategyboard
www.gridcomputingnow.org