Tour of CERN Computer Center and the Grid at CERN
Information Technologies Department
Tour of CERN
Computer Center
and the Grid at CERN
Welcome!
CERN IT Department
1 / 17
Tour of CERN Computer Center and the Grid at CERN
Computing at CERN
1. General Purpose Computing Environment
2. Administrative Computing Services
3. Physics and engineering computing
4. Consolidation, coordination and standardization
of computing activities
5. Physics applications
(e.g., for data acquisition/offline analysis)
6. Accelerator design and operations
http://cern.ch/it
CERN IT Department
2 / 17
Tour of CERN Computer Center and the Grid at CERN
LHC Data every year
• 40 million collisions per second
1 Megabyte (1MB)
A digital photo
• After filtering, 100 collisions of interest per second
1 Gigabyte (1GB)CMS
= 1000MB
5GB = A DVD movie
• > 1 Megabyte of data digitised per collision
recording rate > 1 Gigabyte / sec
• 1010 collisions
recorded each year
stored data
> 15 Petabytes / year
1 Terabyte (1TB)
LHCb
= 1000GB
World annual
book production
1 Petabyte (1PB)
ATLAS
= 1000TB
Annual production of one
LHC experiment
1 Exabyte (1EB)
ALICE
= 1000 PB
3EB = World annual
information production
http://cern.ch/lhc
CERN IT Department
3 / 17
Tour of CERN Computer Center and the Grid at CERN
LHC Data every year
Balloon
(30 Km)
LHC data correspond to about
20 million CDs each year
CD stack with
1 year LHC data!
(~ 20 Km)
Concorde
(15 Km)
Where will the
experiments store all
of these data?
Mt. Blanc
(4.8 Km)
CERN IT Department
4 / 17
Tour of CERN Computer Center and the Grid at CERN
LHC Data Processing
LHC data analysis
requires a computing
power equivalent to
~ 100,000 of today's
fastest PC processors
Where will the
experiments find such
a computing power?
CERN IT Department
5 / 17
Tour of CERN Computer Center and the Grid at CERN
Computing power available at CERN
• High-throughput computing based on
reliable “commodity” technology
• More than 35’000 CPUs
in about 6000 boxes (Linux)
• 14 Petabytes on 14’000 drives
(NAS Disk storage)
• 34 Petabytes on 45’000 tape slots
with 170 high speed drives
Nowhere near enough!
CERN IT Department
6 / 17
Tour of CERN Computer Center and the Grid at CERN
Computing for LHC
• Problem: even with Computer Centre upgrade,
CERN can provide only a fraction of the necessary resources
• Solution: Computing centers, which were
isolated in the past, will be connected,
uniting the computing resources of
particle physicists worldwide
Users of CERN
Europe:
267 institutes
4603 users
Out of Europe:
208 institutes
1632 users
CERN IT Department
7 / 17
Tour of CERN Computer Center and the Grid at CERN
What is the Grid?
• The World Wide Web provides seamless access
to information that is stored in many millions of
different geographical locations
• In contrast, the Grid is an emerging
infrastructure that provides seamless
access to computing power and data
storage capacity distributed over the
globe
CERN IT Department
8 / 17
Tour of CERN Computer Center and the Grid at CERN
One Web but many Grids
Grid development has been initiated by the academic, scientific and
research community, but industry is also interested.
•NASA Information Power Grid
•DOE Science Grid
•NSF National Virtual Observatory
•NSF GriPhyN
•DOE Particle Physics Data Grid
•NSF TeraGrid
•DOE ASCI Grid
•DOE Earth Systems Grid
•DARPA CoABS Grid
•NEESGrid
•DOH BIRN
•NSF iVDGL
CERN IT Department
•UK e-Science Grid
•Netherlands – VLAM, PolderGrid
•Germany – UNICORE, Grid proposal
•France – Grid funding approved
•Italy – INFN Grid
•Eire – Grid proposals
•Switzerland - Network/Grid proposal
•Hungary – DemoGrid, Grid proposal
•Norway, Sweden - NorduGrid
•DataGrid (CERN, ...)
•EuroGrid (Unicore)
•DataTag (CERN,…)
•Astrophysical Virtual Observatory
•GRIP (Globus/Unicore)
•GRIA (Industrial applications)
•GridLab (Cactus Toolkit)
•CrossGrid (Infrastructure Components)
•EGSO (Solar Physics)
9 / 17
Tour of CERN Computer Center and the Grid at CERN
5 big ideas
1. Sharing resources on a global scale
Main issues are trust, different management policies, virtual organisations, 24 hour
access and support.
2. Security
Main issues are well-defined yet flexible rules, authentication, authorisation,
compatibility and standards
3. Balancing the load
This is more than just cycle scavenging, (SETI@home),
need middleware to monitor and broker resources
4. The death of distance
Networks delivered 56Kb/s 10 years ago,
now we have 155Mb/s,
for the LHC anticipate 10 Gb/s
5. Open standards
Grid standards are converging, and
include Web services, the Globus
Toolkit, various protocols
CERN IT Department
10 / 17
Tour of CERN Computer Center and the Grid at CERN
Grid applications for Science
• Medical/Healthcare (imaging, diagnosis and treatment )
• Bioinformatics (study of the human genome and proteome to understand
genetic diseases)
• Nanotechnology (design of new materials from the molecular scale)
• Engineering (design optimization,
simulation, failure analysis and remote
instrument access and control)
• Natural Resources and
the Environment
(weather forecasting, earth observation,
modeling and prediction of
complex systems)
CERN IT Department
11 / 17
Tour of CERN Computer Center and the Grid at CERN
Grid@CERN
• CERN projects:
Worldwide LHC Computing Grid (WLCG)
• External projects with CERN participation:
Enabling Grids for E-SciencE (EGEE)
European Grid Initiative (EGI)
• Industry funded projects:
CERN Openlab
CERN IT Department
12 / 17
Tour of CERN Computer Center and the Grid at CERN
WLCG: Worldwide LHC Computing Grid
• Biggest scientific Grid project in the
world
• More than 140 sites in 33 countries
• 90’000 processors by end of 2009
• All Tiers 1 must store all the data
produced by LHC
• Should run 100 millions
programmes in 2009
• Used by 5000 scientists in 500
institutes
http://cern.ch/lcg
CERN IT Department
13 / 17
Tour of CERN Computer Center and the Grid at CERN
The EGEE vision
Access to a production quality GRID will change the way
science and much else is done in Europe
An international network of scientists will be able to model a new
flood of the Danube in real time, using meteorological and
geological data from several centres across Europe.
A team of engineering students will be able to run the
latest 3D rendering programs from their laptops using the
Grid.
A geneticist at a conference, inspired by a talk she hears,
will be able to launch a complex biomolecular simulation
from her mobile phone.
http://www.eu-egee.org
CERN IT Department
14 / 17
Tour of CERN Computer Center and the Grid at CERN
CERN Openlab III
Framework for evaluating and integrating cutting
edge IT technologies or services in partnership
with industry, focusing on future versions of the
WLCG.
Phase III focuses on
•
•
•
•
•
Security
Automation tools
Data replication and monitoring
Behaviour of a network of thousand
machines
Platform: thermal optimisation, applications...
http://cern.ch/openlab
CERN IT Department
15 / 17
Tour of CERN Computer Center and the Grid at CERN
Computer Centre Tour
Ground Floor
• OpenLab
• CPUs and Disks
• CIXP: CERN Internet Exchange Point
Basement
• Storage Silos: >34 petabytes
• New water cooled racks
IMPORTANT – FOR YOUR OWN SAFETY
PLEASE DO NOT TOUCH
EQUIPMENT AND CABLES DURING VISIT
CERN IT Department
16 / 17
Tour of CERN Computer Center and the Grid at CERN
To know more about the Grid…
CERN IT Department
17 / 17