The Helix Nebula Marketplace
HNX
A federated cloud platform
for data intensive science
Helix Nebula Public Event, CERN Geneva, May 2014
Mick Symonds – Atos
Phil Evans – CGI
Jurry de la Mar – T-Systems
Agenda
• How we got here – what a long, strange trip
it’s been
– Mick Symonds, Atos
• HNX 1.0 – the current state of play
– Phil Evans, CGI
• The future – what’s coming next
– Jurry de la Mar, T-Systems
Helix Nebula GA, CERN Geneva, May 2014
2
March 2012: A European cloud computing partnership:
big science teams up with big business
Strategic Plan
 Establish multi-tenant,
multi-provider cloud
infrastructure
To support the
computing capacity
needs for the ATLAS
experiment
 Identify and adopt policies
for trust, security and
privacy
 Create governance
structure
 Define funding schemes
3
Setting up a new
service to simplify
analysis of large
genomes, for a deeper
insight into evolution
and biodiversity
To create an Earth
Observation platform,
focusing on
earthquake and
volcano research
May 2014: A European cloud computing partnership: big
science teams up with big business
Strategic Plan

Establish multitenant, multiprovider cloud
infrastructure

Identify and adopt
policies for trust,
security and
privacy

Create governance
structure

Define funding
schemes
To support the
computing capacity
needs for the
ATLAS experiment
Setting up a new
service to simplify
analysis of large
genomes, for a
deeper insight into
evolution and
biodiversity
To create an Earth
Observation
platform, focusing
on earthquake and
volcano research
Suppliers
Adopters
4
To improve the
speed and quality
of research for
finding surrogate
biomarkers based
on brain images
There is a broad range of
(competing) suppliers involved
Added
value/
people
Niche
offerings
Data and
information
Commodity
infrastructure
Large
Small
5
The EC is strongly supporting Helix Nebula
This is very welcome “seed money”, and the Suppliers involved have invested far more themselves
6
Flagship deployments
First results
• Proof of Concept stage within the Pilot Phase started
January 2012
• Each flagship was deployed with a series of providers
independently:
CERN, EMBL and ESA succeeded in deploying
scientific applications each involving tens
of thousands of jobs running at data centres
operated by Atos, CloudSigma and T-Systems
Helix Nebula GA, CERN Geneva, May 2014
7
Where does the Blue Box fit in?
Customers,
users
Suppliers,
providers
Front-end
Front-end
Front-end
Front-end
• Each supplier had a working cloud service, but they were all different
Helix Nebula GA, CERN Geneva, May 2014
8
Helix Nebula Blue Box services
REST API,
command line,
web app,
EC2 interface
Atos Canopy
Front-end
•
SlipStream
CloudSigma
Interoute
Front-end
Front-end
T-Systems
Front-end
They still are, but we have been working to make that irrelevant/transparent to
users
9
What do (these) customers need, in general?
• Easy access to services, at an acceptable cost
• Services based on basic, standard building blocks:
– storage
– pre- and post-processing
– access through public and research networks
• Ease of doing business
– short commissioning and deployment cycles
– modular services with minimal bureaucracy
– pay-as-you-go
• Data integrity and security
– including adherence to (local/European) regulations
• Ecosystem environment, adding value through:
– discovery, composition, collaboration etc.
Helix Nebula GA, CERN Geneva, May 2014
10
What is “success” to each party?
Level/sphere
Success
Politics:
e.g. the EU/EC
Public organisations benefit from cloud services, delivered by a thriving EU
cloud services industry, creating (preferably local) employment
Funding agencies
Reduced cost, from capex to opex, and simple predictable service
commissioning
Scientists
Simple access to facilities, and new opportunities emerging by combining
new opportunities within the ecosystem
IT providers within
institutes
A model for standardised services and their delivery by multiple providers,
including themselves
Service providers
A thriving market for service delivery, with significant revenue and
opportunities for adding value, including by finding synergies with users and
other suppliers
Technologists
Facilities, such as Blue Boxes, that are neat
Helix Nebula GA, CERN Geneva, May 2014
11
Timelines
Workshop EMBL
Workshop ESRIN
Strategic Plan agreed
TechArch doc
published
Blue Box and
ServArch docs
published
Catalyst for change in
Europe doc published
Initiative
FP7 project
General Assembly
(GA) 1, CERN
GA3 EMBL
GA2, ESA
public event
Flagships
Flagships selected
(CERN, EMBL, ESA)
2011
2012
Proof of Concept (PoC)
deployments start
2014
Pilot deployments start
(CERN, EMBL, ESA)
PoC deployments
complete
Helix Nebula GA, CERN Geneva, May 2014
2013
PoC deployments assessed (CERN, EMBL, ESA)
New flagships presented (PIC, ECMWF, UNESCO)
12
12
… and at that point, we
had reached an impasse
• We had a chicken-and-egg problem
– do we invest in a production platform, before we get
significant commitment to its usage?
– whereas the Demand side wanted something that
worked before they would commit to using it
• (Some of) the Supply side decided to “call
their/our bluff”, and just do it
– “Build a better mousetrap, and the world will beat a
path to your door”, Ralph Waldo Emerson
– and we were (and are) convinced we had a better
mousetrap
Helix Nebula GA, CERN Geneva, May 2014
13
Timelines
Workshop EMBL
Workshop ESRIN
Strategic Plan agreed
TechArch doc
published
Blue Box and
ServArch docs
published
Catalyst for change in
Europe doc published
HNX Market
launched
Initiative
LoI for Market
announced
FP7 project
General Assembly
(GA) 1, CERN
GA3 EMBL
GA2, ESA
public event
GA4, CERN
public event
Flagships
Flagships selected
(CERN, EMBL, ESA)
2011
2012
Proof of Concept (PoC)
deployments start
2014
Pilot deployments start
(CERN, EMBL, ESA)
PoC deployments
complete
Helix Nebula GA, CERN Geneva, May 2014
2013
PoC deployments assessed (CERN, EMBL, ESA)
New flagships presented (PIC, ECMWF, UNESCO)
14
14