Request for Proposal (RFP)
DSF01-032
APPENDIX A – CUSTOMER SPECIFICATION
This document forms Appendix A to the Request for Proposal (RFP) for Digital Services Framework
Agreement – RM1043, along with Pricing Matrix (Appendix B) and an Award Questionnaire (Appendix C).
CONTENTS
PROJECT START DATE AND TIMEFRAME
CURRENT SITUATION/ BACKGROUND INFORMATION
REQUIRED OUTCOMES
USER NEEDS
CAPABILITIES AND ROLES
PRICING MODEL
CUSTOMER LOCATIONS
TEST & DEVELOPMENT REQUIREMENTS
Digital Services Framework Agreement - RM1043
Document1
Page 1 of 10
Request for Proposal
Digital Services Framework Agreement – RM1043
PROJECT START DATE AND TIMEFRAME
The project is required to be delivered by January 2015, with a period for transition from January 2015
through to March 2015.
Key delivery dates
PROJECT PHASES
START DATE
COMPLETION DATE
Discovery
30/05/2014
[TBC]
Alpha
[TBC]
[TBC]
Beta
[TBC]
[TBC]
Live
[TBC]
March 2015
CURRENT SITUATION / BACKGROUND INFORMATION
Research Councils are the public bodies charged with investing tax payer’s money in science and research in
the UK. Each Research Council funds research and training activities in a different area of research ranging
across the arts and humanities, social sciences, engineering and physical sciences and the medical and life
sciences, together investing around £2.8 billion each year. The Councils employ around 12,000 staff and
support around 30,000 researchers in UK universities and in their own research institutes.
There are currently seven Research Councils:
● Arts and Humanities Research Council (AHRC)
● Biotechnology and Biological Sciences Research Council (BBSRC)
● Engineering and Physical Sciences Research Council (EPSRC)
● Economic and Social Research Council (ESRC)
● Medical Research Council (MRC)
● Natural Environment Research Council (NERC)
● Science and Technology Facilities Council (STFC)
All are Non-Departmental Public Bodies (NDPBs), established by Royal Charter and are independent legal
bodies outside of Government, accountable to Parliament.
Research Councils UK (RCUK) is a strategic partnership between the Research Councils, established in 2002
to enable the Councils to work together more effectively. Through working in partnership, the Councils aim to
enhance the overall impact and effectiveness of their research, training and innovation activities, contributing
to the delivery of the Government’s objectives for science and innovation.
Background
The NERC Environmental Big Data Initiative is a £13M investment to enhance the necessary cyberinfrastructure to facilitate access to and processing of “big data” from across the NERC science remit. “Big
Data”, in this context, may be defined as those environmental data that are large in file size, inter-related data
that may be complex and heterogeneous, or data that need to be brought together, manipulated or analysed
very quickly, particularly in real time. Analysis of Big Data presents opportunities both for new scientific
insights and research to transform the use and exploitation of environmental data and research in the wider
community.
As part of NERC’s response to the scientific imperative to provide technical solutions to data management
and analysis, a new national environmental data computing facility, JASMIN1, was established at the STFC
Rutherford Appleton Laboratory (RAL), Harwell, in 2012. Over the next two years, as part of the NERC
1
http://www.ceda.ac.uk/projects/jasmin/
Digital Services Framework Agreement - RM1043
Document1
Page 2 of 10
Request for Proposal
Digital Services Framework Agreement – RM1043
Environmental Big Data Initiative, JASMIN will be enhanced to provide a world-class data analysis facility to
the entire NERC community - previously access was primarily limited to climate scientists and, in partnership
with the UK Space Agency, some of the earth observation community. JASMIN will provide three types of
computing close to a very high performance data archive: batch computing, a virtual hosting service, and a
hybrid cloud (a private cloud configured to “burst” into commercial public clouds where necessary (see Figure
1)).
The cloud infrastructure will support both the Climate and Environmental Monitoring from Space (CEMS)2
facility and a new software platform for environmental research, the Environmental Research Workbench,
which will enable NERC researchers and others to easily take advantage of the science-enhancing
capabilities offered by cloud computing.
The Workbench will provide researchers with a range of resources (e.g. visualisation, modelling and analysis
tools, data discovery and retrieval services, documentation and tutorials) that will continually be added to as
the Workbench is used increasingly by the community. Building upon the concepts explored by the NERC
Environmental Virtual Observatory (EVO)3 proof-of-concept project, particular focus will be paid on making
the Workbench, and its resources, easy for researchers to use. In so doing, a key hurdle for the many NERC
users who lack experience using high performance and cloud computing infrastructures will be addressed.
A budget has been allocated over a 2-year period (2013-15) for a programme of activity to establish the
Workbench, populate it with resources (services), and roll it out to the environmental sciences community. Of
this, approximately half has been allocated to the development of the basic Workbench, as described in this
document. It is anticipated that the majority of the Workbench’s functionality will be bought-in from third-party
suppliers, as off-the-shelf services, through bespoke software development, or a combination of the two.
Figure 1: Two views of the NERC Big Data analysis environment: the left panel shows a service orientated
view. Application layers such as Environmental Research Workbench are deployed through a SaaS (Software
as a Service) cloud service model. The cloud can also provide IaaS (Infrastructure as a Service) and PaaS
(Platform as a Service) for applications development and data analysis.
The IaaS interface will provide the ability for administrators to create virtual appliance templates. Such
templates are stored descriptions of configurations for one or more virtual machines (VMs). This configuration
consists of ISO images together with CPU, memory, storage and network settings desired for the VMs. These
templates may be deployed to create instances i.e. virtual appliances. A virtual appliance is a running
instance of the template. It will contain one or more virtual machines configured as prescribed by the
template. One example of a virtual appliance is a compute cluster. The template would contain the
specification for a number of virtual machines each with the required batch processing software installed on it.
2
http://sa.catapult.org.uk/cems/climate-and-environmental-monitoring-from-space/
3
http://www.evo-uk.org/
Digital Services Framework Agreement - RM1043
Document1
Page 3 of 10
Request for Proposal
Digital Services Framework Agreement – RM1043
Another example is a LAMP (Linux, Apache, MySQL, PHP) stack. LAMP is a common pattern used to enable
developers to easily create web applications. The template would consist of a VM with the Apache web server
configured with PHP together with another VM with a MySQL database installed.
Underlying the cloud services, a virtual layer can be used for the provision of managed hosting for raw
systems configured for science use. In parallel, a batch compute service provides the ability for longer parallel
data analysis jobs to be queued appropriately. All this uses the basic high performance storage, network and
compute. The right hand panel shows the relationship with other parts of the national e-infrastructure, and
indicates the ability for jobs to “burst” into public clouds where necessary.
The Workbench will provide an intuitive, user friendly interface enabling development, deployment,
management and exploitation of services such as environmental data and models, data manipulation and
analysis tools, data visualisation and transformation tools for spatial and temporal data, workflow
management, commentary and description tools. Through this Workbench, users would develop lines of
research using different data, models and analytical tools, audit and share results exploiting the accessibility
and flexibility of the cloud while abstracted from technical details. The Workbench should be extensible,
capable of supporting a growing ecosystem of services that will serve the varying needs of the environmental
sciences community for many years beyond the timescale of its initial development (this project).
The requirements specified in this document do not cover the establishment of the underpinning cloud
infrastructure. However, it should be assumed that the underpinning infrastructure is to be a hybrid structure
comprising a private cloud having the option of bursting out to one or more commercial public clouds as
necessary. It can further be assumed that the underpinning infrastructure offers IaaS, PaaS and some SaaS
as generic services which the Workbench can utilise to build upon and host applications with. Workbench
resources will comprise services (everything as a service). Every delivered (digital) product should be
available as a service and ideally should be independent of the underlying technology. The services will
execute on virtual machines (VMs) offered by the IaaS infrastructure. Management of the system’s
resources, when deploying jobs/applications (e.g. load balancing, cloud bursting), should be invisible to endusers.
It is envisaged that the Workbench will primarily be used by NERC-funded environmental scientists (the endusers), who would use the resources and services described in this document. Three types of users are
envisaged:
●
Standard Users - limited knowledge about the underlying cloud hosting environment (the majority of
users will fall into this category);
●
Super Users - capable of configuring or optimising certain services/applications to run in the cloud via
a command line interface and who would exploit the cloud capabilities directly to their own native
applications;
●
System Administrators - able to access cloud management interfaces, to set-up and manage VMs
that are running applications belonging to their organisation. (System Administrators will be located at
organisations who are tenants on the cloud).
The project is required to be delivered over the 2014/15 Financial Year (April - March).
Implementation and support arrangements
A Service Owner will be appointed to oversee the implementation, maintenance, use and development of the
Workbench. This is likely to be a senior manager from within NERC, who will report regularly to the NERC
Information Strategy Group. The day-to-day running and maintenance of the Workbench will be the
responsibility of a Central System Administrator, co-located with the JASMIN system’s administrators in
Harwell. Local Administrators will be appointed in each of NERC’s Research Centres, with each providing
technical support and training to local users (Research Centre staff) and users from their relevant science
sectors (i.e. BAS: polar science; BGS: geological science; CEH: terrestrial & freshwater sciences; NOC:
coastal & oceanographic sciences; NCAS/CEDA: atmospheric science & earth observation). The Local
Administrators, along with the Service Owner, also will be responsible for promoting awareness and use of the
Workbench amongst the user communities.
Service Level Agreements (SLAs) will be in place between NERC and the Research Centres in order to
ensure consistency and quality of service. It is assumed that NERC will provide the necessary resources to
Digital Services Framework Agreement - RM1043
Document1
Page 4 of 10
Request for Proposal
Digital Services Framework Agreement – RM1043
maintain the system in the long-term. Further SLAs will be in place to govern the relationship between the
Workbench and those responsible for the underpinning JASMIN infrastructure (e.g. on the provision of VMs,
storage, etc.).
This notwithstanding, we require you to explain how you will support the Workbench in terms of bug fixes,
upgrade paths and second line support.
REQUIRED OUTCOMES & USER NEEDS
The provision of an Environmental Research Workbench and related services to cover the following:
● Provision of a software (SaaS) application (the Workbench) that will enable users to interrogate,
model and analyse data within a hybrid cloud environment (see Figure 1)
● Provision of professional services to work with us to implement the solutions, train and educate us on
the new system capabilities
Provision of a support for the Workbench in terms of bug fixes, upgrade paths and second line
support
NERC ID
Required Outcome
NERC Categorisation
User Interface - A memorable, easily identifiable, brand is
required for the Workbench; the overall “look and feel” should
be consistent with the NERC corporate style. Branding should
include a logo, NERC corporate text fonts and colour palette
that will be used consistently in all parts of the Workbench that
are visible to users.
1.01
1.02
User Interface
Users should be able to store their own files/resources (data,
documents, etc.) in their own folder space on the Workbench.
Users should be able to define their own folders and folder
structure and manage (add, copy, move, delete) their
files/resources therein.
User Interface
1.03
1.04
Users should be able to upload files/resources from their local
systems into their Workbench folder space, subject to their
assigned storage allocation. Conversely, they should also be
able to download files/resource to their local systems.
Searchable help pages, on-line manuals and tutorials should be
readily available for every part of the Workbench.
User Interface
User Interface
A website/portal should be designed, as the primary landing
page for the Workbench, providing basic contextual information.
The portal should allow users to register and log-in to the
Workbench. The Workbench should support single sign on e.g.
OpenID and Shibboleth.
1.05
1.06
1.07
User Interface
Having logged-in, a user-friendly interface should be provided
to all users, giving access to permitted resources/services,
according to their individual credentials.
Super Users logged into the Workbench should have access to
a command line interface. Their credentials should
automatically be authenticated when invoking the command line
interface from the Workbench.
Digital Services Framework Agreement - RM1043
Document1
User Interface
User Interface
Page 5 of 10
Request for Proposal
Digital Services Framework Agreement – RM1043
Users should be presented with a standard toolbox with a
baseline level of services for the Workbench. The toolbox
should contain a selection of standard tools (e.g. data
manipulation, model and workflow services and templates, as
described later).
1.08
1.09
User Interface
Users should also be provided with their own user toolbox for
the services they themselves have developed or acquired.
Users should be able to manage (create, delete, copy) and
share the contents of their own user toolbox with other users,
specific groups of users, or the whole user community.
User Interface
1.10
1.11
1.12
Part of the user interface will include a workspace area to allow
Users to easily configure their own basic applications.
Users should be able to add their own applications by adding
services into available templates.
User Interface
User Interface
Users should be able to run appliances/applications from the
Workbench without needing to know anything of the
underpinning infrastructure. Super Users, however, should
have the means/ability to optimally configure appliances for
their specific needs within predefined parameters.
User Interface
1.13
1.14
1.15
Workbench should be extensible, capable of supporting a
growing ecosystem of services that will serve the varying needs
of the environmental sciences community for many years.
A standard set of tools/services should initially be available from
the basic toolbox to enable users to manipulate, visualise and
analyse environmental data. The tools should be able to deal
with both spatial (raster, vector and point) and time-series data.
The data management tools should enable users to re-project
spatial data (e.g. between coordinate reference systems), resample gridded data, create multi-layer map views, abstract
(slice & dice) multi-layered data, etc.
1.16
There should be tools to allow users to easily, and interactively,
visualise (graphing, maps, etc.) the various types of spatial and
time-series data.
1.17
There should be tools to allow users to conduct statistical
analyses on various types of spatial and time-series data.
1.18
Tools should be available to enable users to discover, view and
download environmental data from different sources.
Digital Services Framework Agreement - RM1043
Document1
Data Manipulation Tools &
Services
Data Manipulation Tools &
Services
Data Manipulation Tools &
Services
Data Manipulation Tools &
Services
Data Manipulation Tools &
Services
Data Manipulation Tools &
Services
Page 6 of 10
Request for Proposal
Digital Services Framework Agreement – RM1043
1.19
1.20
Access to discipline-specific dictionaries, vocabularies and
thesauri should be available from the Workbench to allow users
to appropriately tag their data and metadata using standard
terms and descriptors.
The Workbench must be extensible and able to host, configure
and apply an increasing number of environmental models.
The Workbench should provide a library to store and manage
model output data. Metadata related to model applications (e.g.
input data, parameter settings, model version number, postprocessing software, user comments etc.) should be held for
every set of output data. The documentation should follow a
metadata creation framework which conforms to suitable
standards for the discipline and ensures provenance of the
model output is assured.
Data Manipulation Tools &
Services
Data Manipulation Tools &
Services
Modelling Services
1.21
The Workbench should provide an interface to construct and
manage automated analysis workflows. This should allow
visualisation of the components and tools available for linking
(such as data and modelling services) into an analysis chain.
1.22
1.23
1.24
1.25
Workflow Services
The workflow management facility should be able to orchestrate
different services allowing for pipelining of data, asynchronicity,
service failure and interaction with break-points in the workflow.
The facility should be able to import common workflow formats
such as Keplar, VisTrials and Taverna.
Workflow Services
Workflow Services
The Workbench should enable System Administrators (see
Section 2) to manage user accounts, user groups, user storage
allocations, access rights and permissions, and the
management of the standard toolbox (see also Security).
System Administration
1.26
1.27
1.28
Policies and procedures will be required to manage the
retention and disposal of Workbench content.
The Workbench will need to consume services from a range of
cloud providers so should protect the user from any proprietary
consideration when running or constructing services.
Ideally, the Workbench itself should be platform independent so
that it can run over any cloud service provider.
System Administration
Platform Agnostic
Platform Agnostic
Adequate security is required to ensure that the Workbench is
secure and that users’ data and applications are protected.
Measures should include: identity management; application
security; privacy, controls on the movement of data, logs and
audit trails, legal compliance.
1.29
Digital Services Framework Agreement - RM1043
Document1
Security
Page 7 of 10
Request for Proposal
Digital Services Framework Agreement – RM1043
A campaign to raise awareness and promote the use of the
Workbench amongst NERC scientists and the stakeholder
community will take place in 2014/15. The Supplier will provide
suitable training materials (User Guide, training manuals,
PowerPoint presentation).
1.30
1.31
1.32
1.33
1.34
1.36
Awareness & Training
The Supplier will deliver training to a small group of users that
will be early adopters of the Workbench and/or provide the first
point of contact and training to the NERC community
The Supplier should also provide a number of training options
for each of the user groups (Standard User, Super User,
System Administrator) specifying a day rate and maximum
number of trainee delegates.
We require the system to be operational 24/7 but require
managed service support only between the hours of 9am to
5pm Monday to Friday
We require any faults to be raised with the supplier for
resolution. Faults will be 2nd line having been assessed at 1st
line by our own IT Helpdesk
According to the Digital by Default Service Standard, the
various components that make up the Workbench are required
to be developed using the Agile software development method.
Awareness & Training
Awareness & Training
Managed Service
Managed Service
Development Approach
Wherever possible, the Workbench components will be based
on Open Source tools and services that have already been
developed elsewhere. Any bespoke software development that
may be needed will be required to use Open Source software
and (the product) will be openly available. All services that will
be developed should, ideally, be platform independent and
deployable on any platform/infrastructure, including the
JASMIN-based private cloud and/or the Workbench.
1.37
Development Approach
We expect your proposal to demonstrate:
●
●
●
●
●
Value for money
Market commitment
Identification of risk and its mitigation
A collaborative service and support approach
Innovation and a desire to continuously improve your offering.
Digital Services Framework Agreement - RM1043
Document1
Page 8 of 10
Request for Proposal
Digital Services Framework Agreement – RM1043
CAPABILITIES AND ROLES
Lot 1
CAPABILITY
CUSTOMER’S REQUIRED OUTCOME
Agile Delivery Management
Strong Management of potentially multiple suppliers, together with the need
to co-ordinate integration work with the customer teams, will be essential to
ensure timely delivery of the solution.
Timely delivery of system components to ensure programme launch and
delivery is not delayed.
Software Engineering and
Ongoing Support
Development of a comprehensive and robust solution meeting business and
user needs in an intelligent and intuitive way.
Lot 2
CAPABILITY
CUSTOMER’S REQUIRED OUTCOME
Product Development &
Service Design
The existing user and business need research is validated, together with
further research from users to define the product.
Front End Design &
Interaction Design
Development of screen designs and interaction flows that allow users to
complete workflow tasks and find information. Embedded contextual help,
both for interactions, completeness guidance and compliance requirements
should enable the user to complete their task without having to leave the
system to view external resources.
Content Design &
Development
Works on the content to drive/make clear the user journey, analyzing
information design and writing content in the most user-centric way – across
all formats and products.
Lot 3
CAPABILITY
CUSTOMER’S REQUIRED OUTCOME
User Research
Additional user research is completed and documents to ensure all user
needs are reflected in the product. Research continues throughout the
design, build and operation of service.
Embedding Agile
An agile delivery culture is developed that addresses the challenges of a
distributed team, and enables the effective delivery of the project from all
staff involved.
Digital Services Framework Agreement - RM1043
Document1
Page 9 of 10
Request for Proposal
Digital Services Framework Agreement – RM1043
PRICING MODEL
Customer’s preferred pricing model or models, for SOWs that may be awarded as a consequence of this
Further Competition, are shown in the following table:
PRICING MODEL
PROJECT PHASES
Capped time and materials
Discovery/Alpha/Beta
Fixed price
[Beta/Live]
CUSTOMER LOCATIONS
We require potential suppliers to be able to travel to any CE/NERC location but there is no requirement to be
based from a CEH/NERC location for the duration of the engagement. We will require suppliers to attend
meetings, workshops etc. at our locations from time to time.
UK REGION
CUSTOMER LOCATIONS: CITIES OR TOWNS
London and South East of England
CEH Wallingford, Oxfordshire
West of England and Wales
Swindon, Wiltshire
TEST & DEVELOPMENT REQUIREMENTS
Test and development environments should be available throughout the build via the internet to ensure that
access to test functionality and provide feedback is not dependent on user location.
Code must be fully commented and documented as appropriate throughout the development of the project,
and must be available within GitHub.
Digital Services Framework Agreement - RM1043
Document1
Page 10 of 10