Complex Problem Solving including GRID and
Research Networking Infrastructures
Contributions from the FP6 Internal Reflection Group
Draft report version 2.0 – 14th May 2002
IRG Complex Problem Solving including GRID and RNI
1
Draft V 2.0 - 14/05/02
Summary ........................................................................................................ 3
Excerpts from the Specific Programme............................................................. 5
Vision ............................................................................................................. 6
Complex Problems – Challenges and Motivation ............................................... 7
Grid and Peer-to-Peer...................................................................................... 9
Distributed and Shared Infrastructure for e-Learning ..................................... 12
Research Network Infrastructures ................................................................. 13
Global Monitoring for Environment and Security ............................................. 15
Annex I – GRID Activities .............................................................................. 16
Summary publicly funded R&D Grid Projects .................................................................... 16
Summary of GRID / P2P Activities by IT Companies......................................................... 23
ANNEX II – Complex Problems....................................................................... 26
Application Areas................................................................................................................. 26
Classification of Complex Problems.................................................................................... 29
Annex III – Draft - SWOT Analysis ................................................................. 31
Technology Areas................................................................................................................. 31
Grid computing................................................................................................................. 31
Peer-to-Peer Computing................................................................................................... 32
Research Network Infrastructures .................................................................................... 33
Mobile/Wireless IPv6 ....................................................................................................... 34
Application Areas................................................................................................................. 35
Distributed and Shared Infrastructure for e-Learning ...................................................... 35
Complex Systems in Industrial Engineering (examples) ................................................. 36
Global Monitoring for the Environment and Security (GMES)....................................... 37
Annex IV – List of Background Documents ..................................................... 38
Annex V – Members of the IRG....................................................................... 39
Annex VI – Glossary and Acronyms................................................................ 40
IRG Complex Problem Solving including GRID and RNI
2
Draft V 2.0 - 14/05/02
Summary
Potential research activities in the field of complex problem solving in science, engineering, business
and for the society, GRID and Peer-to-Peer (P2P) technologies able to solve those and the enabling
networking infrastructure are topics of this report. The report highlights different research perspectives
(both from a technical and application viewpoint) but does not pretend to be exhaustive.
The report starts with an overview of what is meant by complex problems in science, industry and
business. There is a wide variety of compute-intensive and/or data-centric problems and ways of
solving them. The requirements these problems have on the underlying computational, communication
and storage architecture can be quite different. The different types of players at all relevant levels of
the value chain that are interested in those problems or can contribute to their solution can be
considered as the potential drivers for Grid/P2P-like technologies that may become the basis for future
ICT infrastructures.
The GRID computing concept (a form of distributed computing) has shown considerable evolution
since its inception a few years ago. Initially the term GRID was used for describing the pooling of
distributed (local and/or remote) compute resources over the Internet. The massive integration of
computer systems (processor and storage) offers performance unattainable by a single system.
P2P technologies have been around in various forms (networking, computing, and applications) for
more than a decade but have become famous through file sharing and distributed applications (e.g.
seti@home). In P2P networks a group of computers communicate directly with each other rather than
via central servers (the opposite of client-server model). The connectivity and diversity of the Internet
is giving P2P computing a new role – a role that allows the growing computing and data resources at
the egdes of the network to achieve their full potential. P2P computing - forming a distributed
information infrastructure, in which all information assets and resources of an organisation are brought
together in a co-ordinated fashion - can be particularly effective in spanning geographical and
organisational boundaries. P2P computing is intended to complement existing client/server
applications and to open up new opportunities to take advantage of the computing assets and resources
of both individuals and organisations.
P2P and Grid computing are two collaborative computing technologies with different origins but many
common applications and concerns. Grids are historically high-end, predominately e-Science oriented.
P2P is more low-end, strongly commodity oriented. Many middleware requirements, like security, are
similar.
More recently the term Grid evolved to mean also an inclusive structure linking people, communities,
information, tools and facilities1 . Lately IT companies are picking up GRID/P2P concepts and see
them as means to deploy utility computing and as enabler for webservices.
Without widespread availability of fast and reliable Internet connectivity Grid and P2P applications
would be restricted to resource sharing within a lab or organisation. Today research teams and
research facilities, both in industry, business and universities are geographically dispersed, and only
high speed networks (comparable to the bus of the computer used) will enable them to share data and
computing infrastructure in real time. Global high-speed Internet connectivity assuring meaningful
performance end-to-end is the first layer of useful Grid architecture. High-speed networks open up
new possibilities for collaborative learning and researching. We have seen important shifts in the way
we use networks: rather than simply transmitting lots of raw bits, we are sending information that is
formatted in standard ways (html tags and data) that convey content and actions to be done (service
protocols, executable content). These developments lead to information networking and the “semantic
web”.
1
“The term GRID refers to an emerging network-based computing infrastructure providing security, resource access,
information and other services that enable the controlled and coordinated sharing of resources among virtual organisations
formed dynamically by individuals and institutions with common interests. The Anatomy of the Grid: Enabling scalable
virtual organizations, I. Foster, C. Kesselman and S. Tuecke, Int. J. Supercomputer Applic. 15, 3 (Fall 2001); see also
www.globus.org/research/papers.html
IRG Complex Problem Solving including GRID and RNI
3
Draft V 2.0 - 14/05/02
The SWOT analysis in Annex III try to make an in depth critique of the technologies and assess
opportunities for deployment.
The EU and individual countries have already funded several substantial grid-related projects
in the last two years, some for infrastructure and middleware development and some for
applications that take advantage of the Grid concept. They are all addressing different
research topics and fields spanning from High Energy Physics to bioinformatics and medical
applications, yet using a de facto US developed open middleware standard (with one notable
exception). Applications developed in academia today are often the basis for the commercial
applications of tomorrow. The Member States and Commission must ensure that this potential for
innovation is fully exploited.
Europe should adapt a coherent strategy with respect to computing and communications, which, on
one hand, takes an active role in the establishment of global standards and which, on the other,
promotes the early development of commercial applications. Follow some recommendations from
previous workshops:
•
Grid technology must be developed for an open, mobile and connected environment.
•
New developments need to focus on industrial applications both to establish a long-term
sustainability and to stimulate the rapid development of an infrastructure, which responds to real
commercial requirements.
•
There is an urgent need to bridge the gap between technology push and application pull in the
development of this infrastructure. Science-based ‘testbeds’ will not deliver industrial-strength
middleware.
•
Early industrial applications are needed to prove and stimulate middleware development. The
early involvement of EU software houses in the development of these new applications should be
a priority.
•
The development and integration of existing technologies into prototypes and early applications is
the most important short-term issue. The development of applications using results, for example
from the Globus project, is important to demonstrate both the validity and applicability of standards
and models.
•
The development of new technologies to enable the refinement of data into information into
knowledge is central to the functioning of any global infrastructure and is an important mediumterm objective.
•
Developments to support this infrastructure should be based on Open Source/Open Standards
following models such as those of Linux, MPI and the Object Management Group.
•
Joint US-EU-Japanese ‘global’ projects are important to establish a global infrastructure which
presents real business opportunities. It is important to complement rather than compete with other
initiatives in the US and Japan.
Integrated Projects (IP) and Networks of Excellence (NoE) are the primary instruments to implement
this new disruptive paradigm in FP6. However, existing (industry) players are generally not motivated
to develop or deploy disruptive technologies. The aim is to create critical mass and incentives
compelling enough to attract early adopters not just in the academic and research sectors but in
industry and business. Co-operation with national programmes is highly encouraged. Timely Public
funding and private investment could lead to significant payoff.
IRG Complex Problem Solving including GRID and RNI
4
Draft V 2.0 - 14/05/02
Excerpts from the Specific Programme
The group confirmed that the scope and nature of the Complex Problem Solving activities is well
described in the modified proposal for the Specific Programme of FP6.
Complex problem solving in science, engineering, businesses and for society:
The objective is to develop technologies for harnessing computing and storage resources which are
distributed in geographically dispersed locations, and for making them accessible, in a seamless way,
for complex problem solving in science, industry, business and society. Application fields include
environment, energy, health, transport, industrial engineering, finance and new media.
Research will focus on new computational models, including computing and information
GRIDs, peer-to-peer technologies and the associated middleware to make use of large scale
highly distributed computing and storage resources and to develop scalable, dependable and
secure platforms. It will include novel collaborative tools and programming methods supporting
interoperability of applications and new generations of simulation, visualisation and
datamining tools.
European researchers already enjoy one of the world’s fastest and most extensive research networks,
developed through the GEANT project. The support for the research networking infrastructure in FP6
and the extension of GEANT will be provided through the Specific Programme: Structuring the
ERA. The objective is to upgrade the research infrastructure to beyond 100 Gbit/s, enabling
researchers across Europe to share knowledge and collaborate on solving complex scientific problems.
Such networks will be a major cornerstone towards the realisation of the ERA.
Research into the GRID technologies in the IST priority and the upgrade of the research infrastructure
complement each other. In addition to applications in science, they will enable complex problems
solving for societal (e.g. environment, health,…), engineering and business needs. These will pave the
way for the full rollout of the next generation Internet. Close articulation is therefore needed between
the work on the research infrastructure in the relevant specific programme and the IST priority.
IRG Complex Problem Solving including GRID and RNI
5
Draft V 2.0 - 14/05/02
Vision
The continuing progress in computing and communication technologies (hardware and software) and
the use of advanced applications allow - like never before - tackling complex problems across all
domains of science and research and many dimensions of life (social, economical, political and
personal) and performing simulations which begin to match the real world. Advanced computing and
networking is becoming scalable and affordable and is no longer restricted to the realm of research or
big science. Advanced (local and wide area, fixed and wireless) networks constitute the pervasive and
persistent infrastructure to connect to information and instruments. Through progress in Web server
and browser technology access to content has become easy and intuitive.
Yet, the current computer is still based on the “von Neumann stored programme architecture”,
networks still follow Shannon’s “store and forward” concept. The above-described customary
advances are merely achieved through incremental technical improvements in the current paradigm.
The Grid and P2P computing model coupled with the concept of ambient intelligence is a step
towards a new paradigm. As computers are becoming intelligent devices and ubiquitous, users will
control their functioning and not vice-versa and be able to choose the resources they need to connect
to. User-to-user and user-to-community networking will transform bit/data processing into information
processing/sharing and ultimately knowledge processing. Users will be able to work across traditional
boundaries and utilise the most up to date tools, to combine data and models from many resources,
simulate complex interrelations. Access to information and knowledge will be via the Grid. The move
is away from the server-centric web model to a “many-to-many” model with direct interaction
between/amongst individuals.
The Grid and P2P should not be limited to the deployment of distributed computing on a known set of
computers, of static topology and a stand-alone network of computer nodes. Furthermore it must take
into account the dynamics of the computations. Nodes may join and leave the grid at any time. The
computational threads are allowed to travel. The nodes themselves may be mobile (as in the case of
fleets of vehicles). It must be relying on existing infrastructures, such as the Internet.
Grid and P2P technology raise a key challenge in terms of security. It requires a shift from the present
algebraic approach (where A and B share a secret) to a new geometric approach where a group of
subjects with dynamic links share parts of a secret. Another problem is trust (how to establish and how
to maintain it) and getting companies to change their business processes to allow each to see the
other’s information. The challenge is to make them understand that by sharing they will gain.
This new computing paradigm, grounded on open standards and sound architectural principles, has
manifold implications and the potential to pave the way to an innovative replacement of the current
omnipresent monopoly operating system. Monopolies are blocking innovation. Seminal contributions
from virtually all major computer science disciplines, developed in collaboration with scientists and
engineers, will be necessary. The final resting place of Grid and P2P as the next generation Internet,
the architecture, should be in the operating system and not a separate add-on utility.
IRG Complex Problem Solving including GRID and RNI
6
Draft V 2.0 - 14/05/02
Complex Problems – Challenges and Motivation
As bandwidth capacity becomes ubiquitous and persistent, resources for computing and storage
become available at geographically dispersed locations, creating a novel systems and application space
available for complex problem solving in science, industry, business and society. There is a wide
variety of such compute-intensive and/or data-centric problems and ways of solving them. The
requirements these problems have on the underlying computational, communication and storage
architecture can be quite different. The different types of players at all relevant levels of the value
chain that are interested in those problems or can contribute to their solution can be considered as the
potential drivers for Grid/P2P-like technologies that may become the basis for future ICT
infrastructures.
The Initial Drivers
Two sets of applications have given a prominent visibility to respectively Grid and Peer-to-Peer
applications. For the grid, this role was held by major scientific efforts, from high-energy physics,
space and astrophysics, biotechnology, or climate change, that have reached a great visibility, and
mobilise large resources. Peer-to-peer applications were made visible by grassroots efforts from the
free / open source software and open contents movements addressing data sharing, co-operative
computation, co-operative publishing, or distributed software development. During FP6, the prominent
role of these driving Grid and Peer-to-Peer applications will remain very important.
Commercial and Societal Applications
The FP6 proposal texts situates the development of the basic technology for complex problem solving
in the perspective of challenges in science, industry, business and society and of application fields
such as environment, energy, health, transport, industrial engineering, finance and new media. One
can list applications in these domains that can act as drivers for technology development and
deployment:
•
Distributed and Collaborative Product Development and Production: for instance, integrated
development and manufacturing environments; collaborative development for just-in-time
production, service and maintenance operations; multidisciplinary design and optimisation.
•
Application Services Provision: scalable networking and computing infrastructure offered as a
service for complex problem solving
•
Virtual Enterprises (scalable dynamic virtual organisation): interoperability for inter-enterprise
collaboration.
•
Educational applications: large-scale peer-to-peer educational resources and “educational
Napsters”.
•
Co-operative publication or content creation networks: Open publishing and distributed content.
•
Global information access: distributed searching for sound, image, 3D or other type of rich media.
•
Health applications: the interactive simulations for clinical operations planning and real-time
support, and the control of micro robotic operation facilities.
•
Critical infrastructures security for information and communications, water and energy supply
including electricity transportation, banking and finance and government services
•
Global Monitoring for Environment and Security (GMES): making use of highly distributed large
data sets and complex models and moving from data and map based systems towards fully
operational systems and information.
•
Intelligent Transport Systems: multi-source data fusion for prediction and decision support in multimodal traffic systems and integration with air pollution modelling.
IRG Complex Problem Solving including GRID and RNI
7
Draft V 2.0 - 14/05/02
Classification of Complex Problems
If we now try to identify various classes of complex problems that arise in these domains, in view of
identifying the potential underlying technical challenges, one can list:
•
Very large data flow driven problems
•
Large scale algorithmically complex problems
•
Large scale heterogeneous (multi-physics) models
•
Large-scale data with complex distributed analysis or interpretation
•
Synchronous distributed co-operation around one model
•
Managing co-operative production of entities by a large number of independent players
•
Searching in a very large number of distributed sources
A more detailed analysis of driving applications and taxonomy of complex problems can be found in
Annex II of this report.
IRG Complex Problem Solving including GRID and RNI
8
Draft V 2.0 - 14/05/02
Grid and Peer-to-Peer
Grid and Peer-to-Peer (P2P) computing can in the widest sense be considered a visible implementation
of the concepts of ambient intelligence and pervasive computing. It empowers the individual user and
fosters the creation of flexible dynamic virtual communities, initially in the academic and research
sector but with pending uptake possibilities in industrial and business sectors. Public seed funding will
encourage the stakeholders to engage and invest in Grid computing (technologies and applications)
and revolutionise in the long term the way individuals and communities use the computer, computing
as a utility. The Grid will really come into its own only when people learn to build “virtual
organisations”. Virtual organisations are noteworthy because of their constantly shifting landscape of
data and computer resources, and the authentication on which they rely. The aim is to go for the best,
most exciting technology based on open standards and ensuring it truly serves human needs and create
a robust Grid and P2P infrastructure at reasonable cost with high degree of functionality: evolvable,
scalable, pervasive, accountable and competitive. In the business world this means taking eCommerce
to next level by giving the customer a resilient, flexible, virtual IT infrastructure readily available on
demand from any location. The emerging commercial support is going to accelerate the process of
moving Grid and P2P technologies out of the labs and universities into the commercial mainstream.
Grid, P2P have also something in common with web services. All lay claim to defining the next
generation Internet. Web Services are enterprise applications that exchange data, share tasks, and
automate processes over the Internet, solve the problem how to collaborate across multiple platforms
and systems. Web services are driven by big companies and are mainly addressing businesses and
enterprises. Priority is on interoperability: from their perspective what they are trying to accomplish is
to get their business processes interconnected. Web services build on well-defined standards for
communication between client and server, standards such as XML, SOAP, WDSL and UDDI. Grid,
P2P and web services will become increasingly symbiotic, see the report on Open Grid Services
Architecture (OGSA).
Finally, perhaps the biggest question about P2P computing concerns the financial viability of the
emerging business models.
Where do we stand today?
The EC and individual EU countries have already funded several substantial grid-related projects in
the last two years, some for infrastructure and middleware development and some for applications that
take advantage of the Grid concept. They are all addressing different research topics and fields
spanning from High Energy Physics to bioinformatics and medical applications, yet using a de facto
US developed open middleware standard (with one notable exception). See Annex I for details.
Whilst the expectation is high, the number of organisation exploiting the technology is comparatively
low. Partly this slow take up can be explained by the fact that the technologies (GRID, P2P,
Webservices) are still a very young area and there is a lot of work being carried out developing
standards and creating practical, usable and user-friendly tools. However, the biggest inhibitor to the
uptake of these technologies is identified as a simple lack of security. Security is a major area of
concern to all business today.
Timely Public funding and private investment could lead to significant payoff.
What research is required in FP6?
The needs and problems of virtual organisations (in research, business and industry) and communities
should define the Grid roadmap and research agenda. This is where the future Grid would have its
greatest value. It is such problems that could lead to new business models—just as the Internet created
the conditions for e-commerce. Present EU funded grid projects are using in one way or the other
existing (US developed) middleware toolkits and are strongly big-science oriented with rather shortterm goals. To fully exploit the capabilities of the Grid in the sense of utility computing one should
aim at creating a truly open co-operative and competitive industrial-strength Grid-framework
(collaboration with US and Japan to avoid diverging standards) considering the above mentioned
research challenges.
IRG Complex Problem Solving including GRID and RNI
9
Draft V 2.0 - 14/05/02
•
Development of an ”open” Grid architecture (OS, middleware)
•
Development of interoperable security and directory architectures
•
Development of Grid-aware applications in research, business, industry, health, transport,
environment
Key Challenges
Since a Grid will consist of vastly heterogeneous sets of compute nodes, especially commodity
clusters, storage resources, platform portability which enables programs to be downloaded from
networks and to be executed is required. In addition, performance portability, which provides good
performance irrespective of platforms, is also needed.
To achieve this vision of the Grid as the next computing paradigm (render obsolete much of the
computing world as it is today) a set of interrelated crucial challenges need to be tackled, at the
network level, the computer level (OS, middleware, API), the application layer and the business layer.
R&D for Grid is inspired by the confluence of several technological trends, application push and
technology pull.
Challenges at the network layer:
•
Network capacity; bandwidth provision (QoS and flexible, scalable);latency issues;
•
End-to-end issues (NATs and firewalls);
•
Security and reliability issues; untrusted resources;
•
Mobility aspects;
Challenges at the OS and middleware layer:
•
Start from open software OS kernel;
•
Resource discovery, scheduling and management;
•
Accounting (virtual brokerage of computer power) and billing issues; cross-organisational policy
issues;
•
Security issues (authentication, authorisation, integrity…);
•
Directory and naming issues (LDAP, UDDI);
•
APIs and open protocols; interoperability;
•
Use of agent technologies;
•
Standards;
Challenges at the application layer:
•
Develop grid-aware applications;
•
Integrating Grids with existing applications; databases poorly matched to grids;
•
Application fusion for improved performance; application portability;
•
Simple visual user interface (hiding the underlying complexity);
•
Standards;
•
DRM;
Challenges at the business layer:
•
Grid business models;
•
Integration of Grid/P2P with Web Services (Internet accessible software components) and
databases;
•
Regulatory, legal and social issues;
•
Legacy issues;
•
Integrating P2P in the supply chain and workflow control;
IRG Complex Problem Solving including GRID and RNI
10
Draft V 2.0 - 14/05/02
Building the Constituencies
Research labs and universities already are well known actors in the field. Awareness rising is needed
in business, industry, health, environment, transport, etc.
IRG Complex Problem Solving including GRID and RNI
11
Draft V 2.0 - 14/05/02
Distributed and Shared Infrastructure for e-Learning
Background
Technology supported learning is a very broad area covering a number of different realities, depending
on the target audience (schools, universities, vocational training, life-long learners), the pedagogical
model and the socio-economical context.
Following are some of the key issues where ICT is expected to play an essential role:
•
Distance learning;
•
Collaborative learning (e.g. problem solving by teams of learners);
•
Use of virtual or mixed reality for learning in complex training environments;
•
Sharing of educational material, using metadata and raising complex IPR issues;
•
Personalised learning (learning path determined by the needs and background of the learner);
•
On-the job learning (integration of training and work);
•
Learner assessment;
Motivation for GRID in e-learning
Areas of e-learning suitable for GRID-based applications
Some areas of e-learning can only really take off if a GRID type infrastructure is available.
•
Advanced distance learning, implementing the concept of virtual classroom, where teachers and
learners are geographically dispersed and where the use of interactive multimedia objects is
required.
•
Collaborative learning, which requires advanced communication facilities (e.g. video-conferencing)
and application sharing facilities.
•
The implementation of the concept of e-campus, where universities not only share course material
but also research and computing facilities is only possible with a GRID infrastructure.
•
The use of virtual and mixed reality simulations in a number of complex training areas (e.g.
training of surgeons, training of maintenance staff of complex equipment, training of mechatronics
engineers).
Contribution of e-learning applications to GRID technology development
The GRID technology is not only an enabler of e-learning applications, it could also benefit from the
special requirements imposed by these applications on the GRID:
•
Advanced multicasting facilities;
•
High-class visualisation environments;
•
Reliability and availability of the infrastructure;
•
e-learning specific middleware (e.g. for content representation, specific IPR issues).
IRG Complex Problem Solving including GRID and RNI
12
Draft V 2.0 - 14/05/02
Research Network Infrastructures
The Framework Programme text highlights that Research infrastructures address (amongst other
issues): The support of existing research infrastructures to create a denser network between related
initiatives, in particular by establishing a high-capacity and high-speed communications network for
all researchers in Europe (GEANT) and specific high performance Grids and test-beds (GRIDs).
Background on Research Infrastructures
The two primary aims of GEANT are the provision of a high-speed state-of-the-art research network
backbone infrastructure for European researchers and the extension of this network to the EastEuropean candidate countries (associated to the Framework Programme). Both of these objectives
have been fully achieved, as 10 Gb/s connectivity in the core of the network is fully operational since
1st November 2001 and more than 30 European countries are connected to the infrastructure.
These achievements of GEANT have a significant impact on overall research in Europe. First and
foremost as the GEANT network is the foundation of ERA. Second, because Europe starts attracting
connectivity from other parts of the world and has established itself as a global player. Both are
fundamental in keeping European researchers at the leading edge. The GEANT network is now
recognised as an important global player. This is underlined by the attempts from the various sides
(e.g. Mediterranean, Asia Pacific, Latin-America) to establish a direct interconnection between these
areas and Europe.
The new paradigm of Grids is to become the next revolution in networking: spectacular advances in
both commodity computing power and network bandwidth have encouraged the belief amongst the
scientific and the business communities around the globe that it will be possible to embrace new
methods of global collaborative research and enterprise. Major projects have been launched into this
area. This strong European interest has already been recognised by partners around the globe and
collaboration has been established with similar initiatives in the US to ensure a coherent development
of this new technology.
The current Internet Protocol IPv4 (version 4) is reaching its limits mainly in terms of address space
and network management. IPv4 can provide – in theory – 4 billion addresses that have to be compared
to the current estimated world population of 6 billions people. The new version of the Internet
Protocol, called IPv6, has been designed having the experience of IPv4 and provides solutions to the
current IPv4 limitations as well as new features like plug and play auto-configuration. Considering
especially the area of mobile communications (2.5 G and 3G systems), where Europe has a leading
position, the limitations of IPv4 make the development of new applications either not possible or at
least very difficult. IPv6 has furthermore the potential to bring amongst other things end to end
security, end to end Quality of Service that are key enabler factors also for the implementation of the
GRID concept.
The 2 large pan-European infrastructure test-beds on IPv6 that started in January 2002, are seen as
precursors for the Integrated Projects in FP6 as they constitute microcosms on their own (budgets
beyond 15 M€ and more than 20 European participants each) and since research activities related to
IPv6 are still needed (e.g. improvement of the Internet routing algorithms, large scale experimentation
about security or Quality of Service). These test-beds will prepare the ground and provide the
necessary know-how for the deployment of IPv6 in Europe.
Work on Research Infrastructures in FP6 (GEANT and GRIDs)2
The following research activities need to be carried out in order to foster and enhance Europe’s
position in the integrated area of GEANT and specific high performance Grids and test-beds:
•
Exploit the opportunities of a liberalised telecommunication environment (“own” fibre, “own” the
network, etc.). Thus moving into a completely new and yet unexplored area of research
2
The Council of Minister proposed on its meeting on 10th December 2001 to allocate 300 M€ for the further
development of GEANT and GRIDs.
IRG Complex Problem Solving including GRID and RNI
13
Draft V 2.0 - 14/05/02
networking in Europe. Examples from Canada and US show that through this a paradigm shift
might occur in the way networking services can be provided.
•
Support and integrate research projects on top of the network infrastructure.
•
The current growth in demand for bandwidth clearly shows the need to address the range of
terabit/s communication capacities. However, it still needs to be demonstrated that these speeds
scan be effectively managed and deployed in a production-class network.
•
Enhance the “inclusiveness” of the research network infrastructure by taking into account the
demands of various actors in the field (e.g. schools, educational networks, libraries, e-learning,
etc.). This activity will widen the scope of the European research network backbone substantially
and will have a profound impact on its structure.
•
Strengthen Europe’s position as a global player in the filed of networking by maintaining and
extending the current international connectivity to the National Research and Education Networks
(NRENs) in regions outside Europe (e.g. Mediterranean, Far-East and Pacific-Rim, Africa, SouthAmerica). Thus using this research infrastructure as a political instrument to support the
development and foster cohesion in these countries.
•
The future European research backbone network will need to go beyond the “simple” provision of
“bit-pipes”. This infrastructure will have to provide advanced middleware services and GRID like
services for the research community to facilitate their ever-growing needs.
•
Build advanced test-beds to test, validate integrate and demonstrate new technologies and
services (like Grids, GMPLS, new routing schemes, access technologies, photonics) in real-world
settings.
In particular in relation to Grids:
•
Carry out advanced Grid experiments (on production level) to pave the way for the creation of a
new class of infrastructure to enable next generation research and business over the internet
based on global collaborations and on the sharing of information resources. As important step
here is seen the creation of distributed tera-scale facilities (in terms of computing, storage,
communication power) across Europe for the generic use across different research disciplines.
Use advanced communication facilities (e.g. GEANT, broad scale utilisation of IPv6) to
interconnect those facilities with an aim to lead to the creation of a virtual GRID-based research
infrastructure across all Europe.
•
Foster interoperability of solutions across many different disciplines in an effort to achieve broad
scale uptake of Grid technology across numerous application areas (and user communities) and
economies of scale; in the same context contribute to the creation of new standards; continue the
strong contribution to open-source.
•
Put particular emphasis on access facilities to the Grid infrastructure (e.g. wireless access); if
Grids is to become a utility, a broad range of access devices need to be able to be connected to
Grid infrastructures and user friendly interfaces are necessary to be in place.
•
Interconnect major Grid-based testbeds/experiments in Europe with corresponding Grid
testbeds/experiments in the world, e.g. US, Asia-Pacific region and others.
•
Enable the smooth evolution of the current ICT-environments to pervasive computing ones based
on flexible and co-ordinated schemes for the control and sharing of resources.
IRG Complex Problem Solving including GRID and RNI
14
Draft V 2.0 - 14/05/02
Global Monitoring for Environment and Security
Background
The initiative on Global Monitoring for the Environment and Security (GMES) is aimed at the
provision of independent operational information for monitoring and management of the environment
and security, in fulfilment of European Policies.
“GMES aims to support Europe’s goal regarding sustainable development, global governance, by
facilitating and fostering over the next decade the provision of enhanced quality data, information and
knowledge. It will do so by paying particular attention to a better use of information technologies”.
The aim is to achieve a significant leap forward in the quality of information and services
delivered. GMES’s goal is to provide information in a transparent and user-friendly manner, allowing
access to high quality services. The concept of an open information architecture will provide means
for a more democratic participation in policy-making.
A key emphasis is put on the integration of data and information gathered from space and from
terrestrial sensor networks. Data acquisition from Earth observation will provide a critical
component in building the type of information services which are needed. A transition from
experimental, research type systems towards fully operational systems and services is needed.
GMES is about exploiting Europe’s industrial and technological competencies in information and
communication technologies to respond to new challenges concerning sustainable development,
crisis management, peace keeping, operations and humanitarian and development aid.
GMES priority areas
•
European Regional Monitoring


•
Global Monitoring



•
C. Global vegetation monitoring
D. Global Ocean monitoring
E. Global Atmosphere monitoring
Security related Aspects



•
A. Land cover change in Europe
B. Environmental Stress in Europe
F. Support to regional development aid
G. Systems for risk management
H. System for crisis management and humanitarian aid
Horizontal Support Action

I. Information management tools and contribution to the development of a European spatial data “
Infostructure”
Motivation and suitability for GRID related RTD on environmental applications (GMES)
Most of the thematic priority areas in GMES require a combination of information and computing
GRIDs, which is due to the complexity of the problems to be addressed, the large size of distributed
data sets and the huge variety of scientific complex models to be deployed.
Example 1: Global Climate change prediction requires the use and integration of different
atmospheric, chemical, meteorological models which are to be fed by earth observation data from
satellites, data from marine boys, sea temperature, etc. including terra-bits of historical data from large
distributed data sets.
Example 2: Risk modelling and management for flood prediction or forest fire management: Such
information systems and services require the integration of mete-data, earth observation data with
digital terrain data, flood or fire propagation models and appropriate decision support tools.
IRG Complex Problem Solving including GRID and RNI
15
Draft V 2.0 - 14/05/02
Annex I – GRID Activities
Summary publicly funded R&D Grid Projects
Country
Description
EU funded
EuroGrid: Application Testbed for European GRID computing. The EUROGRID project will
demonstrate the use of GRIDs in selected scientific and industrial communities, address the
specific requirements of these communities, and highlight the benefits of using GRIDs.
www.eurogrid.org
DataGrid is a project funded by European Union. The objective is to enable next generation
scientific exploration which requires intensive computation and analysis of shared large-scale
databases, from hundreds of TeraBytes to PetaBytes, across widely distributed scientific
communities. http://www.eu-datagrid.org/
Damien: Distributed Applications and Middleware for Industrial use of European Networks.
The project is coming from the are of High Performance Computing and deals with the problem
of how to develop applications for computational grids. In High Performance Computing the
application developer is used to some standards, like the Message Passing Standard MPI, and
several tools, which ease the development and analysis of his application. The central objective
of the project is therefore to provide these well-known and highly accepted tools to
Computational Grids, too. To reach this goal, the tools have to be extended to the properties of
Grids. Additionally, these tools will be tested in industrial environments with application used in
every day production. http://www.hlrs.de/organization/pds/projects/damien/
DataTAG project will create a large-scale intercontinental Grid testbed that will focus upon
advanced networking issues and interoperability between these intercontinental Grid domains,
hence extending the capabilities of each and enhancing the worldwide programme of Grid
development. The project will address the issues which arise in the sector of high performance
inter-Grid networking, including sustained and reliable high performance data replication, endto-end advanced network services, and novel monitoring techniques. The project will also
directly address the issues which arise in the sector of interoperability between the Grid
middleware layers such as information and security services. http://datatag.web.cern.ch/datatag/
GRIA project will develop, apply and evaluate a Grid testbed, based on an existing open-source
infrastructure but incorporating services for: end-to-end quality of service to provide reliable and
manageable performance; support for secure, end-to-end business models and processes,
enabling the Grid to be used for outsourcing computational services. The testbed will employ
open standard interfaces to access Grid resources, and a toolkit will be included to make it easy
to develop services and applications for the Grid. The results will be evaluated for two industrial
applications (in structural analysis and in digital film post-production), and disseminated to
promote further standardisation and take-up by European industry. IST-2001-33240.
Grip will develop interoperability software for EUROGRID and Globus and demonstrate the
feasibility to exploit the specific strength of each implementation, the seamless User interface of
EUROGRID and the protocol focus of Globus. Biomolecular and meteorological applications
will be used as inter-grid examples in GRIP. Project resources will be dedicated to standards
work in the Global Grid Forum.
GridLab: A Grid Application Toolkit and Testbed. The GridLab project will develop a easy-touse, flexible, generic and modular Grid Application Toolkit (GAT), enabling todays applications
to make innovative use of global computing resources. The project is grounded by two
principles, (i) the co-development of infrastructure with real applications and user communities,
leading to working scenarios, and (ii) dynamic use of grids, with self-aware simulations adapting
to their changing environment. www.gridlab.org
IRG Complex Problem Solving including GRID and RNI
16
Draft V 2.0 - 14/05/02
EU funded
UK
eScience
ESGO will create a virtual archive by federating solar data centres scattered across Europe into a
data Grid. This will dramatically enhance access to data for both the solar and non-solar
communities and will for the first time make data available on demand to the user. The project
will employ intuitive web-like interfaces and advanced search and visualisation tools to match
observations made by numerous space- and ground-based observatories. A major innovation will
be the provision of a Solar Feature Catalogue that will allow complex searches based on solar
phenomena and events. The EGSO will also provide the facility for the evaluation of extended
time series of large, complex data sets at source, without the need to download them. This Grid
test bed will federate six data centres located in France, Italy and the United Kingdom, but the
design of the EGSO will scale to include as many data archives world-wide as wish to
participate. http://www.mssl.ucl.ac.uk/grid/egso/
Cross Grid project will develop, implement and exploit new Grid components for interactive
compute and data intensive applications like simulation and visualisation for surgical procedures,
flooding crisis team decision support systems, distributed data analysis in high-energy physics,
air pollution combined with weather forecasting. The elaborated methodology, generic
application architecture, programming environment, and new Grid services will be validated and
tested thoroughly on the CrossGrid testbed, with an emphasis on a user friendly environment.
The work will be done in close collaboration with the Grid Forum and the DataGrid project to
profit from their results and experience, and to obtain full interoperability. This will result in the
further extension of the Grid across eleven European countries. http://www.crossgrid.org
GridStart: The GRID is widely seen as a step beyond the Internet, incorporating pervasive high
bandwidth, high-speed computing, intelligent sensors and large-scale databases into a seamless
pool of managed and brokered resources, available to industry, scientists and the man in the
street. The potential benefits and social impact of the GRID are so great, that it is imperative to
involve industry and the service-provision community at an early stage to ensure that the
European economy and society can take full advantage of this revolution. The objective of the
GRIDSTART Accompanying Measure is to maximise the impact of EU-funded Grid and related
activities through the clustering of the currently funded projects and thereby enhance the
potential of the new Grid technologies to benefit the people of the European Union.
eScience Initive:
•
£120M 3 Year Programme to create the next generation IT infrastructure to support eScience and Business
• £75M is for Grid Applications in all areas of science and engineering
• £10M for Supercomputer upgrade
• £35M for development of ‘industrial strength’ Grid middleware
• Require £20M ‘matching’ funds from industry
Essential that UK plays a leading role in Global Grid development with the USA, EU and Asia
Grand Challenge Projects:
•
•
•
•
•
Equator: Technological innovation in physical and digital life
AKT: Advanced Knowledge Technologies
DIRC: Dependability of Computer-Based Systems. http://www.dirc.org.uk
MIAS: From Medical Images and Signals to Clinical Information
AstroGrid: links to EU AVO and US NVO projects
eScience projects
•
•
•
•
•
•
Comb-e-Chem:Structure-Property Mapping Southampton, Bristol, Roche, Pfizer, IBM
DAME: Distributed Aircraft Maintenance Environment York, Oxford, Sheffield, Leeds,
Rolls Royce
Reality Grid: A Tool for Investigating Condensed Matter and Materials QMW, Manchester,
Edinburgh, IC, Loughborough, Oxford, Schlumberger
My Grid: Personalised Extensible Environments for Data Intensive in silico Experiments in
Biology: Manchester, EBI, Southampton, Nottingham, Newcastle, Sheffield, GSK, AstraZeneca, IBM, Sun
GEODISE: Grid Enabled Optimisation and Design Search for Engineering: Southampton,
Oxford, Manchester, BAE, Rolls Royce
Discovery Net: High Throughput Sensing Applications, Imperial College, Infosense,
Grid Support Centre. http://www.grid-support.ac.uk/
IRG Complex Problem Solving including GRID and RNI
17
Draft V 2.0 - 14/05/02
NL
GridPP will deliver the Grid software (middleware) and hardware infrastructure to enable
testing of a prototype of the Grid for the Large Hadron Collider (LHC) project at CERN of
significant scale. The GridPP project is designed to integrate with the existing Particle Physics
programme within the UK, thus enabling early deployment and full testing of Grid technology
and efficient use of limited resources. The project will disseminate GridPP deliverables in the
multi-disciplinary e-Science environment and will seek to build collaborations with emerging
Grid activities both nationally and internationally. http://www.gridpp.ac.uk/
DutchGrid: a national Grid initiative in the Netherlands as a local testbed for the European
Datagrid.
DE
VLAM-G: The Grid-based Virtual Laboratory AMsterdam (VLAM-G), provides a science
portal for distributed analysis in applied scientific research. It offers scientists remote experiment
control, data management facilities and access to distributed resources by providing crossinstitutional integration of information and resources in a familiar environment. The main goal is
to provide a unique integration of existing standards and software packages.
UNICORE (UNiform Interface to COmputing REsources) provides a science and engineering
GRID combining resources of supercomputer centers and making them available through the
Internet. Strong authentication is performed in a consistent and transparent manner, and the
differences between platforms are hidden from the user thus creating a seamless HPC portal for
accessing supercomputers, compiling and running applications, and transferring input/output
data. http://www.unicore.de/
FR
Grid funding approved by French Government.
JACO3 is a Grid environment that supports the execution of coupled numerical simulation. It is
being developed jointly by EADS, INRIA, INTECS and KTH. It is a set of CORBA services that
manage remote computing resources through the Internet.The PARIS research group at INRIARennes is conducting research activities in Grid computing. A Computational Grid acts as a
high-performance virtual computer to users to perform various applications such as for scientific
computing or for data management. It is made of several computing resources interconnected
together by various networks. Although Grid Computing is still an emerging academic research
field, it is foresee that the industry will soon make a stronger demand on Grid environments.
Indeed, today in the industry the trend is to replace prototyping as much as possible by
simulation to reduce costs and time to market.
http://www.ercim.org/publication/Ercim_News/enw45/priol.html
IT
Décrypthon est une nouvelle initiative de l'AFM et d'IBM, qui permet de rassembler des milliers
d'ordinateurs personnels pour participer activement à la recherche contre les maladies génétiques
et les maladies rares. Un logiciel réalise des calculs complexes de comparaison de protéines,
répartis sur des milliers d'ordinateurs personnels à travers le réseau Internet. Dans cet objectif,
l’AFM rassemblera, une nouvelle fois, le grand public autour d’un projet scientifique
exceptionnel
pour
le
Téléthon
2001,
les
7
et
8
décembre.
http://www.telethon.fr/recherche/recherche1.asp
There is increasing interest in Grid Computing in Italy. This is reflected by the growing number
of projects in this area. Projects are underway in two major Italian research institutions: the
Italian National Research Council (CNR) and the Italian Institute for Nuclear Physics (INFN).
CNR is also involved in the DataGrid project.
Grid for Remote Sensing: The Grid Resource Broker (GRB) is one of the current Globus
projects of the HPC Lab of the University of Lecce, Italy). GRB is a grid portal that allows
trusted users to create and handle computational grids on the fly exploiting a simple and friendly
gui. We introduce GRB features and discuss its implementation status.
http://sara.unile.it/grb/grb.html
INFN Grid – will develop a Grid Infrastructure that will allow INFN users a transparent and
effective use of computing and storage resources distributed in 26 INFN nodes of the Italian
research network GARR-B. INFN is also involved in the DataGrid Project.
http://server11.infn.it/fabric-grid/
IRG Complex Problem Solving including GRID and RNI
18
Draft V 2.0 - 14/05/02
Nordic
Countries
Hungary
Ireland
NorduGrid: the Nordic Testbed for Wide Area Computing project is a part of the Nordunet2
programme, aimed to develop networked applications with extensive usage of modern utilities
and tools in the Nordic countries. The aim is to establish an inter-Nordic test bed facility for
implementation of wide area computing and data handling. The facility will provide the
infrastructure for interdisciplinary feasibility studies of GRID-like computer structures. The
project shall collect and document experience, as an input to the decision process on the future
computer infrastructure strategy for sciences with distributed PByte storage requirements and
processing power in the order of multi-teraflop.
http://www.nordugrid.org
The Laboratory of Parallel and Distributed Systems (LPDS) of SZTAKI has a long and
successful history in the research and development of distributed systems, tools and applications.
A number of these activities and results are closely related to the current grid-oriented projects of
the laboratory. The largest computer cluster in Hungary runs under the supervision of LPDS.
SZTAKI participates in the EU DataGrid project.
http://www.ercim.org/publication/Ercim_News/enw45/kacsuk.html
Grid-ireland: Initial seed funding has been provided by Enterprise Ireland to guarantee the
establishment of a working grid between compute clusters at three partner sites in Ireland, that is,
the Departments of Computer Science in Trinity College Dublin, University College Cork and
NUI Galway. This project has one primary objective: to establish grid-ireland. This requires:
•
Poland
Hardware: Compaq will donate a 4-way symmetric multiprocessing gateway machine per
site
• Software: initially the Globus services
• Management: the Dept. of Computer Science at Trinity College will do this
• Interconnect: the Irish academic network services will be used.
http://www.ercim.org/publication/Ercim_News/enw45/coghlan.html
Poznan Supercomputing and Networking Center works on development of tools and methods for
metacomputing. The tools include:
•
•
•
MetaLearn - knowledge acquisition and rules’ generator system (knowledge about the state
of the metacomputer, application behavior).
MPI tools: MPIVIS, Visual MPI - integrated, knowledge based environment for developing
MPI programs,
PERVIS - knowledge based performance tunning and visualisation of metacomputing
applications. The project is done within co-operation with APART working group.
•
US
KB Metascheduler - knowledge based scheduling of the jobs in a metacomputing (Globus)
environment.
• All the PSNC metacomputing projects are being developed within the national
metacomputer infrastructure, basing on POL-34 network
http://www.man.poznan.pl/metacomputing/
Access Grid: Create and deploy persistent collaborative spaces on the Internet for group
collaboration using commodity technologies. The advanced video-conferencing technology of
the Access Grid, which was developed by the Argonne National Laboratory Futures Lab, enables
wide-area collaborative computational science, large-scale distributed meetings and collaborative
work sessions, and training. Each Access Grid node offers multimedia displays, presentations
and interactions environments, and interfaces to Grid middleware and visualisation
environments. Funding agencies DOE, NSF. www.mcs.anl.gov/FL/
ASCI DISCOM: Create operational Grid providing access to resources at three U.S. DOE
weapons laboratories. Funding agency: DOE; www.cs.sandia.gov/discom
Particle Physics Data Grid (PPDG): The Particle Physics Data Grid collaboration was formed
in 1999 because its members were keenly aware of the need for Data Grid services to enable the
worldwide distributed computing model of current and future high-energy and nuclear physics
experiments. Initially funded from the NGI initiative and later from the DOE MICS and HENP
programs, it has provided an opportunity for early development of the Data Grid architecture as
well as evaluating some prototype Grid middleware. Funding agency: DOE.
http://www.ppdg.net/
Earth Systems Grid: Funding agency: DOE. Turning Climate Model Datasets Into Community
Resources. Delivery and analysis of large climate model datasets for the climate research
community. http://www.earthsystemgrid.org/
IRG Complex Problem Solving including GRID and RNI
19
Draft V 2.0 - 14/05/02
Science Grid: The DOE Science Grid's major objective is to provide the advanced distributed
computing infrastructure based on Grid middleware and tools to enable the degree of scalability
in scientific computing necessary for DOE to accomplish its missions in science. The vision for
"Grids" is to revolutionise the use of computing in science by making the construction and use of
large-scale systems of diverse resources as easy as using today's desktop environments. Funding
agency: DOE. www.sciencegrid.org
Fusion Collaboratory: Create a national computational collaboratory for fusion research.
Funding agency: DOE. www.fusiongrid.org
ß-Grid: A National Infrastructure
http://www.globus.org/beta/
for
Computer
Systems
Research
(ANL/NCSA).
Information Power Grid (IPG): The IPG is NASA's high performance computational grid.
Computational grids are are persistent networked environments that integrate geographically
distributed supercomputers, large databases, and high-end instruments.
These resources are managed by diverse organizations in widespread locations, and shared by
researchers from many different institutions. Funding agency: NASA. www.ipg.nasa.gov
National Technology Grid: Funding agency: NSF.
Network for Earthquake Engineering Simulation (NEES): NEESgrid is an integrated
network that will link earthquake engineering research sites across the country, provide data
storage facilities and repositories, and offer remote access to the latest research tools. It was
established in 2001 by a consortium of institutions led by the National Center for
Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign.
Funding agency: NSF. www.neesgrid.org
Grid Application Development Software: Funding agency: NSF.
Grid Physics Network (GriPhyN): Technology R&D for data analysis in physics experiments:
ATLAS, CMS, LIGO, SDSS. Funding agencies: DOE and NSF. www.griphyn.org
TeraGrid: is a multi-year effort to build and deploy the world's largest, fastest, most
comprehensive, distributed infrastructure for open scientific research. When completed, the
TeraGrid will include 13.6 teraflops of Linux Cluster computing power distributed at the four
TeraGrid sites, facilities capable of managing and storing more than 450 terabytes of data, highresolution visualization environments, and toolkits for grid computing. These components will
be tightly integrated and connected through a network that will initially operate at 40 gigabits per
second and later be upgraded to 50-80 gigabits/second—16 times faster than today's fastest
research network. Funding agency: NSF. www.teragrid.org
Distributed Tera-Scale Facility and Extensible Tera Scale Facility part of the NSF
Cyberinfrastructure initiative.
International Virtual Data Grid Laboratory (iVDGL): The iVDGL will provide a global
computing resource for several leading international experiments in physics and astronomy,
including the Laser Interferometer Gravitational-wave Observatory (LIGO), the ATLAS and
CMS experiments at CERN, the Sloan Digital Sky Survey (SDSS), and the proposed National
Virtual Observatory (NVO). Funding agency: NSF. www.ivdgl.org
The Brain Data Grid: National Scale Testbed for Federating Large Databases Using NIH High
Field NMR Centers; Cyber Infrastructure Linking Tele-instrumentation, Data Intensive
Computing, and Multi-scale Brain Databases.
Grid Application Development Software (GrADS): The goal of the Project is to simplify
distributed heterogeneous computing in the same way that the World Wide Web simplified
information sharing over the Internet. The GrADS project will explore the scientific and
technical problems that must be solved to make grid application development and performance
tuning for real applications an everyday practice. www.hipersoft.rice.edu/grads
IRG Complex Problem Solving including GRID and RNI
20
Draft V 2.0 - 14/05/02
JP and
Asia
Pacific
Scalable Intracampus Research Grid (SInRG) The project will deploy a research
infrastructure on the University of Tennessee, Knoxville campus that mirrors the underlying
technologies and the interdisciplinary research collaborations that are characteristic of the
emerging national technology grid. SInRG's primary purpose is to provide a technological and
organizational microcosm in which key research challenges underlying grid-based computing
can be attacked with better communication and control than wide-area environments usually
permit. The project is supported by a grant from the National Science Foundation.
http://icl.cs.utk.edu/sinrg/
FIVE of Japan's national universities have launched a grid computing project based on MEM
(micro-electronic mirror) technology that will eventually become Japan's national computing
grid, according to Dr. Ken Miura, a researcher with Fujitsu's computer systems group and a
speaker here at the Global Grid Forum.
Called Super-SINet (science information network), Japan's grid currently links five of the
country's top universities in Osaka, Tohoku, Tokyo, Nagoya, and Kyoto, Miura said.
"This is a national project that provides a 10Gbps photonic backbone network," Miura said. "The
backbone network branches from the nodes and bridges with optical cross-connects called
MEMs, [which is] optical-to-optical switching without converting the signal to an electronic
signal by flipping very tiny mirrors at very high speed."
http://www.idg.net/ic_817739_1794_9-10000.html
ApGrid is a partnership for Grid computing in the Asia Pacific region. ApGrid focuses on (1)
sharing resources (2) developing Grid technologies (3) helping the use of our technologies in
create new applications (4) building on each other work, etc., and ApGrid is not restricted to just
a few developed countries, neither to a specific network nor its related group of researchers.
http://www.apgrid.org/
Grid Data Farm is a Petascale data-intensive computing project initiated in Japan. The project
is collaboration among KEK (High Energy Accelerator Research Organization), ETL/TACC
(Electrotechnical Laboratory / Tsukuba Advanced Computing Center), the University of Tokyo,
and Tokyo Institute of Technology (Titech). The challenge will involve construction of a data
processing framework that will handle hundreds of Terabyte to Petabyte scale data emanated by
the ATLAS experiment of LHC. Both KEK and the Univ. of Tokyo will collaborate for building
a Tier-1 regional center in Japan.
http://phase.hpcc.jp/people/tatebe/research/gfarm/abs/gfarm-ETL-TR2001-4
Canada
India
THAIGRID Thailand High-performance Advanced Infrastructure Grid Toward a Unify
Computing Infrastructure for Thailand. Building a grid infrastructure based on Globus
technology and some local technology and software tool in order to:
• Facilitate the sharing of resources among major supercomputing sites in Thailand.
• Explore and use the emerging grid computing technology.
• Build a test bed system for the development of grid base application, software tools, and
algorithm.
• Stimulate a collaboration project among high performance computing research group in
Thailand.
http://www.gridcomputing.com/
Grid Canada is a partnership between CANARIE, the National Research Council, and C3.ca. We
want to grid-enable compute, storage, and other resources across Canada and make them
available to research communities. We want to help these research communities deploy
applications that can use the grid. We want to build a Canadian Grid where researchers can
access resources in an integrated and secure way.
http://www.gridcanada.ca/
India's state-run agency for advanced computing plans to build a nationwide grid of
supercomputers for mammoth applications. Such a grid would share or combine diverse
computer memories and software in parallel processes to aid environmental modelling, fast
analysis of satellite images, advanced chip design and simulation of heavy-duty equipment like
turbines.
http://www.forbes.com/work/managementtrends/newswire/2002/03/05/rtr530735.html
http://www.cdacindia.com/
IRG Complex Problem Solving including GRID and RNI
21
Draft V 2.0 - 14/05/02
AU
GRIDSIM: The primary objective of this project is to investigate effective resource allocation
techniques based on computational economy through simulation. We like to simulate millions of
resources and thousands of users with varied requirements and study scalability of systems,
algorithms, efficiency of resource allocation policies and satisfaction of users. We are also
interested to explore how significantly the local economy and the global positioning (e.g., the
time zone) of a particular resource play role in securing jobs under various pricing and
demand/supply situations. http://www.gridcomputing.com/
IRG Complex Problem Solving including GRID and RNI
22
Draft V 2.0 - 14/05/02
Summary of GRID / P2P Activities by IT Companies
Company
IBM
Description
eLiza: Project eLiza addresses a looming crisis in the business world -- in five years, given
the rapid proliferation of technology and the growing shortage of trained systems
administrators, the majority of corporations will not be able to manage their systems using
the current computing technologies. Project eLiza's goal is to give businesses the ability to
manage systems and technology infrastructures that are hundreds of times more complex
than those in existence today.
http://www-1.ibm.com/servers/eserver/introducing/eliza/qanda.html
BlueGrid: Grid testbed linking IBM laboratories
Autonomic computing is an approach to self-managed computing systems with a minimum
of human interference. The term derives from the body's autonomic nervous system, which
controls key functions without conscious awareness or involvement. Autonomic computing
is an emerging area of study and a Grand Challenge for the entire I/T community to address
in earnest. Most immediately, the automated management of computing systems. But that
capability will provide the basis for much more: from seamless e-sourcing and grid
computing to dynamic e-business and the ability to translate business decisions that managers
make to the I/T processes and policies that make those decisions a reality. Ultimately,
autonomic computing is a challenge that must be met before the industry can deliver 'the next
big thing.' http://www.research.ibm.com/autonomic/overview/faqs.html
SUN
Microsoft
HP
IBM's Grid Innovation Centre (Montpellier, France) is a place where users can explore a
variety of grid technologies as well as learn about the potential commercial benefits of it.
Avaki's grid software will be available to IBM's European users and partners for
demonstrations, prototyping of applications, and evaluations, a company spokesman said.
"Based on open standards including OGSA, the Grid Innovation Center is made up of Linux
clusters running the latest IBM eServer xSeries and eServer pSeries, including the eServer
P690 previously codenamed Regatta IBM's 'Shark' storage, and IBM's DB2, Websphere and
Tivoli software. These are all linked to the Internet with Grid software from companies such
as Avaki, Platform Computing, Nice and Science Computing, and open Grid technologies
from Globus and Unicore..." Avaki, Platform To Be Core Technologies At IBM Center
Sun Microsystems is strongly committed to Grid Computing, expressed in its visions about
"The Network is the Computer", and "Sun ONE Open Network Environment", or in Sun’s
contribution to open technologies like Grid engine, Java, Jini, and Jxta. The objective of
Sun’s cross-functional Grid Computing organisation is to establish Sun as the leader in Grid
Computing by helping to build the Grid Community that will foster innovation, engagement,
investment and awareness at all levels, and by delivering the products and services required
by Sun’s Grid Computing partners and customers. http://www.sun.com/2001-1113/feature/
Microsoft is funding Argonne National Laboratory (Globus home) to exploit advanced
Windows features. Microsoft UK is participating tin the GEODISE project, to set up UK
Grids for scientific computing and to co-operate with international Grid programmes. See
also:.NET web service project.
http://www.microsoft.com/net/default.asp
Planetary computing: They call their model planetary computing. The goal: infrastructure
on demand. Infrastructure that's scalable, flexible, economical and always on.
http://www.hpl.hp.com/news/2001/oct-dec/planetary.html
Cooltown: HP vision of a technology future where people, places, and things are first class
citizens of the connected world, wired and wireless - a place where e-services meet the
physical world, where humans are mobile, devices and services are federated and contextaware, and everything has a web presence. The cooltown vision of a responsive world of
mobile services requires clear, creative thinking about technology. For several years, HP
Labs has been working at the intersection of nomadicity, appliances, networking, and the
web. Our model for this research is one of open collaboration and partnership with others
who share similar goals.
http://www.cooltown.hp.com/mpulse/backissues/0601/0601-cooltown.asp
IRG Complex Problem Solving including GRID and RNI
23
Draft V 2.0 - 14/05/02
Entropia
Intel
Platform
Computing
Compaq
Avaki
Entropia's enterprise distributed computing technology harnesses the vast untapped
processing power of PCs on corporate networks to perform that enterprise's computationally
intensive work. Entropia's highly scalable technology delivers computing capacities on par
with that of clusters and supercomputers at a fraction of the cost. Companies deploy
Entropia's solution to reduce their time to market by leveraging the latent computing power
in PCs they already own, dramatically increasing the return on investment in their desktop
computing and network assets. Commercial applications include critical pharmaceutical,
chemical and materials research, and financial services. http://www.entropia.com/
Intel P2P initiative: peer-to-peer computing is the sharing of computer resources and services
by direct exchange between systems. These resources and services include, but not limited to
the exchange of information, processing cycles, cache storage, and disk storage for files.
Peer-to-peer computing takes advantage of existing desktop computing power and
networking connectivity, allowing economical clients to leverage their collective power to
benefit the entire enterprise. In early 2002 Intel will release its Peer-to-Peer Accelerator Kit
for Microsoft* .NET* with reusable infrastructure middleware, demos, documentation and
reference source code for peer-to-peer application prototyping and development on the .NET.
http://cedar.intel.com/cgi-bin/ids.dll/topic.jsp?catCode=BYM
Platform Computing Inc., the world’s leading distributed computing software provider, and
the Globus Project™, a research and development initiative focused on enabling the adoption
of Grid computing, announced (November 7, 2001) that they will collaborate to provide a
commercially supported version of the Globus Toolkit™.
http://www.platform.com/
Compaq Computer recently launched its own worldwide Grid Computing Solutions Program
to provide customers with Grid software, computer systems, and custom installation and
support services to enable users to share computing, storage, data, software and other
resources. Through an alliance with Platform Computing, one of the oldest companies in the
Grid space, Compaq plans to utilise Platform's commercialised Globus Toolkit and
Platform's Grid Suite to provide customers with a complete, integrated open Grid solution to
take advantage of Compaq's Tru64 UNIX Alphaserver systems and Compaq ProLiant servers
running Linux. Compaq plans to sell, install and support Platform's Grid Suite when it
becomes available. To test and demonstrate these solutions, Compaq has established a globespanning internal Grid within its corporate firewall, connecting high-performance computing
sites in Marlboro, Mass., Nashua, N.H., Annecy, France, Galway, Ireland, and Tokyo.
Compaq has also established an Advanced Center for the Study of Grid Computing in
Nashua. http://www.compaq.com/newsroom/pr/2001/pr2001111403.html
Providing wide-area access to critical resources: Processing Power, Data, Applications.
Companies in life sciences, manufacturing, and financial services sectors are taking
advantage of AVAKI grid software to streamline and accelerate compute-intensive and dataintensive work so they can bring higher-quality products to market faster and at lower cost.
IBM, Avaki team to advance grids : Trying to hasten the acceptance of its Grid Computing
initiative outside the walls of academia and among commercial users, IBM on April 23, 2002
announced it is working with global grid pioneer Avaki. As part of the alliance, Avaki will
make its Avaki 2.1 software, a commercial package that integrates both data grid and
compute grid functions, available to IBM as part of Big Blue's recently announced Grid
Innovation Centre in France.
IBM's Grid Innovation Centre is a place where users can explore a variety of grid
technologies as well as learn about the potential commercial benefits of it. Avaki's grid
software will be available to IBM's European users and partners for demonstrations,
prototyping of applications, and evaluations, a company spokesman said.
http://www.avaki.com/news/release20020423.html
McAfee
McAfee unveils security GRID (16th April 2002). The initiative, dubbed “Grid Security
Services” will use distributed computing techniques to provide real-time dynamic security for
McAfee customers. Grid Security Services, which will be free to present customers, will
allow McAfee to offer security measures more directly in tune with the threats they face.
http://staging.infoworld.com/
IRG Complex Problem Solving including GRID and RNI
24
Draft V 2.0 - 14/05/02
Fujitsu
Fujitsu Ltd. will launch a new consulting service in June, aimed at universities and research
institutes wanting to hop on the grid-computing bandwagon. Under the umbrella of "Grid
Solution for the Sciences," Fujitsu will offer a range of services that includes construction of
a grid-computing environment and training and operational support of the system and
network, the Tokyo company said 23rd April 2002. The initial focus will be on Japan but the
company does not rule out working overseas. The company is already participating in
Unicore, a part of the European EuroGrid research project, and is involved in the Global
Grid Forum in addition to its work in Japan, where it is involved with several university and
public sector grid-computing trials. http://www.itworld.com/Net/4235/020423fujitsugrid/
Working with its research arm, Fujitsu Laboratories Ltd., Fujitsu said it has been a pioneer in
Grid development. Fujitsu supplied Grid middleware to the IT-based Laboratory (ITBL)
project, which is part of the e-Japan initiative, and, as part of the SuperSINET project, ported
the Globus Toolkit Grid middleware to its VPP Series supercomputers installed at the
Universities of Kyushu, Kyoto, and Nagoya. In addition, Fujitsu is conducting research on
Globus Toolkit portability and other Grid-related technologies in conjunction with the
National Institute of Advanced Industrial Science and Technology (AIST), an independent,
public sector institution.
http://www.internetnews.com/infra/article/0,,10693_1022781,00.html
SGI
SGI has partnered with IONA to incorporate IONA’s Orbix E2A Webservices Integration
platform to SGIs array of advanced high-performance computing systems. Orbix E2A
leverages Webservices standards and service-oriented architectures to provide an open lowcost integration solution. This integration will allow customers to more easily capture and
utilise business logic and application components stored in disparate systems across their
enterprise. http://www.sgi.com; http://www.internetnews.com
GridSystems
Research carried out by the GridSystems Research and Development Department (R+D)
forms the basis for the developments and products offered by the Company. At the present
time, research is focused on the development of a new technology known as Innergrid
Technology which is adaptable to different markets and enables the integration of high
performance computation in modern business environments.
Spanish Science and Technology Ministry funds several of GridSystems Grid projects. Since
2000, the Spanish Science and Technology Ministry, with the PROFIT program ("Programa
de Fomento de la Investigacion Tecnica"), has given Spanish company GridSystems
subsidies for more than 1 Million Euro to study the applications of Grid technology in
several projects that have more than 2 Million Euro as a total budget.
http://www.gridsystems.com/
IRG Complex Problem Solving including GRID and RNI
25
Draft V 2.0 - 14/05/02
ANNEX II – Complex Problems
Application Areas
As bandwidth capacity becomes ubiquitous and persistent, resources for computing and storage
become available at geographically dispersed locations, creating a novel systems and application space
available for complex problem solving in science, industry, business and society. There is a wide
variety of such compute-intensive and/or data-centric problems and ways of solving them. The
requirements these problems have on the underlying computational, communication and storage
architecture can be quite different. The different types of players at all relevant levels of the value
chain that are interested in those problems or can contribute to their solution can be considered as the
potential drivers for Grid/P2P-like technologies that may become the basis for future ICT
infrastructures.
In a first step, a group of typical application areas with problems to be solved that can be considered as
“complex” in the sense of the IRG are analysed. In a second step, it is tried to categorise these
problems into classes and tentatively map them to underlying architectures.
The Initial Drivers
Two sets of applications have given a prominent visibility to respectively Grid and Peer-to-Peer
applications. For the grid, this role was held by major scientific efforts, from high-energy physics,
space and astrophysics, biotechnology, or climate change, that have reached a great visibility, and
mobilise large resources. Peer-to-peer applications were made visible by grassroots efforts from the
free / open source software and open contents movements addressing file sharing (for instance
Gnutella), co-operative computation (Seti@Home), co-operative publishing (Publius) or software
development (arch). During FP6, the prominent role of these driving Grid and Peer-to-Peer
applications will remain very important. We do not describe them in detail in this section, as they are
well known, but of course consider them in priority when mapping applications to architectures and
research needs in the next sections.
Other Commercial and Societal Applications
In the following, some of the potentially most important commercial (and more generally societal)
applications that are expected to drive the development of Grid/P2P environments are analysed.
Distributed and Collaborative Product Development and Production: The most challenging
complex problems in the industrial engineering area are to set-up integrated development and
manufacturing environments across distributed plants of large manufacturers in different countries
often resulting from mergers (short term); to integrate large numbers of suppliers (often SMEs) into
collaborative development just-in-time production, service and maintenance operations (medium
term); and to address the complexity of multidisciplinary design and optimisation (long term).
Whereas the development and design parts of these integrated applications are mostly computeintensive (high-end Grid applications), the problems in just-in-time production, service and
maintenance are more data-centric (low-end P2P applications, see also paragraph on “virtual
enterprises”). Industrial control becomes more and more based on intelligent, networked sensors,
which themselves can be considered as GRIDs.
As in the past, when large scale industrial engineering was the major driver for transfer of High
Performance Computing and Networking technologies from the research labs to industry, being
closest to the traditional Grid application area of fundamental sciences and applied physics, distributed
and collaborative product development and production has the potential to become one of the first
areas of commercial use of Grid/P2P technologies. Driven by the market pressure of higher quality at
lower cost, Europe’s traditional strength in technologies enabling these applications (collaborative
working, virtual reality, middleware for application integration, advanced software engineering
technologies (e.g. agents, cognition), real-time control) offers excellent opportunities for European
industry across several levels of the value chain to drive the development of mature Grid/P2P
environments for commercial use. Among the major barriers to be overcome, the needs for more
security and industrial strength, interoperable middleware range highest. The open source model for
middleware may be an attractive solution acceptable to industry, because it reduces the dependency on
IRG Complex Problem Solving including GRID and RNI
26
Draft V 2.0 - 14/05/02
monopolistic HW and SW suppliers from outside Europe. However, stability and maintenance are
major barriers for this business model.
Application Services Provision: After recent decline of the ASP sector, Grid/P2P technologies have
the potential to offer a new dimension to application services provision by offering solutions for
complex problems, which may make the sector more profitable again in certain areas. By offering not
only the applications but all the necessary computing and networking infrastructure in a scalable way
(Grid/P2P-based), users of application services can reduce their initial investment to a minimum and
instead pay per use. This scheme is particularly attractive for SMEs (e.g. with compute-intensive
applications such as in engineering and design) or smaller hospitals, who are enabled to run complex
applications that they currently cannot afford due to the high costs of the necessary computing and
networking infrastructure. The scheme also allows for use of multiple tools of the same kind at low
cost to give impetus for new solutions (e.g. simulation tools), which currently for many organisations
is not affordable due to the high overhead related to installation and maintenance.
Due to its industrial structure being dominated by SMEs, Europe is well positioned in the area of
application services provisioning. The most important aspects that are needed to be addressed for this
area are appropriate legal and billing frameworks including single sign-on, effective brokerage
systems as well as issues of stability, security, and quality of service.
Virtual Enterprises (scalable dynamic virtual organisation): The biggest challenge in the area of
dynamic virtual enterprise systems is the problem of interoperability for inter-enterprise collaboration.
Interoperability is today considered as too complex and too expensive and must in the future proceed
far beyond current B2B practices. Grid/P2P-like environment could form an ideal infrastructure for
dynamic virtual enterprises. However, current virtual enterprise systems lack common accepted
reference models and have therefore resulted in each of them implementing their own infrastructure,
consequently resulting in a low level of interoperability between these platforms. The requirements for
virtual enterprise systems stem from the requirements of the targeted functionality plus general
requirements on security, resource management, etc. In addition, virtual enterprise infrastructures
should have extensive and rich support for information and knowledge management including elearning. Related to Grid/P2P environments, requirements are more data-centric for the commercerelated applications (low-end P2P) and more compute-intensive for engineering applications related to
design, production, and control. European industry is well placed: The automotive manufacturers for
example have tackled the interoperability problem on an industry sector basis (ENX). They are
however still far away from their vision of a super platform supporting seamless integration of the
OEM’s internal applications with the external applications of dealers, logistic operators, suppliers and
others.
Educational applications
The educational sector is by nature a domain in which there a great number of sources of valuable
contents, and for which incremental creation and refinement of contents is essential. This has lead to
proposals for large-scale peer-to-peer educational resources and “educational Napsters”. The affinity
of this domain with open source software is essential, as the limit between software and contents is
often very thin in educational applications. The driving forces (including in technology development)
are the teachers themselves, which raises issues of how to be able the related constituency can
organise itself.
Co-operative publication or content creation networks
Open publishing and distributed content creation are emerging revolutions. Open publishing is driven
by open scientific publishing, while distributed content creation and sharing is the creator’s and
societal answer to the difficulties facing centralised media and content distribution in offering
attractive offers on digital networks. In the future, it is likely that new forms of commercial services
adding value to the peer-to-peer production of contents will develop. Part of the centralised
distribution industry will re-engineer itself in the direction of these new applications and develop and
fund their own infrastructure developments in terms of networks, storage, and other hardware and
software layers. Besides digital right management and interoperability issues between providers and
across countries, aspects like data compression, broadband access to and from the home, privacy, and
user rights enforcement need to be addressed.
IRG Complex Problem Solving including GRID and RNI
27
Draft V 2.0 - 14/05/02
Global information access
The generalisation of non-textual networked information will raise new challenges in information
access, and the development of technology answers to these challenges will in turn result in enabling
technology. Searching for sound, image, 3D or other type of rich media contents calls for a
combination of content processing technology (falling out of the scope of this document) and
distributed computation and storage. Both Grid (in search engines types of applications) and peer-topeer technologies (in distributed search) have a potential in this field.
In the health sector, high-end Grid-like environments are likely to enable the broad use of interactive
simulations for clinical operations planning and real-time support, and the control of micro robotic
operation facilities. Quality of service and broadband access are among the major issues to be
addressed. Low-end P2P environments are likely to enable the broad use of digital libraries, and
medical information and knowledge networks by hospitals and individual practitioners. Privacy,
confidentiality, and mobility are among the major issues to be addressed.
Our societies’ critical infrastructures such as information and communications, water and energy
supply including electricity (generation, transmission and distribution), transportation, banking and
finance and government services (including emergency services) are becoming increasingly
interconnected and dependent on information and telecommunication technology components and
infrastructures exposing them to new vulnerabilities such as cyber-attacks. In particular after 11
September 2001, these vulnerabilities have received increased attention by utility providers and
(multi-) national security bodies. The “global” problem of modelling and simulating these
interdependencies and the resulting cascading effects is extremely complex and infrastructure
providers and research in Europe and the US have only recently started to address it. The requirements
of decision support tools (for long term reduction of vulnerabilities and/or time critical damage
limitation in case of accidents or attacks) from ITC infrastructures based on Grid, P2P and networks of
sensors are extremely stringent related to dependability and all aspects of security / trust and
confidence.
Global Monitoring for Environment and Security (GMES): On the research side making use of
highly distributed large data sets and complex models and on the commercial side moving from data
and map based products used by the research communities towards commercial, fully operational
systems and information services are among the biggest challenges in the area. With its strength in
earth observation, environmental modelling, meteo services, earth sciences and the enabling
information technologies, Europe is well positioned to tackle the vision of Europe-wide harmonised,
high quality environmental information services. These applications set strong requirements on the
underlying ICT infrastructures, in particular strong computing capacities for multi-physics simulations
and distributed data, information and knowledge management. Though there is a strong political
willingness in Europe to have its own independent environmental observation capability, the area
suffers from the absence of a real business case, the currently high cost and limited availability of EO
data, and the question how far the large institutional users are prepared to pay.
Intelligent Transport Systems : The biggest challenges in the area of multi-source data fusion for
prediction and decision support in multi-modal traffic systems are to be expected in the areas of
enhanced network-wide traffic control and multi-modal traveller information services, improved
forecasting capacity of the state conditions of complex networks, and the integration with air pollution
modelling. These challenges set strong requirements on the underlying ICT-infrastructures, in
particular complex integrated networks of sensors, large amounts of heterogeneous and distributed
data and an increasing demand for high computing capacity. Issues around mobility, privacy, and
quality of service are among the most important ones to be tackled. With broad experience and knowhow in intelligent traffic systems and the underlying hard- and software technologies, Europe is well
positioned to maintain its leading role. Though there is a strong political willingness to support urban
sustainable development and to control urban air pollution by promoting transport inter-modality and
demand management, the area suffers from the absence of a real business case, end-user reluctance to
pay for the services, and the absence of widely accepted system architectures and a unified vision.
Overall, in all application areas, there is a need for developing an infrastructure to support flexible,
secure resource sharing among dynamic collection of individuals, institutions and resources. There is a
demand for developments to enable sustainable business opportunities and to complement and extend
IRG Complex Problem Solving including GRID and RNI
28
Draft V 2.0 - 14/05/02
present technologies for business-to-business applications, application service providers, system
integrators, and storage service providers. All applications share a need for extensive and rich support
for information and knowledge management including e-learning capabilities. In the short term, in
most applications the integration of existing technologies, the agreement on reference architectures,
and the provision of broadband, low-latency networks for access of all communities are the most
pressing issues. For user communities to be willing to accept Grid/P2P technologies, also the security
mechanisms have to be significantly improved.
Classification of Complex Problems
In the following it is tried to categorise the complex problems identified above for various
applications:
•
Very large data flow driven problems (ex: high-energy physics, direct observation at extremely
low scale or time granularity of chemical and biological processes, some astrophysical problems).
Data is generated by a relatively small number of sources (instruments) in huge quantity. The
computation chain involves steps of raw data reduction and data analysis. Served as a model for
the initial design of grid-architectures. Some of these problems have been mapped rather to peerto-peer architectures (ex: seti@home), but it seems that it is mostly because of the “non –urgent,
non security-critical” character of the problem that this was possible.
•
Large scale complex algorithms (ex: number crunching, more generally all NP-complete or
provenly non-polynomial problems). Similar to previous, but the complexity does not arise
necessarily from data quantity, but from the complexity of computation. Solutions depend on the
ability to segment and parallelise this computation. Solutions often involve parallel computation
and algorithmic speed-up techniques.
•
Large scale heterogeneous (multi-physics) models (ex: climate change, environment
modelling and monitoring, weather). Large date sets are produced from geographically distributed
sources. Interpreting the data is complex in the model(s) associated with a given type of data, but
even more in understanding their inter-relation, or calibrating them. The models can be more or
less closely coupled, ranging from direct equational coupling to separated models with mapping in
interfaces. This is likely to result in different suitable architectures.
•
Large-scale data with complex distributed analysis or interpretation (genomics and
proteinomics, virtual observatories). Large data set(s) exist and are accessible from most often
distributed databases. The nature of the problem is such that a team can work on parts of the data
relatively independently. Analysing and understanding it involve a mix of sometimes very largescale computation, human analysis and possibly confrontation with additional evidence
(experiments, validation by new observations). Distributed annotation by many teams of the data
is key to the overall progress. Interestingly this type of problem might involve a mix of grid
computing (for computation), classical distributed databases (for the data) and peer-to-peer
databases (for interpreted data and annotations).
•
Synchronous distributed co-operation around one model (distributed engineering). Timeconstrained design process. A large part of the complexity (and risks) arises from the
interdependency between design decisions taken by the various teams for parts of the designed
system. This calls for permanent shared simulation of the overall system. Iterative process based
on different types of models at different stages, and ability to move from one stage to the other.
Much of the infrastructure planned for grid-computing can apply, with some additional
complexity/constraints from larger scale geographical distribution. Probably calls for multiscale
organisation (grids of grids?).
•
Managing co-operative production of entities by a large number of independent players
(distributed software and content production, e-learning resources). Specificity is that this is about
creating (possible multiple versions) of entities in a distributed parallel manner. Complex issues
(particularly for software) on versioning, merging, consistency, resiliency, manageability, etc. See
arch and at a lesser extent subversion versioning systems for free software development.
Naturally maps to peer-to-peer storage and management, but still uncertain how peer-to-peer
solutions will compete with centralised hosting. Peer-to-peer architectures particularly attractive
when entities are not too much dependent on each other (e-learning).
•
Searching in a very large number of distributed sources (Web searching, distributed
bibliography and bibliometry). As the data-repatriation and centralised indexing models of search
engines poorly apply to non-textual data, and as progressive standardisation of metadata gives
more credibility to the possibility of de-centralised search, there are growing efforts at building
decentralised search systems. In such systems a query would be transmitted to the local storage
IRG Complex Problem Solving including GRID and RNI
29
Draft V 2.0 - 14/05/02
server, “executed there”, and the consolidation would occur without a centralised search engine.
This problem is at the same time a component of a peer-to-peer system (discovery) and a
possible application.
IRG Complex Problem Solving including GRID and RNI
30
Draft V 2.0 - 14/05/02
Annex III – Draft - SWOT Analysis
Technology Areas
Strengths
Grid computing
§
§
§
§
§
§
§
§
§
§
§
§
§
Opportunities
Weaknesses
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
Threats
§
§
§
§
§
§
§
§
God network infrastructure in Europe and experience in distributed computing;
More efficient use of computing resources (CPU, storage, network); a seamless infrastructure;
Enabler for virtual communities
Make processing power available on demand anywhere, anytime;
Single-sign-on utility computing
Improved collaboration and information sharing (information bus);
Resource optimisation and access; cost sharing;
Creation of scalable dynamic multi-institutional virtual communities;
Improved Data mining and pattern finding;
Real-time engineering and design;
Real-time instrument linking;
Major corporations start to commercialise Grid technologies;
Potential general uses include capacity-on-demand, ability to improve job turnaround; business
continuity and disaster recovery through storage of critical data off-site; and visualisation of
large data sets.
Improved knowledge sharing;
Middleware issues; interoperability
Resource discovery and management; heterogeneous performance in the resources;
economic model for resource utilisation;
Billing and accounting issues;
Cross-organisational policy issues;
Grid used mainly used in “big science”;
Software not yet industrial strength;
Underlying security issues: security and attitudes not yet ready for commercial Grid computing
over the Internet;
Lack of Grid-aware applications;
Network latency and performance issues;
Skill gap;
Grid protocols and tools are US driven;
Grid as the next internet/web;
Capability computing;
Offers new models for services, including mobility, any time any place;
Use Open Source for all layers;
Europe has proven records and potential for middleware development;
Leverage national Grid activities;
Better distribution of resources where needed;
Just in time calculations/operations/monitoring;
Distributed agents;
Improve business processes and supply chain management by better integration of diverse
computer systems;
Integration of “event-driven” devices and sensors;
Challenge to turn the promises of the Grid into something to be of use to business and
industry;
Standards will play an increasingly important role as Grid computing begins to gain wider
acceptance;
Potential for improved application integration;
Underlying complexity: complex and unpredictable concurrent environment;
Security issues: potential presence of untrusted resources;
Impediments to connectivity, including firewalls and oversubscribed network resources; widearea connectivity/bandwidth issues;
Intellectual property in an environment explicitly designed for instant sharing;
Integration of mixed resource types into problem solving environments, novel application
areas, programming environments;
Over-hyped; protocols and tools US dominated.
IRG Complex Problem Solving including GRID and RNI
31
Draft V 2.0 - 14/05/02
Peer-to-Peer Computing
§
§
§
Strengths
§
§
§
§
§
§
§
§
§
Weaknesses
§
§
§
§
§
§
§
§
§
§
§
Threats
Opportunities
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
Europe has the competency to build P2P applications
P2P resource sharing: get more from the IT resources in terms of processing power and storage;
exploit idle compute power, but P2P not just distributed computing, most applications aim at
collaboration without the need for formal IT structures;
Potential for reducing costs in business scenarios; solve real corporate information sharing
problems; flexible, autonomous & distributed systems for information exchange
Increased efficiency (no bottleneck at central server), serverless collaboration;
Increase knowledge worker productivity enabling the knowledge worker;
Content distribution mechanism; Sharing content cache over high speed LAN; data mirroring and
striping using shared storage; deep search beyond the web;
Ability to provide economies of scale through decentralised person-to-person or device-to-device
model;
Timely collaboration; instant messaging; Dynamic and adaptive discovery of information;
Collaboration: P2P computing empowers individuals and teams to create and administer realtime and off-line collaboration areas in a variety of ways, whether administered, un-administered,
across the Internet, or behind the firewall;
Edge services: P2P computing can help businesses deliver services and capabilities more
efficiently across diverse geographic boundaries;
Distributed Computing and resource sharing: P2P computing can help businesses with largescale computer processing; can be combined with the conventional client/server model,
complementary to centralised solutions needs;
Intelligent agents: P2P computing allows computing networks to dynamically work together using
intelligent agents;
Unclear business model; unclear standards; unclear how to make money from P2P applications,
value chain and revenue opportunities unclear;
P2P technology relatively immature;
Europe is lagging behind in development of P2P applications;
Copyright infringement issues; standards issues;
Middleware and Security issues; Resource allocation and scheduling;
Archiving, administration, accountability;
Network capacity, performance, Bandwidth and latency issues;
Challenges: peers are not always on and connected, have no durable names, firewalls lock
transfer between peers;
P2P communication & interoperability problems; issues caused by NATs and firewalls;
Integration into user workflow and applications;
Like any early stage concept with a disruptive potential, P2P appears like a business concept
innovation in search of a problem;
More then distribution of MP3 files; more than academic and philanthropic computing;
P2P is most powerful when combined with a compelling business need;
Content delivery (audio, video, software, games) network to end users targeted at enterprises and
media companies;
Collaborative writing, knowledge discovery, distributed information access;
Use of untapped computational resources can easily surpass the normal available power of an
enterprise system without distributed computing Utilisation of edge resources in new ways;
Decentralised applications to replace centrally managed applications;
Manage information overload; P2P search engines;
P2P delivery services for Internet video and other digital media; ASP offerings;
P2P technology to enable commerce, workflow, supply chain, work between users (serverless);
P2P applications open revenue opportunities for service companies;
Messaging between apps and people: Alerts; App-to-app communication;
Real-time element in static applications: Collaborative design, Games, e-Learning;
Bringing e-Commerce to the desktop;
P2P file swapping for PDA (Jxta); P2P may drive deployment of IPv6;
Over-hyped; US dominated;
A system administrator’s nightmare; P2P applications may undermine backbone performance by
causing unmanageable congestion at peering points;
Difficulty locating and fixing problems;
Difficult to control, might prove to be source of security breeches;
Reliability and security; peers don’t necessarily have trust relationships; missing code of conduct;
Accountability problems;
IRG Complex Problem Solving including GRID and RNI
32
Draft V 2.0 - 14/05/02
Research Network Infrastructures
Weaknesses
Strengths
§
§
§
§
§
§
§
§
§
§
§
Opportunities
§
§
§
§
§
§
§
Threats
§
§
§
§
§
§
§
Clear rationale for EU support (subsidiarity principle easily applicable as national infrastructure
initiatives exist e.g.NRENs and GRIDs)
Global leader ship in Research networking backbones (GEANT)
Europe is a global player in the area of research networks
Leading edge developments with US initiatives on GRIDs
Global leadership on advanced IPv6 test-beds
Strong governmental support (ENPG and political support on Parliament & Council)
Downturn of the ICT sector
Europe not being able to be in the forefront of the technology (USA dominance)
Organisational structure of the National Research and Education Networks (NRENs) too rigid
Dependency on big science disciplines to be able to launch GRID infrastructures – unability to
use state of the art GRIDs
Build on results from leading edge existing test-beds for the introduction of the next generation
GEANT
Make GEANT IPv6-enabled and later on GRID-enabled
Build on the results of the photonics area to create a Terabit/s GEANT
Extend the reach of GEANT by interconnecting it with the research network backbones in other
regions (US, Canada, Asia Pacific, Mediterranean, Latin America)
Support and integrate research projects on-top of the network infrastructure
Exploit the opportunities of a liberalised telecommunication environment (“own” fibre, “own” the
network, etc.)
Enhance the “inclusiveness” of the research network infrastructure by taking into account the
demands of various actors in the field (e.g. schools, educational networks, libraries, e-learning,
etc.)
Use research infrastructure as a political instrument to support the development and foster
cohesion.
Become The Next Generation Network by using advanced test-beds to evaluate and validate
new technologies (like Grids, GMPLS, photonics).
Deploy GRIDs infrastructures to support virtual communities
Fragmentation of funding in FP6
Application of two different procedures in FP6
Usage of two different budget Lines in FP6
Possible fragmentation on the national approaches to Research Infrastructures
GRIDs remaining at the level of a “fashion”
IRG Complex Problem Solving including GRID and RNI
33
Draft V 2.0 - 14/05/02
Weaknesses
Strengths
Mobile/Wireless IPv6
§
§
§
§
§
§
§
§
§
§
§
§
§
§
Opportunities
§
§
§
§
§
§
§
§
§
§
§
§
§
Threats
§
§
§
§
Larger address space
Support for hierarchical address aggregation
Simplified IPv6 data packet header for routing efficiency and performance
Autoconfiguration
Security with mandatory implementation of IP Security (IPSec)
Restores end-to-end connection
Improved support for Mobile IP and mobile (and ad-hoc) computing devices
Enhanced multicast networking support.
Larger header (extra 24 bytes per packet above IPv4) decreases efficiency, particularly critical
for mobile/wireless
Extensive trials and interoperability tests required.
Lack of IPv6-enabled applications.
Overhead due to the co-existence of IPv4 and IPv6 (resulting from tunnelling, triangulation, etc)
is tremendous.
Lack of IPv6 backbone infrastructure.
IPv4 address is reaching exhaustion (the number of Internet enabled devices is growing
dramatically)
End-to-end global reachability and Internet scalability, eliminating the need for NATs
Easier network renumbering for improved plug and play support, flexibility and easier network
administration and management
Enables improved business models and potentiates new applications for a broad range of
organisations and lower costs
Meeting the requirements in some standards (e.g. IPv6 is required for 3G IP multimedia - IMS
domain)
Coexistence with IPv4 enabling to leverage the current investments
Major economy blocks around the globe recognised the need to move to IPv6
To become “The Protocol” in the convergence of different access communications platforms
Essential IPv6 components developed and standardised
The three regional registries (RIPE NCC, APNIC and ARIN) share a common IP address
allocation policy
Major vendors are ready to satisfy the demand for IPv6 deployment
Favourable political environment for its rapid integration in the existing networks.
Uncertainty on the speed that IPv6 will be widely deployed
ISPs may see the introduction of IPv6 as an avoidable expense and delay its introduction as
much as possible
Lack of expertise in IPv6. This is particularly critical in what concerns mobile/wireless
applications.
Its introduction requires careful planning as it needs to be done while maintaining (and even
extending) the existing IPv4 infrastructure.
Since IPv4 and IPv6 will coexist for a long time, care needs to be taken to minimise the
overhead (particularly in Mobile/wireless environments) resulting from tunneling and/or to run
dual protocol (IPv4 and IPv6).
Many of the transition mechanism, which translates between IPv6 and IPv4, are more complex
than existing IPv4 NAT mechanisms, but the solution is much more flexible and certainly more
scalable.
IRG Complex Problem Solving including GRID and RNI
34
Draft V 2.0 - 14/05/02
Application Areas
Threats
Opportunities
Weaknesses
Strengths
Distributed and Shared Infrastructure for e-Learning
§
§
§
§
Europe's experience in distance learning and pedagogical innovation
EU's political commitment through the e-Learning action plan
Long history of Cupertino among Universities
Strong VR simulation expertise
§
§
§
§
Fragmented market
No large industrial players
Significant infrastructure costs
Only small part of the e-e-learning issues addressed
§
§
Possibility to implement new pedagogical paradigms, e.g. distributed collaborative learning
Challenging technological problems to be solved:
§



Demanding end-user needs:


§
§
§
§
multicasting
virtual/mixed reality environments
high-quality visualisation environments
reliability/availability of the infrastructure
user-friendly interfaces
Too high infrastructure costs
Complex application development required
Technology not reliable enough
Resistance to change
IRG Complex Problem Solving including GRID and RNI
35
Draft V 2.0 - 14/05/02
Complex Systems in Industrial Engineering (examples)
Weaknesses
Strengths
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
Opportunities
§
§
§
§
§
§
§
Threats
§
§
§
§
§
§
§
§
§
§
§
§
Large industrial players with distributed teams and infrastructures
Supply industries closely linked to large manufacturers
Strong experience in application integration (legacy and new)
Large scale industrial engineering was a major driver for moving HPCN technologies from the
research labs to industrial use
Strong expertise and experience in advanced application development
Ever increasing demand for increased ICT resources to solve complex multidisciplinary
systems of applications
Strong expertise in collaborative working
Strong expertise in visualisation incl. VR
Strong demand for general uses such as for just-in-time production, capacity-on-demand,
visualisation/VR driven by the objective of better quality at reduced costs
Real-time engineering
Industrial support for open source developments
Insufficient commercial middleware including services
SW not industrial strength
Long term maintenance and commitment for open source
Fragmented SW market – no integrated/system view
Grid/p2p mainly used in research
Research infrastructures not available for industrial research
Heterogeneous intra- and inter-organisational policy issues
Firewalls between organisations
Discovery, management, scheduling of heterogeneous resources
Insufficient or immature security mechanisms
Insufficient awareness of security needs; billing and accounting issues
Skills gap
Industrial engineering as the driver for Grid/p2p to move from research to industry
Grid/p2p as the next supercomputing platform
Grid/p2p as a next generation platform for application integration
Grid/p2p as the next “intra”-net for (virtual) enterprises – improved information/knowledge
sharing
Grid/p2p as platform to integrate the engineering tools of large manufacturers and their supply
industries
Creation of scaleable multi-organisational virtual enterprises (supply chain management,
business processes)
ASP using Grid/p2p platforms allows organisation (in particular SMEs) access to
compute/data-intensive engineering without large initial investments
Reduced infrastructure costs through better use/distribution of resources
Modelling, simulation, and control of complex, integrated, distributed systems (including
multidisciplinary systems, distributed sensors, event driven)
Make systems more intelligent by introduction of “cognition“ technologies using advanced SW
engineering concepts such as intelligent and mobile agents
Real-time control of complete industrial processes: planning, production, sales and delivery
(e.g. just-in-time production, capacity-on-demand)
Grid/p2p platforms for modelling and simulation of (inter) dependencies of critical
infrastructures aiming at reduction of vulnerabilities (short term) and real-time decision support
in case of faults or attacks including mitigation effects
Resistance to open ICT infrastructures for collaboration across company limits
Vulnerability through insufficient awareness and thoroughness related to security
Potential presence of untrusted resources
Inhomogeneous ICT infrastructures
Dependency on proprietary, often monopolistic HW and SW products
Immature, unreliable, or inefficient middleware
Complexity
Attempt to solve too many problems at a time (e.g. mixed resource types in PSEs, novel
application areas, new programming environments)
Sharing of IPRs
Reduced research efforts of large companies due to predominance of short term goals
Over-hyped
IRG Complex Problem Solving including GRID and RNI
36
Draft V 2.0 - 14/05/02
Global Monitoring for the Environment and Security (GMES)
Opportunities
Weaknesses
Strengths
The attached draft SWOT analysis is deemed to support the decision-makers with a strategic
assessment of the area, the risks and opportunities and the associated actions of the field/theme under
consideration.
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
Threats
§
§
§
§
§
§
§
Europe’s know-how in environmental science
Europe’s capability in monitoring (sensors) and modelling
Europe’s capability in solving complex problems (GRID)
Europe’s political willingness to fight global climate change, to support sustainable
development
Europe’s strength in meteo services (EUMETSAT)
Providing Earth Observation based data & information services
Europe’s dedication to develop its own, independent environmental observation capability.
Fragmented service-market
Unstructured value-added companies
Lack of EO data policy
Absence of standardised information/service architectures
No user champions
Developments driven so far by space industry and science & research may not deliver easy-touse; easy-to understand services. (for experts use only?)
Absence of business case
Low commitment of ICT sector
To capitalise on Europe’s strength in earth observation, environmental modelling, earth
sciences and Information Technology
To deliver European-wide, harmonised, high quality environmental information services
To move from data and map based products used by the research community towards
commercial, fully operational systems and information services
To improve both the quality of environment policy making (aim at better confidence levels in
the prediction of trends) and day-to-day environmental management on local, and regional
levels (“best data, best models, best support tools – anywhere, anytime”)
To make use of highly distributed large data sets and complex models, and to make scientific
knowledge widely available for the benefit of society.
Dependency on continuity and coverage of EO-satellite data (e.g. ENVISAT, ERS)
High cost of EO data
Lack of clarity to exploit dual-use technologies
Political pressure to deliver short-term services (2005) may not allow to develop for more
visionary infrastructures (beyond 2008)
What are the users (largely institutional market) willing to pay?
Dependency on successful implementation of public-private partnerships on precursor
initiatives, such as GALILEO
IRG Complex Problem Solving including GRID and RNI
37
Draft V 2.0 - 14/05/02
Annex IV – List of Background Documents
The Grid: Blueprint for a New Computing Infrastructure
Ian Foster and Carl Kesselman, editors, Morgan Kaufmann Publishers, Inc, 1999
The Anatomy of the Grid: Enabling Scalable Virtual Organizations.
I. Foster, C. Kesselman, S. Tuecke. International J. Supercomputer Applications, 15(3), 2001.
Defines Grid computing and the associated research field, proposes a Grid architecture, and discusses
the relationships between Grid technologies and other contemporary technologies.
http://www.globus.org/research/papers.html#OGSA
Workshop on Grid Technologies, Brussels, 22 – 23 June 2000
Report available on demand; contact [email protected]
FP6 Programme Consultation Meeting PCM3: “Large-scale Distributed Systems and
Platforms, including GRID-based Systems”, Brussels, 10 – 11 April 2001
ftp://ftp.cordis.lu/pub/ist/docs/pcm3-final-report.pdf
Peer-to-Peer, Harnessing the Benefits of a Disruptive Technology
Edited by Andy Oram, O’Reilly & Associates, March 2001
The Physiology of the Grid: An Open Grid Services Architecture for Distributed
Systems Integration. I. Foster, C. Kesselman, J. Nick, S. Tuecke; January, 2002.
http://www.globus.org/research/papers.html#OGSA
Grid Service Specification
S. Tuecke, K. Czajkowski, I. Foster, J. Frey, S. Graham, C. Kesselman; February, 2002.
http://www.globus.org/research/papers.html#OGSA
UK Research Councils e-Science core Programme
http://www.research-councils.ac.uk/escience/
Gridforum
http://www.gridforum.org/
IRG Complex Problem Solving including GRID and RNI
38
Draft V 2.0 - 14/05/02
Annex V – Members of the IRG
Chairman: Wolfgang Boch (B4)
Members:
Joseph Bremer (D3)
Mario Campolargo (F2)
Joao da Silva (E4)
Thierry Van der Pyl (C4)
Jacques Bus (E2)
With contributions from:
Philippe Aigrain (E2)
Kyriakos Baxevanidis (F2)
Max Lemke (C4)
Roman Tirler (C4)
IRG Complex Problem Solving including GRID and RNI
39
Draft V 2.0 - 14/05/02
Annex VI – Glossary and Acronyms
API
ASP
B2B
B2C
CDN
DNS
DRM
Globus™
GMES
GRAM
GRID (FR)
ICT
ISP
J2EE™
JXTA
LDAP
MPI
NAT
.NET™
OGSA
OS
OS
P2P
QoS
SOAP
UDDI
WSDL
WSIO
XML
Application Programming Interface
Application Service Provider
Business to Business
Business to Consumer
Content Delivery Network
Domain Name System
Digital Right Management
Middleware toolkit
Global Monitoring for Environment and Security
Globus Resource Allocation Manager
Globalisation des ressources informatiques et des données
Information and Communication Technologies
Internet Service Provider
Java2 Enterprise Edition
The JXTA technology is a set of open, generalised peer-to-peer
protocols, defined as XML messages, allowing any connected device
to communicate and collaborate in a P2P manner
Light Weight Directory Access Protocol
Message Passing Interface
Network Address Translators
Microsoft Webservices Framework
Open Grid Services Architecture: a proposed evolution of the current
Globus toolkit towards a Grid system based on the integration of Grid
and web services concepts and technologies
Operating System
Operating System
Peer-to-Peer
Quality of Service
(Simple Object Access Protocol
Universal Description, Discovery, and Integration
Web Services Description Language
Web Services Interoperability Organisation
eXtensible Markup Language
IRG Complex Problem Solving including GRID and RNI
40
Draft V 2.0 - 14/05/02