FLUME
A Flexible Unified Model Environment
D4 High Level Design
Version 1.4
October 10th 2003
�
Rupert W. Ford and Graham D. Riley
c CROWN COPYRIGHT 2002, Met Office, All Rights Reserved.
Met Office Project No. PB/A4872
D4 High Level Design
Rupert W. Ford & Graham D. Riley
Manchester Informatics Ltd.
The University of Manchester,
Manchester,
M13 9PL,
U.K.
email: [rupert,griley]@cs.man.ac.uk
1 Introduction
The aim of this high level design document is to define the nature of an environment and a set of working
processes which must be supported to satisfy the requirements for a flexible coupling architecture as
described in the F LUME document D1 Requirements and D2 Constraints. An informal summary of the
proposed environment and processes is given in Section 2 to provide a context for this report. Section 3
provides an overview of the complete design, Section 4 discusses the top-level architecture and Section 5
describes the system components in more detail. The layered software architecture of the system is
described in Section 6 and Section 7 provides a brief summary. In many places, aspects of lower level
design issues are discussed but it is not the intention to provide a complete low level design in this
document.
The design builds in particular on the work in the W RF, PALM and CCSM projects referred to in
F LUME-related document D3 Review. The design builds on current and planned practices and technolo­
gies from environmental agencies around the world, and may be thought of as providing a super-set of
much of their functionality. One result of this is that the Met Office may follow a development path
which is “future proofed” in the sense of being able to use a sound initial technology base which allows
additional functionality to be added over time within the scope of the overall design. A specific out­
line implementation plan based on the PALM environment from Cerfacs is described in F LUME-related
project deliverable D5 Outline Implementation Plan.
2 Requirements Summary
The requirement is to provide an environment to allow scientists to develop and construct coupled mod­
els to simulate weather and climate phenomena. The environment should also support the development
of implementations of new models and the integration of existing models generated both within and
external to the Met Office.
Coupling models entails configuring them together such that the coupling data which the models
make available (or a subset thereof) is used, along with appropriate transformation functionality, to
satisfy the coupling data required by each of them. Figure 1 illustrates a simple, general case, one-way
coupling between two models using transformation functions to satisfy the input requirements (uses) of
2
one model from the output (provides) of other models 1 . The totality of data made available (provides)
and required (uses) by a model constitutes its interface. Figure 1 shows data fields in Model 1 (v1 to v3)
being output at different intervals, where ��� , the coupling interval, is the maximum interval. Each data
field is shown having some transformation function(s) applied in order to ensure that the requirements
of the receiving model are satisfied. Transformation functions include Grid transformation of the data,
for example from a high resolution grid to a low resolution grid. For data sent out at an interval different
to the coupling interval (as in the case of v1 and v2), transformations include the selection of one of the
instances (e.g. for v1, emitted at intervals ��� , the latest instance may be selected or the instance with the
smallest norm) or the summation of the instances which are emitted may be required (v2). Finally, data
from a number of models may have to be combined, with appropriate transformations, in order to satisfy
the requirements of the receiving model. This is depicted in the combination of v3 from Model 1 with
data emitted from an unspecified model at the interval ��� , which is different to the coupling interval.
Transformer
t3
iteration
control
C
S
G
C
LC
t1 .. t3 - model timestep (ti ≤ tc)
v1-4 - coupling data
tc
Input from
another model
Model 1
- Coupling interval
G
t2
v3
v4
iteration
control
tc
v2
tc
tc
tc
Σ
t1
v1
t1
G
Model 2
S
- Scale
G
- Grid transform
LC
- Linear combination
Σ
- Accumulation
C
- Choice/Select
Figure 1: Coupling Definition—General Case Overview
Specification of the sequencing and execution rates of models and transformers (indicated by the
iteration control arrows in Figure 1) and the definition of the coupling intervals between models is also
required. Note, transformation functions should generally be thought of as being equivalent to scientific
models–for example, they may be called explicitly from the control code. However, it is anticipated that
many transformer functions will actually be implemented “in-line”, within the functions implementing
data exchange between models, for performance reasons. Further, specification of the allocation of
model implementations and transformer functionality to executable files and their deployment on a set
1
For reference, in later FLUME documents, the terms put and get are used in reference to the mechanism by which models
exchange data.
3
of available computing resources must be provided. This control must be flexible enough to enable
developers of coupled systems to explore the space of configurations to investigate both scientifically
meaningful and high performance implementations. The layered architecture proposed to implement
the coupling is described in Section 6.
3 Design Overview
During the construction of a coupled model, developers and users will engage in three basic phases of
activity. The design provides explicitly for these. These are:
Definition—of the models and transformer functionality (“entities”) to be coupled. This is princi­
pally an activity associated with a developer. These entities will exist in some form of repository;
�
Composition—of the selected entities in the repositories into a coupled system;
�
�
Deployment—of the coupled system onto a set of computing resources.
These phases together will be referred to as the DCD system. An overview of the system is presented
in Figure 2.
Selection
Definition
update
Developer
update
update
User
Metadata
Repository
Model
Interface
Repository
Transformer
Interface
Repository
Composition
m
(coupling)
n
Implementation
Repository
m
n
Implementation
Repository
Deployment
(sequencing/concurrency/
code/machines)
save
Script
Repository
Figure 2: System—Overview
Users of the system will principally be involved with the selection of entities (from those in the
repositories, including the selection of any configuration options an entity provides), their Composition
(i.e. the coupling of models), and with Deployment issues (related to machine-level details). Each phase
will require the provision by the user or developer of metadata which will drive the activity in each
4
phase and capture cross-dependencies between the activities in each phase 2 . There are dependencies be­
tween entity interfaces and implementations. For example, an implementation of a model may support
configuration “switches” which affect the interface it displays. Also, there may be multiple implemen­
tations which support the same interface. This is depicted by the m:n relationship between interfaces
and implementations described in Section 4. As a user makes selections (of interfaces and/or models
and transformer implementations and their configuration switches) these entities become “visible” to the
Composition and Deployment phases. Metadata, provided by users and developers and possibly based
on their past experience using models, will be used to capture constraints in the selections of entities and
their configurations. This will be used to prevent the selection of incompatible models etc.
Note that transformers and models are treated as equivalent entities in the DCD system and in general
the term model will be used to represent either.
Each phase has an associated Environment through which a User may interact with the system.
These are described in Section 5.
The design is extensible in each phase, supporting, for example, the development of new models and
also their deployment on a variety of machine architectures and configurations. Repositories of model
and transformer interfaces and implementations will be developed.The system provides flexible support
for composition through the use of these extensible repositories.
The control information required to support decisions at each level is able to be defined at the inter­
face between layers. Such an approach supports clear allocation of responsibility for functionality, ease
of maintenance and results in an extensible system.
Initially, the system will support manual decision making in the above areas, but, in the longer
term, the aim would be to allow the incorporation of automated decision support in this space. Manual
control will take the form of the setting of the metadata which is used to implement these decisions. The
metadata associated with components of the system could be extensive. Initially, the focus will be on
the use of metadata pertinent to the composition and deployment phases.
The system will support an amount of automatic code generation — e.g. the driver sequencing con­
trol code (or the generation of appropriate parameters for a generic controller), the contents of routines
supporting communication between models and compilation and linking scripts etc.
Model deployment has two parts—the agglomeration of functionality , essentially into executables,
and the selection of computing resources. Metadata is required to describe the user’s requirements with
automatic generation of implementation being applied where possible, to enforce consistency.
The design will allow for deployment of the coupled system on a general clustered architecture
model. This implies that. in general, two levels of parallelism exist: parallelism across models (concur­
rency) and parallelism within models. In the PALM system (see D3 Review), parallelism across models
is handled with branches, which achieve concurrency, and units which may themselves be parallel im­
plementations of models. In the W RF system (see D3 Review), Michalakes addresses the same issue
within the context of a single model and refers to regions and tiles which are equivalent to MPI and
OpenMP parallel structures, respectively.
It is important to differentiate two levels of communication in the system. The first involves com­
munication between models (to implement coupling exchange) and the second is communication within
models (supporting parallel execution). The latter is an implementation issue for individual models and
is beyond the scope of this document. It is a lower level design task to ensure that communication
technologies used at each level can inter-operate.
2
Often, the term metadata will be used in a relatively loose sense to refer to “information about data”. For example, the
units in which a temperature field’s values are expressed is metadata about the temperature field and, for models which present
optional scientific algorithms, the description of the options available (which may be stored as text strings in XML documents)
is metadata. However, the values of switches (possibly logical flags appearing in computer source code) which enable specific
selections is data. It should be noted that metadata at one level may be considered as data at another. For example, for code
which manipulates the information related to the switches described above, perhaps for display in a user interface, the metadata
is actually data.
5
User decisions made in each environment may be used to constrain choices in other environments.
For example, a coupled system may be defined in the composition environment and then model imple­
mentations selected before selecting hardware resources in the deployment environment. In the deploy­
ment environment certain choices may be unavailable as implementations for the models may not be
available for all architectures. Conversely, a User may choose a specific hardware resource first and this
may constrain the models available to be configured into a coupled system.
4 Architecture Overview
The system will have the following components:
Models Implementations—implementing aspects of the application domain functionality (weather
and climate phenomena in the case of the UM) and support sub-systems such as the diagnostic
processing system and transformers.
�
Model Interfaces—metadata specifying the coupling data a model provides and uses and the
diagnostics provided by a model.
�
Transformer Implementations—implementations of functions used to facilitate the use of data
by models. Transformer functions include summation and regridding,for example.
�
Transformer Interfaces—metadata describing the coupling data a transformation function pro­
vides and uses.
�
Model Metadata—capturing information other than a models coupling data including the defi­
nitions of the knobs and switches provided by the model, constraints between switches, coupling
compatibility information that is known about the model (whether there are certain models with
which this model is incompatible in a scientific sense) and possibly pointers to published papers
relating to the science of the model etc. which may be of use to users constructing coupled models.
�
Repositories—in which instances of the above items are stored.
�
The system also provides the following environments:
A Definition environment—to support the creation and updating of model interfaces and trans­
former functionality. This environment will be tightly integrated with the coding environment.
�
A Composition environment—to compose models (via their interfaces using) and transformer
functionality to achieve the desired coupled model.
�
A Deployment environment—which supports:
�
– the specification of the software structure (number of executables and the allocation of func­
tionality to them) and
– the deployment of implementations of the components of the coupled system on the selected
target architecture(s), including the handling of concurrency, parallelism and communication
issues both between models and internal to models which exploit parallelism.
�
Repositories of (user) Selections—in which the activities in the composition and deployment
environments may be kept for future use.
The Definition environment is used principally by scientific developers to populate the repositories.
Users select entities and then interact with them through the Composition and Deployment environments
in order to create and configure the coupled model they require. Composition focuses on defining
6
the coupling between models, Deployment focuses on specifying control information related to the
sequencing and concurrency of models and also on which machines the models will execute. User
selections include the specification of “switches” controlling optional algorithms that may be available in
a model along with the specification of any associated “knobs” which set parameters for the algorithms.
Repositories of interfaces and models will be maintained. Logically, there will be several reposito­
ries: one for each category of functionality. It is to be expected that implementations may use the same
technology to implement each of the repositories, possibly as a single consolidated repository.
Descriptions at each level will capture information which is required between components. Rela­
tively simple and robust metadata technologies, such as XML, are recommended. User interaction to
set up metadata will be supported by each environment (see Section 5). It will also be possible to save
composition and deployment definitions/selections made during their creation by Users. Composition
definition will support the reuse of “standard” couplings between particular (sets of) models. Deploy­
ment definition will support reuse of common deployment strategies. It is anticipated that such scripts
will be used on a personal basis, but will also promote the sharing of “best practice” within the commu­
nity.
The system is depicted in Figure 2 and the following section describes the components in more
detail.
5 System Components
5.1 Models and Model Interfaces
A model will define an interface which describes its provides and uses fields. A “field” is to be thought
of as a basic entity (e.g. an array of fluxes). It is not envisaged that a model would provide access to
the same field in a number of different ways, as is the case in the current UM. This functionality will be
achieved by applying transformation functions to the basic field outside of the model.
Implementors of models may provide any number of implementations which conform to the inter­
face. For example, models may be provided in the form of computer programs (e.g. as sets of FOR­
TRAN subroutines) to support weather and climate prediction or, at the other extreme, a model may be
implemented as a set of files which can be used to provide pre-computed coupling input, for example,
to models under development or models executing in “stand-alone” mode. Model implementations are
described further in Section 5.1.1.
It is important to distinguish between the interface of a model (i.e. the information defining an atmo­
sphere model or an ocean model) and the information capturing the composition of particular models.
This latter, often termed a coupling interface, is a description of how the provides fields (in the interface)
of one model are used, with appropriate transformation functionality, to satisfy the uses fields (in the
interfaces) of other models. In this document, the term interface means the description of the provides
and uses data associated with an individual model.
Both model interfaces and model implementations will be kept in repositories. These will provide
persistence and support configuration management, updating and browsing. Repositories are discussed
in Section 5.1.3.
A Model and Interface Description Language, (MIDL) will support these activities. Declarations
in an object-oriented style will be used to describe the contents of model interfaces. This will include
declaration of the “types” of data which can be included in the interfaces as well as instances used in
specific interfaces (i.e. the actual data fields). The resulting metadata is used during composition to
capture the required model coupling etc.
Interface definitions will include information relating to nature of the fields (e.g. name) and their
data volume (grid size and primitive data type size).
It is anticipated that the definition of the contents of interfaces will require collaboration between
scientists who are specialists in the fields being modelled. It is to be expected that, through such collabo­
rative design, the interface repositories will become populated with standard coupling interfaces (which
7
satisfy couplings between specific pairs, or sets, of models) as they become mature. The existence of
the repository will encourage developers of new models to reuse existing, accepted, interfaces where
possible. However, innovative interfaces may also be developed within the framework.
5.1.1
Model Implementations
Model implementations will consists of a set of source and/or object code. A single model interface may
be implemented in a number of ways. It is also possible that an implementation may satisfy a number of
interfaces. That is, one particular implementation of an atmosphere model may provide the possibility
of implementing the provides and uses fields contained in two interfaces which describe atmosphere
mapping in Figure 2. A more practical view is that users
models. This situation is depicted by the
will focus primarily on the scientific algorithms which constitute a model and consider the model’s
interface to be derived from this3 .
Internally, models may be developed to any standard, for example, in the style suggested by the
W RF project—see Deliverable D3 Review, but, from a coupling perspective this is an implementation
detail at a lower level of concern.
Implementations of models may exploit parallelism (including, for example, mixed mode paral­
lelism to exploit clustered architectures using MPI and OpenMP). Internal details such as this are not
required to be visible in the coupling activity (model and transformation composition). However, the
metadata for models will need to capture this information and the Deployment Environment will be
required to use this information in the allocation of appropriate machine resources and the deployment
of the coupled system upon them. Further, the communication mechanism between models (put()s and
get()s) will have to respect any doomain decomposition in the model implemented to support its parallel
execution.
�����
5.1.2
Model Metadata
Metadata will be required on a per-model basis to support the automatic generation of driver code
etc. Metadata will also be required on a per-field basis, consistent with the contents of the model
interface. This metadata will describe, for example, the units of the field. Such metadata captures
formally much of the information which is currently only held informally, often only in comments in
code. In the future, this metadata will be able to be used to support the integrity checking of proposed
coupling scenarios and the compile-time and, possibly, run-time generation of transformation functions
in appropriate circumstances (for example, where two models keep temperature field in different units).
Standards for metadata formats will clearly be necessary to promote these activities. As discussed in
Deliverable D5 Outline Implementation Plan, in the short term, the aim could be to simply capture
metadata, adding functionality using it over time.
5.1.3
Model and Interface Repositories
The repositories will support the configuration management (versioning etc.) of the instances of inter­
faces and model implementations they contain. The functionality which repositories will present include
the adding, editing and deleting of instances and the browsing of the repository contents. In the short
term, a repository may be implemented as a directory structure with suitable edit and configuration
control mechanisms.
The existence of repositories will support the development of “best practice” coupling interfaces
and the sharing of this information both locally (within the Met Office and Hadley Centre) and with the
wider modelling community.
3
A Computer Science-based view of the relationship between model interfaces and model implementations and its rela­
tionship to the, somewhat more pragmatic, view of model development in the Met Office is discussed in FLUME product, D1
System Architecture, V1.3.
8
The metadata stored with repositories will support the development of more rigorous and automated
checking and composition of models.
5.2 Transformations
5.2.1
Transformer Interfaces
Transformer interfaces capture the description of transformation functionality available to implement
coupling between models. Transformer functions are the “glue” used to link models together.
As with model interfaces and implementations, transformer functionality and implementations which
implement the functionality will be stored separately in their own repositories.
The transformer interfaces will be available for use in the composition environment where cou­
pling solutions are developed. It is anticipated that standard coupling solutions (on a per-model or percoupling field basis) using the transformation functions will be developed and it will be possible to save
these solutions for future reuse from the composition environment. Such solutions will be associated
with the appropriate model interfaces through metadata descriptions.
5.2.2
Transformer Implementations
The transformation functionality available has to be flexible enough to allow models developed by third
parties, and hence developed to no particular standard, to be be incorporated. Transformation functions,
therefore, often represent computation that is not strictly necessary, in that many of them could be
“designed out” if the models to be coupled were developed with integration in mind. This computation
may represent overhead in the execution of the coupled model. As a matter of principle, the amount of
transformation code should be minimised and it is to be expected that within the Met Office environment,
an integrated approach to model development will be taken which will minimise such code.
Some systems impose the idea of a standard grid to which all model’s data must be transformed
during the process of coupling exchange. Such a choice is not required to be made explicit in this design
document. The design is capable of supporting a decision of this kind but does not dictate it. It is
recognised that decisions of this type will have implications both for the quality of science resulting
from the coupled system (arising from the need for interpolation schemes) and also for the performance
of the system (due to extra computation incurred). The former of these issues should be addressed by
the scientists and the latter by implementors of the system but also by scientists during the composition
process.
5.2.3
Transformer Repositories
Repositories for transformer interfaces and implementations will provide the same functions as those
described for models in Section 5.1.
5.3 User Selections
Many models provide configuration options from which users may choose. These options may provide
alternative modelling of certain physical phenomena, for example multiple cloud schemes may be avail­
able, or provide for additional processing within a model—for example, aerosol processing may be able
to be enabled or disabled within an atmosphere model. The selections take the form of “switches” which
control the configuration of optional modelling features. Associated with each switchable element will
be a number of “knobs” which provide parameter values for the code. Clearly, the configuration se­
lections may affect the coupling interface made available by the whole model. Generally, there will be
constraints associated with the configuration selections which protect against the selection of incompat­
ible modelling schemes etc. The constraints are an example of metadata associated with the models. It
should be noted that the issue of the selection of of configuration options is beyond the scope of this
9
document. Further, user selections affecting anything other than coupling are outside the DCD metadata
structures discussed in this document 4 .
5.4 Definition Environment
The definition environment will provide facilities to:
Define and update (add, edit, delete) model interfaces.
�
Define and update transformation functions and interfaces.
�
The environment operates at the level of metadata which describes the content of interfaces. Imple­
mentations of models which satisfy the interfaces will be provided by Users and placed in the appro­
priate repositories. Note that more than one implementation satisfying an interface may be provided, as
mapping in Figure 2.
denoted by the
��� �
5.5 Composition Environment
The composition environment provides control over:
Which models are to be coupled.
�
Which interface elements (provides and uses fields) of models are to be utilised in the coupled
system.
�
What transformation functions are required to satisfy all uses fields given the selected set of pro­
vides fields.
�
The rate of execution of each model.
�
�
The coupling intervals between models.
Composition will take place entirely at the level of model and transformer interfaces. Suitable
implementations of models and transformer functions will be selected in the deployment phase.
A language (the Composition Specification Language, (CSL) will support this activity. CSL may
be based on an existing language, such as Python, Perl or the language used in PALM. Selection of a
specific language is an issue for low level design.
A GUI would naturally fit on top of this scripting language but it is recommended that the “tan­
gling” of composition and deployment issues which is evident in PALM should be improved upon. This
involves resolving certain dependencies between models during deployment, after a composition has
been specified. For example, the PALM interface, described in Deliverable D3 Review, tangles the con­
trol over composition and deployment. An initial task demanded by PALM is to allocate a number of
branches. This determines the level concurrency in the deployed system. In our design, the primary
task (in the sense of being conducted first) emphasises the composition of models and transformation
functionality to achieve the desired scientific objectives. In the following, deployment phase, it will be
possible to deploy the same composed system, including determining the level of concurrency, in several
ways, depending, for example, on the particular resources available. Further, the mixing of composition
information with GUI display information in the same script should be avoided.
CSL will allow composition of models and transformation functions in a data flow style, where the
provides and uses ports of models are connected together via appropriate transformation functionality.
As stated in Section 4, composition specifications and related information, or fragments of them
pertaining to coupling solutions between specific models or fields, may be saved for future reuse in the
repositories.
4
The issues of user selections are discussed further in FLUME product D8, User Interface.
10
Note that a relationship will exist between the various execution rates of the models (timesteps) and
the coupling intervals selected. This relationship is determined by the scientists. Standard relationships
of this form may also be captured in the composition scripts and associated model metadata.
The composition specification, together with the per-model metadata describing execution rates and
coupling intervals, will be processed by a “compiler” to produce information relevant to the deploy­
ment phase. The compiler may, for example, produce source code implementing the required control
structures for the coupled system.
5.6 Deployment Environment
The deployment environment supports:
The definition of which models will execute sequentially and which concurrently.
�
The selection of appropriate implementations (i.e. code) of models and transformation functions
which satisfy the interfaces specified in the composition. In practice, the scientist will primarily
be selecting models to couple rather than interfaces. Because models usually have only a single
implementation, this selection would have been defined in the composition stage. This alternative
approach is suggested as it would allow a more automated approach to be developed for model
selection.
�
The deployment configuration of models and transformation functions into executables.
�
�
The choice of which computational resources are to be assigned to models and transformation
functions.
Support for the specification of concurrent and sequential flows of control will be provided in a
Deployment Specification Language (DSL). The specification of execution rates for models and trans­
formation functions will have been specified during composition. Further, it will be possible for the
User to control the allocation of the functionality of transformation functions. For example, they may
appear explicitly in the control layer in the case where a single executable deployment has been selected
or as a separate executable requiring it’s own “coupling” exchange with models. Clearly, performance
considerations will inform the decision.
Driver control (sequencing) code may be automatically generated from the composition and deploy­
ment specifications, and other metadata, via a “compiler”. This will support ease of maintenance and
ensures consistency of use between environments.
The information defined on a per-model basis in the Deployment phase may be used to integrate im­
plementations of the model and transformation functions (subroutines) with the automatically generated
driver/control layer and with the coupling exchange communication facilities. This activity will result in
a one or more executables. The Deployment Environment will also support the mapping of executables
to machine(s) and the deployment of the coupled system upon them.
6 Layered Software Coupling Architecture
The proposed layered software architecture for the coupling system is shown in Figure 3. This is the
architecture for the deployed coupled system of models not the architecture of the environments used
to compose and deploy the coupled system. This architecture does not make reference to any memory
management functionality which may be required by particular implementations. The aspect of memory
management is the subject of lower levels of design.
The Control layer invokes models at a rate consistent with the coupling intervals defined between
the models which have been composed together in the Composition Environment. The control code will
implement the sequencing of the models, as defined in the Deployment Environment, as a mixture of
11
Control
(including�concurrency�and�parallelism)
Model
wrappers
Inter-model�
Communication
Transformer�function
wrappers
Models
Transformer�
functions
Intra-model
Communication
Figure 3: Layered Architecture
sequential and concurrent invocations. The transformation functionality required to satisfy the coupling
requirements will also be invoked from this level.
In order to reflect the layered architecture in actual implementations, it is proposed that the control
layer invokes wrapped versions of models and transformation functions which encapsulate the coupling
exchange communication (inter-model communication) and calls to their actual implementations. These
concepts are illustrated in the example described in Section 6.1. This choice is consistent with the
requirements of the PALM system—see Deliverable D5 Outline Implementation Plan.
The control code may, for example, be automatically provide by the deployment environment, or the
deployment environment could generate input to a generic control interpreter.
In Figure 3, the intra-model communication which is a consequence of models exploiting parallelism
in their implementations is shown at the bottom of the layered architecture. This reflects the current
implementation choice for the Met Office models where such communication takes place from within a
model. This layer could be moved to the same level as the inter-model communication and this is the
approach favoured in the W RF system (described in Deliverable D3 Review).
6.1 Example
An example of a simple, single field, two way coupling composition of an Ocean and an Atmosphere
model is shown in Figure 4. The figure shows the data flow between the (interfaces) of the components
and indicates how frequently they are invoked.
For illustration purposes, the Ocean model is considered to execute with a 24 hour timestep and the
atmosphere model with an 8 hour time step. The coupling interval is every 24 hours. Summation of the
atmosphere-to-ocean coupling data is required and regridding is required as the Ocean and Atmosphere
models use different resolution grids. The version information indicates that the models have revision
history. This is managed by the repository.
12
Coupling�
interval
Coupling�
interval
Timestep
Regrid
Ocean
Interface
Version�1
Regrid
Sum
Coupling�
interval
Atmosphere
Interface
Version�2
Timestep
Figure 4: Coupling Composition Example
The control structure for this system when deployed as sequentially invoked models in a single
executable is shown in Figure 5. The models, the transformation function to sum the atmosphere data
and the regridding function are executed in a sequential manner, but each may exploit parallelism and
be executed on the same number of processors. The figures represent the “steady state” execution and
ignore details of starting up the coupled simulation. A time-line illustration of the coupling is shown in
Figure 6.
In Figure 5, subfigure (a) shows pseudocode for the top level control structure for the coupled Ocean
and the Atmosphere model system. This code is automatically generated by the system. In this example,
the coupling output from the Atmosphere model takes place every timestep and and is summed by the
transformation function TRANSF SUM(). Regridding of the coupling data is required between calls to
the Ocean and Atmosphere models as is illustrated by the calls to TRANSF REGRID().
Inside the top level call, each model and transformer function performs its coupler exchange (shown,
for illustration, as a call to a single puts() and gets() routine) before and after calls to the model
implementation routines (see subfigures (b), (c) and (d)—the regridding function takes the same form
as the sum function).
Coupling data is passed between models and the transformation functions. The communication
required to enact this coupling data exchange takes place within the top level invocations of the models
and transformation functions. This communication may be implemented in a “message passing” style,
as in PALM, or via argument lists in FORTRAN. The choice of mechanism is a low-level design issue.
More details on communication mechanisms can be found in Section 6.2.
6.2 Inter-model Communication
There are several options for inter-model communication mechanisms to implement the coupling ex­
changes. The choice is a low level design issue. The choices include:
13
WHILE (NOT DONE) {
CALL TRANSF_REGRID(A-to-O)
CALL OCEAN()
CALL TRANSF_REGRID(O-to-A)
DO for N TSTEPS {
CALL ATMOS()
CALL TRANSFORMER_SUM()
}
}
(a)
OCEAN() {
gets()
CALL OCEAN_()
puts()
}
(b)
ATMOS() {
gets()
CALL ATMOS_()
puts()
}
(c)
TRANSF_SUM() {
gets()
CALL SUM_()
puts()
}
(d)
Figure 5: Coupling Control Structure
Arbitrary placement of communications—PALM-like, asynchronous, put and get functions
which may be placed anywhere within a model and which imply a tuple-space model. Put is
typically non-blocking communication implying that the calling code does not wait for a put to
complete before proceeding and get is blocking, so the receiver may have to wait until a sender
puts the requested data. PALM uses a tuple-space model. The tuple-space model is a simple “onesided” protocol and models place put and get calls where they are required in the code. However,
the tuple-space model also requires a garbage collector and/or explicit management of allocation
and deallocation of memory.
�
�
Layered placement of communications—One attractive idea, developed in the W RF project (see
Deliverable D3 Review), is that a model should be implemented as a subroutine and communica­
tion should only occur through its argument list. In this scenario, communication is a higher layer
function, typically being placed in the control layer.
The final choice of communication mechanism is an issue for low level design.
Note that in implementations, blocking receives should time out to prevent deadlock and allow the
reporting of an error. Such an error could result in the “clean” termination of the application, for exam­
ple. The structuring provided by the layered approach provides a better opportunity for automatically
generating and placing communication calls and avoiding deadlock.
7 Summary
This document has provided an overview of a high level design for the future coupling architecture of the
UM. An environment to support the processes of defining, composing and deploying coupled models of
weather and climate functionality together has been presented. The design provides for “best practice”
14
O-to-A
regridding
Atmos
Day�2
Single
Atmos
timesteps
A-to-O
regridding
Control
layer
12�Ocean
timesteps
Ocean
Day�1
Ocean
Day�2
Calls�to�Sum
Transform�
function
Figure 6: Coupling Sequential Timeline Example
and standard couplings to be developed and reused both within the Met Office and Hadley Centre and
within the wider environmental modelling community.
The components of the architecture have been described in some detail and the layered software
architecture to support the design introduced.
The design provides a superset of the functionality of the PALM system developed by Cerfacs. The
use of PALM as a basis for developments within the Met Office leading toward a full implementation of
the design described in this document is contained in the F LUME-related project deliverable D5 Outline
Implementation Plan.
15