The Portable
Parallel Implementation
Biology
Marios
D.
Daphne
of Two Novel Mathematical
Algorithms
in ZPL
Dikaiakos*
Calvin
Manoussaki$
Diana
Abstract
calculations
ical
This paper shows that
mathematical
models of biolog-
parisons
and the Intel
against
Paragon
sequential
that
include
This
ical
hibit
com-
Fortran.
is
one
of
the
fastest
growing
modeling
is inevitable.
involve extensive numerical
these problems
over large computational
amenable
to dat ~parallelism.
of pattern
formation
tions
one
that
els described
that
ods
cannot
are
sity
in
derives
in
dynamics
partial-differential
thus,
understanding
of
the
the
which
been
extensively
their
continuum
FM-2o,
Science
University
and
require
sit y, Dallas,
TX
Applied
of Mathematics,
Mathematics,
Southern
Methodkt
interact
ical
mathematical
observations,
The
The
ents
crete.
of
Univer-
75275.
This
the
365
the
ety
of complex
on
the
solid
In
patterns
densities
an-
and
based
local
on biolog-
creation
of our
new
developed
nism
which
that
they
cells
themselves
on the
cultured,
they
patterns
evidence
for
produce
us to
verify
proliferating
that
geometric
The
numerical
solution
models
provides
a realistic
an array
sublanguage
conditions
form
a vari-
[5, 23].
Based
model
that
cells
is crucial
di-
is re-
chemotaxis,
a cell-chemoattractant
the
ex-
which
gradient,
Depending
are
of
to gradi-
movement,
a concentration
haa enabled
they
in response
spatio-temporal
between
observed
the self-organization
directed
biological
have
ZPL,
.$3.50
up
which
tractant
u ~
motion
to as chemotaxis.
of the
is given
Machine
or to
attractants
under
for profit or commercial
advantage,
the ACM
copyright/server
notice, the title of the publication
and its dote appear,
and notice
that copyright
is by permission
of the Association
for Computing
Inc. (ACM).
To copy otherwise,
to republish,to
post on servers
redistribute
to lists, requires
specific
permission
and/or
fee.
lCS ’95
Barcelona,
Spain o 1995 ACM
0-89791-726-6/95/0007.
the
at different
were
the
tissue.
theories.
dramatic
local
to
the
these
meeting
aggregate
chemically
ferred
teraction
usbeen
within
form
pat-
have
in quantifying
change
have
patterns
theories
work
studies
which
of chemical
Permission to make digital/hard
copies of all or pm-t of this material without fee is granted provided that the copies are not made or distributed
that
Fibroblast
movement
led
algorithm
bacteria,
rects
Univer-
and
of fibrob-
tissue
little
on how
and
in-
model.
second
motile
and
of cells
assumptions
that
structure
connective
fibroblasts
pro-
machines.
the
to validate
arrays
ZPL
algorithms,
of the problems
[11],
culture
parallel
the
to fingerprint
their
First,
mathemat-
these
by biologists.
has been
culture,
Our
long
of the
in topology
within
mod-
University
that
represen-
solving
by using
models
compared
there
equa
of Washington.
Of
been
orientations
of Washing.
Engineering,
biolog-
contributions.
use of parallel
to describe
of many
meth-
behavior.
often
the
patterns
for their
for
many
studied
theorems
gles.
solves
are cells
have
primary
Second,
algorithm
lasts,
equations
numerical
domains
algorithms
widespread
a confluent
in the field
continuum
Moreover,
complex
[15] to implement
ZPL
first
influences
Many of
computa-
systems,
generally
explicitly;
*Department
of Astronomy,
Seattle,
WA 9S195.
t Department
of Computer
t Department
Washington.
$Depmtment
the
However,
and are easily
For example,
biological
are
by nonlinear
be solved
crucial
complex
ton,
domains
very
problems.
that
devised
mathematics.
the increased
use of mathematical
point.
form
two
new
language
we show
terns
biology
makes
two
biology
The
and most exciting applications
of modern
As biology
becomes more quantitative,
data
computational
paper
gramming
ing
tions
large
we describe
Introduction
Mathematical
each
often
tation.
ing a high level data parallel language called ZPL, and
we give performance
results for the Kendall Square Research KSR-2
for
structures
require
ical pattern formation
are ideally suited to data parallelism. We present two new algorithms,
one for simulating the dynamic structure
of fibroblasts,
and the other
for studying
the self-organization
of motile bacteria.
We describe implementations
of these algorithms
us-
1
Lint
E. Woodward$
it
and
to the
we
mecha-
is the
in-
chemoatformation
patterns.
for
the
equations
litmus
test
of the
more
for
of
the
general
these
use
of
Orca
C programming
language
[14].
We show that ZPL
is a convenient
platform
for implementing
these applications,
yielding
clean solutions
and good performance across different
parallel machines.
In part icular, convenience comes from the high level nature of
ZPL that frees the programmer
from the
explicit communication
and synchronization;
sequential
semantics
of the language
allow
portability—porting
ZPL applications
across platforms
only requires recompilation
of the C code that is produced by the ZPL compiler.
The ability
to produce
eficient
portable
code comes from ZPL’s underlying
programming
model [2], a claim that is supported
by
the results presented here on two very different parallel
KSR-2
Concluding
remarks
and
(parallel
there
will
on the
is great
we assume
arrays),
be simply
local
local
that
observations,
best
describe
cell
dif-
densities.
variation
in cell
cell movement
N and
cell forms
the following
et= f
the
we conclude
will
that
the cell distribution
cell orientation
with
a fixed
model
(in
be
the vari-
are the cell
0; 0 is the angle
reference
axis.
non-dimensional
K(X, y, o(E, ~),~(y,t),
that
This
leads
form)
[11]
N(x, O
to
dy –
~(y,t))
B(E)
(1)
V . Jo + random
and
are given in Section
other
movement
depend
where
on these
that
density
This paper is organized
as follows.
Section 2 describes the new algorithm
for simulating
fibroblast
structures.
Section 3 discusses the self-organization
of
motile bacteria and the new model for simulating
this
process. Section 4 provides some basic background
on
ZPL. The next two sections describe the ZPL implementation
of the two algorithms
and give performance
results.
will
Based
pro-
the
Research
fusive
to each
cell
hindered.
gram development
and debugging to proceed on familiar workstation
environments;
and from source level
Square
that
In areas
ables
computers—the
Kendall
the Intel Paragon.
parallel
we assume
orientation,
details of
from the
that
are aligned
where
l?(e)
small
rate
denotes
neighborhood
of change
neighboring
of the
a ball
of radius
of X.
The
in angle
due
to the
cell populations.
equations
c around
kernel
K
angle
In our
X, i.e.,
measures
and
a
the
density
numerical
of
solution
we set
7.
k,N(y)
A’
2
The
2.0.1
Fibroblast
Modeling
Fibroblasts
nective
When
and
in
culture
patterns
a model
tion
and
easily
Fibroblasts
of orientation.
sign
a local
culture.
ical
●
an
axis
The
few
Fibroblast
shape,
cells
to
almost
where
which
and
is based
to
move
level
every
cell.
along
this
we can
points
the
Cells
orient
move,
local
cells move
effects
the
the orientation
particular,
ation
neighbors.
do
not
upon
similar
change
its
Local
its
cells,
to
each
its
of their
own,
orientation
other’s
a cell
and
The
diffusion
orientation
are
neighboring
will
align
tend
itself
dissimilar,
orientation
densities
are
distribution:
– o(y~ll
is, the
slowly
=
so that
to
set
ation.
O.
The
orientation
as they
in
that
move.
We
at each time
step
and
completely
Therefore,
we make
diffusive
flux
of the
by:
= -[F.
D is a function
variation
(~):
biolog-
with
(3)
N(39N[X)+k2
them
time
Jo
is given
JN,
that
with
orient
JN
cannot
a cell
W
v(ND)]i
(4)
the
of discontinuity.
meeting
orientation,
If the
affect
cell
have
cells’
D(Z,
influence
In
carry
approximation
Here,
an
~ =
direction
to
(cos(0),
ae
N)=
N(%)2
sin(e))
of orientation,
of the cell density
is the
a
and the
(5)
+ b
unit
vector
n in
the
and
with
(6)
cells
movement.
is the
.
that
observations:
cells:
(~3”
l\e(x)
of angle,
to
as-
in
on the following
flux
tend
determine
us
each
point
the
they
allows
cells
assume
equa-
in culture
60=
J8 is the
the
described
an orientation
are called
model
arrays
differential
When
a macroscopic
points
assigned
parallel
sin
which determines
the maximum
I is a step function
difference M in angle at which cells may influence each
ot her’s orient at ion.
con-
tissues.
of orientation
motile
At
orientation
be uniquely
[8].
or bipolar
of the
and
has been
of a parabolic
are highly
axis
forming
equation
attribute
organs
interaction
consisting
a mono-
cells
between
interact
This
an integral
also acquire
The
spindle-shaped
space
they
[11].
with
to
long,
the
1(60)
where
Application
Fibroblasts
are
tissue,
=
inj?uenced
In
areas
by the
local
where
all
cell
local
cells
rate
are able
366
of change
orientation
to move).
(the
of orientation
direction
in
along
the
direction
which
the
of
cells
2.1
Numerical
lasts
To
calculate
equation
proximate
ical
the
meshsize
within
such
integral
shown
=
a cell
Ay
will
a choice
is written
for
using
have
is Ax
which
With
(3)
integral
simulations
for
Simulation
of
of
a meshpoint
rule.
the
we apNumer-
a reasonable
d, where
influence
Ax,
(i, j)
Simpson’s
that
=
Fibrob-
choice
d is the
radius
a neighboring
approximation
cell.
to
the
as:
Figure
1: Fibroblast
n x n is the
with
simulation,
to
AZ
At
Ay
66ij,i-lj
=
for the other
IId$,j –O* ~-l,j
three
Il. Similar
definitions
hold
We can thus approximate
terms.
the integro-differential
equation
by a system of nonstiff ordinary
differential
equations for which the time
derivative
can be discretized
using forward Euler:
=
shows
determines
the
much
aligned
paper
correspond
(t=O.
small
The
cell
winding
density
for
the
was
flux
calculated
term
using
J = (Jl,
first
order
up-
the
J2):
randomly
density
that
near
parallel
the point
JNi+
of the point
the
no
other
flux
(i+
three
for
the
The
numerical
a square
grid
methods
value
~ ,j is the
~, j)
along
terms).
cells
or
domain
of size
require
at
0(n2)
of JN
calculated
the x direction
On
their
for
least
flops
the
boundary
upstream
below
the
in real
cell
(similarly
for
ity).
for
resulting
simulations
40 x 40.
each
At
Being
time
consists
explicit,
step,
of
because
continuity.
where
away
367
triradii,
decreased
in cell
model
the
on the
the
due to the
we assume
angles.
our
higher
from
areas
of
themselves,
actual
large
anAs
while
at
orientation
2 we see that
of disconare lower,
with
the
cell
above
the
discontinu-
appear
clumping
movement
angle
are
confluence
densities
predicted
cell
the
average
densities
This
discon-
Triradii
at the points
(curve
cell
some
Figure
cell
cultures,
clumping.
the
large
stage.
at the end of the
From
except
cell
a short
we notice
at this
appear
diffusion
we allow
In
After
as triradii.
arches,
arch
initially
orientations
marked
of different
are uniform
being
we have
of confluence.
t inuit
as observed
where
are
cell
Moreover,
align
which
cell densities
y. Just
densities
are due to the
others
arrays
in this
Results
cell orientations
patterns
pretty
results
decreases.
plot,
and
a
of iter-
iterations.
persistent
further
the
as arches
of three
which
cells
On this
3-point-star
The
to align.
are still
1 shows
produce
discontinuities
Local
of discontinuities
simulation.
to
at each meshpoint.
begin
progresses,
tinuities
cell
grid.
variations
number
dish.
of 50,000
the
1) cells
Figure
the
the
We keep
number
of angle
simulations,
gle variations
time
calculation
The
our
values,
< 0.001.
Simulation
throughout
are assigned
time
For
mesh-points
parameter
if At
number
to runs
parallel
our
mesh.
steps.
t > 100 the cells become
throughout
In
our
the
For
Numerical
the
50,000
between
of iterations.
the
dish.
2.2
uniform
For
solutions
throughout
within
in
dist antes
0.01.
number
ations
after
of points
the
stable
constant
predictable
Here,
number
we set
=
system
cell-orientation
cultures,
variations.
density
higher,
by our
too,
model,
occurs
towards
cells
in our
the
will
Hence,
disstay
they
Densities
Cell
dividuals
[1, 4].
known
The
biological
model
It is the interaction
●
tradant
that
served
patterns.
cells
start
erate,
between
causes
on the
at
sense
substrate.
the
the
produce
only
following
and
center
and
ob-
the
the
dish,
prolif-
chemoattractant;
chemoattractant
cells
the
medium,
of
degrade
chemoat-
into
semi-solid
and
the
The
the cells
self-organization
In
out
and
they
is based
fact:
and
not
chemoattractant
the
are both
diffusive.
We
denote
the
concentration
centration
Figure
2: Fibroblast
cell-density
after
50,000
steps.
the
Therefore,
tration
We
at lower
are
densities
currently
at the
refining
our
center
model
to
address
this
Our
numerical
terns
that
culture
closely
[3].
The
numerically
that
the
show
pat-
our
patterns
interactions.
model
patterns
between
r
Further
indication
biological
TtC
values
●
. J. +
have
terns
agar,
respond
they
themselves
J.
=
-Dn Vn
and
JC =
of
cells
and
the
form
Bacteria
~c)z,
where
taxis,
and ~ the saturation
found
motile
[22]
chemically
in
under
aggregate
(7)
c,s)
–
~3(Tz,
(8)
c,s)
-DCVC
are
the
diffusive
respec-
C)VC
a new
class
random
of
(at
chemotactic
experimentally,
a
measures
high
levels
X(n,
the
flux.
response
The
is the
c) = om/(1
strength
of
+
chemo-
of the chemotactic
sen-
of chemoattractant).
which
ex-
kinetic
c,s)
are proliferation
of
~1 (n, C, s), ~z(n, C, s), f3(n,
cells, and production
and degradation
of chemoat-
tractant.
that
fascinating
motion
movement
●
pat-
on semisolid
attractants
These
sitivity
is the
for the chemotactic
to form
These
inoculated
of chemical
both
directed
chemotactic
complexity.
[5, 23].
are
which
which
species
cells,
excrete
patterns
and
c,s)
chemoattractant,
x(n,
to gradients
terns
discussion):
$2(7L,
JChemO =
of different
when
perimental
It is
terms
– V . JChemO + fl(n,
functional
bacterial
patterns
form
concen-
a parameter.
in non-dimensional
Application
been
of different
geometric
is negligi-
substrate
s is just
a general
–V
one determined
Modeling
strains
the
=
fluxes
for
●
Conditions
that
model
. J.
S. typhimurium
of substrate
thus
–V
the
the
con-
tively.
Bacteria
3.1
In
n,
the
where:
simulations.
3
s.
by
c, and
of
experiments
parameter
the
cells
by
d
and
characteristics
accurate
=
in
biological
strong
important
us with
observed
the
provides
captures
provide
the
similarity
model
cell
of the
resemble
found
our
could
simulations
and
[17] provides
a
problem.
by
we assume
to cast
(Murray
of a t riradius.
(motile)
consumption
is constant,
convenient
appear
of
of substrate
experiments,
ble.
density
of chemoattractant
We
choose
are consistent
the
simplest
with
the
forms
possible
experimental
obser-
vations:
pat-
(diffusion)
(chemotaxis)
are esc, s)
=
pn(l
amenable
to mathematical
analysis,
are the periodic
arrays
of continuous
or perforated
rings generated
by
f,(n,
c,.)
=
~
l+yn
(lo)
the
fs(n,
c, s)
=
–c
(11)
The
simplest
bacterium
Salmonella
diffusion-chemotaxis
mechanism
[18]
components
terns.
This
coupled
ever,
provides
such
continuum
of using
thus,
model
ant-
A
of the
simple
consists
actuations
be shown
in
‘random
for
of
high
the
In
how-
from
to be the
walk’
parameter
p measures
-y the saturation
pat-
of a system
interactions;
can
The
essential
periodic
substrate
‘rules’
most
reaction-
differential
models
behavioral
those
;)
on the Oster-Murray
a description
partial
tract
based
to generate
mathematical
nonlinear
and
typhimurium.
model
required
bacteria-chemoat
result
patterns,
–
(9)
fl(%
sential.
dish,
in-
and
368
levels
the
the rate
of proliferation,
chemoattractant
production
and
(at
of cells).
biological
an initial
with
of the
experiments,
inoculum
no cells
proliferate,
of cells
elsewhere.
spreading
patterns
at the
These
out
radially.
are
center
cells
then
At,
formed
of a petri
the
diffuse
same
time,
they
produce
to aggregate.
tive
It
chemoattractant
is the
destabilizing
coefficient
effect,
(cr),
and
sion of the cells
that
neous
to
in
stabilizing
this
the
them
aggrega-
chemotactic
effects
and of the
the
causes
between
reflected
the
(Dn)
is crucial
which
interplay
of the
diffu-
chemoattractant
formation
of spatially
Solution
of the
(DC),
heteroge-
pattern.
3.2
Numerical
Bacteria
Model
In our
numerical
model,
we focus
tractant
(or
quantity
agar,
dimensional
grid.
equations
ated
scheme.
area
density
the
n. =
+
with
metry.
so we impose
pattern
simulations)
the
flux
takes
of the
tions
which
is 0.005.
was monitored
the
time
sults.
We have
method
little
of the
To examine
IIn
p =
analysis
(n,c)
=
tial
the
that
the non-trivial
biological
condition,
chemotactic
lVith
can be driven
w > CY.rjt
patterns
pattern
rings
response
z
(a)
most
uniform
0.818.
[17].
likely
is low
3 which
pattern
then
steep.
precisely,
cal-
in the soluis lmore
in our
collapse
prevents
a
up
numerical
This
size
at
breaking
the peaks
step
has
forming
) the
chemotactic
—pnz,
Figure
of a (more
&/D,,
and
we
the
co-
11)
periodic
likely
numerical
[7] since
formation
experiments.
the
of singu-
sensitivity
nature
the
in
of the
and
reported
cell-chemoattractant
candidate
found
hloreover,
the
confirmed
the
a likely
patterns
chemotactic
pattern.
bifurcation
S.
Detailed
ty-
as our
coefficient,
response
pattern
gener-
the
observation
(biological)
elsewhere
for
in
by choosing
biological
chernotactic
non-dimensioualized
we fix
that
is
parameter
have
is the
is
suggest
simple
lations
parameters,
unstable
formation
a is
concen-
a,
that
it
that
affects
numerical
simu-
sequences,
as well
parameter
values,
as
are
[23].
~ = 1.0, ~ = 0.2,
are generated
are
values
of the
than
(10)-(
bifurcation
be-
chrmotaxis
when
recruitment
the
and
because
sharp
in
wit h rings
ratio
fail
very
therefore
phirnuraum
the
there
of bacteria,
with
these
{L1.
methods.
of the
self-organization
= 0.3, together
shows
when
equations,
larger
term,
mechanism
equa-
However,
implicit
role
and
differential
the
steps,
one another,
limitation
removal
3.0,
outward,
of the
rather
ating
et
time
For even
became
We
re-
by Scribner
rings
for
s is
larities.
halving
of bacteria,
model
these
1.0.
spatial
and
w =
from
values
be the
cell
for
300,000
ultimately
scheme
method
similar
equations.
of the
301
after
distance
tions
than
when
substrate).
in chemotactic
to be so steep
sequentially
culations
integra-
need
of the
likely
gradients
is illustrated,
how,
large
to
be more
that
(or,
consumption
corre-
production
clusters.
spots.
for
Crank-Nicholson
partial
s =
way that
for
bands
and
(1,1/1.2)
size
used
the
explicitly
CY, in the
turbations
mesh
into
shape
at least
in the
numerical
an implicit
in using
so the
qualitatively
differential
= 0.1, and D.
().()3,
the
produced
stiffness
boundary,
used
do not
into
fixed
the
edge of
containing
step
and
is the
300,000
also
to slow
of chemoattractant
will
large
tration
spread
leading
nor
y of this
traveling
or no gain
efficient,
time
to reduce
to ordinary
cause
domain
was successfully
of lines
the
relevant,
doubling
[20] to investigate
tions
before
the
also tested
which
method
well
after
This
problem
to rapid
spots
This
synl-
However,
reliability
which
the
conditions.
is a square
by
step,
break
consumption
shows
experiments
The
The
to
cell-density
to propagate).
dimensional
of cells
c
both
are not
points.
Initially
perturbed
boundary
reaches
mesh,
x 301 grid
order
cell
patterns
constant,
However,
to a dish,
place
conditions
which
for
in the
sufficiently
a 10 x 10 mesh
are then
the
non-
of a dish,
in
value
or rapid
dif-
are gener-
center
elsewhere.
cells
cal
sponds
model
finite
are confined
(in
perturbation
boundary
in
the
zero
0.0
is
of bacteria
steps.
of
and
the
3: Patterns
time
on a two-
Euler
domain
conditions
noise
formation
density
conditions
n =
Experimentally
cell
patterns
of the
These
there
thickness
analyze
at the
initial
random
l%
and
experimental
center
1.0,
the
forward
inoculum
as our
at
since
Figure
is the
the
can be solved
develop
The
O everywhere.
with
for
an explicit
an initial
so we choose
=
To
we use
from
point
(10)-(11)
chemoat-
as this
through
concentration
of the
than
Moreover,
patterns
equations
chemoattractant
ference
interest.
the
behavior
(rather
concentration)
of primary
in
of the
density
substrate)
no variation
the
investigations
on cell
(but
linear
steady
state
by spatial
per-
This
is the
in most
For
4
usual
ZPL
models
The
is a data
subarrays
a given
ini-
language
when
the
grams
to form
above
the
indexing.
criti-
369
ZPL
Array
parallel
language
to be manipulated
provides
while
language’s
that
as whole
constructs
eliminating
The
Sublanguage
that
tedious
allows
entities
lead
and
conciseness
arrays
and
[15].
The
to concise
error
was
pro-
prone
array
first
illus-
trated
by
the
benchmark,
coded
which
in
and
SIMPLE
In
mant
the
ficiency
ZPL
more
considerably
development
and
of ZPL
is to provide
diverse
parallel
produces
code,
Iinked
with
and
produce
executable
hand-coded,
strated
is
on both
computers
[16].
is of high
code
We
and
now
types.
applications,
[15, 21].
real,
integer,
sion,
the
operators
(e.g.
ZPL
sic units
the
(ZPL
dense
managed
of variables
all
arrays,
compiler
some
distinguishing
central
to
simply
sets
Line
scope
declared,
of
this
but
)
can
be used
case U and V, whose
point
can
be used
and
South
define
R, and
to define
values.
to
region
offset
of
Finally,
line
from
RI
R by
22 shows
In
array
x (i.
the
array
vector
Line
(Q).
22 also
For example,
whose
South.
as a “shift”
set
line
Temp whose
.n).
operator
parallel
by
can be viewed
program
indices
Thus,
are
the
@
operator.
Bacteria;
-beta
R = [1. .n,
i..
North
=
[-1,
01;
south
=
[+1,
o];
direct
ion
10
procedure
11
var
Bacteriao
: integer
= 2.0;
nl;
;
U,
V:
Temp:
[R]
double;
[R]
double;
--
Initialization
u
:=
0.0;
u
:=
0.0;
--
begin
16
[R]
scans).
17
[South
of
R]
across
18
[Morth
of
RI
[R]
Temp
19
20
concurrency
the
--
constant
array
system,
Declarations
region
15
distributed
21
is
22
shield-
tedious
:=
(U@South/
( ( 1 + beta* V@South)
(1 + beta*VWouth)
23
de-
*
)) ;
end;
24
regions
which
5
how
are
Implementing
this
not
are beyond
with
be
the
arrays,
in
described
in Sections
In
18 show
tion
direction
is a keyword
which
and
(n+l
)s~ row
of the
how
regions
are used
ized
and
for
the
ZPL
That
is,
ria
models
index
ble
space.
to specify
1) than
step
of the
programs
Implementing
370
time
for
to be more
the
original
the
for
same
applications
users
3 are both
inherently
by
the
by diffusion
can
be
readable
(and
step.
and
formacell
from
Fortran
applications
with
denlocal-
We found
the
shorter—see
sequential
orien-
information
in which
time
Fibroblast
data
cell
bacteria
computed
previous
the
lim-
algorithms
the
determined
Similarly,
with
The
simulation
are
cells.
information
to
2 and
fibroblast
are caused
one
biology
even
experience.
densities
patterns
sities
together
the
neighboring
adjacent
South.
Issues
mathematical
to be easy,
parallel-programming
tations
regions
proved
parallel.
from
of
at ion
the
ZPL
ited
a region
how
Implement
are
are double-precision
regions:
vector
In
need
of parallel
is disjoint
of
illustrates
First,
11 shows
17 and
the
and
(Regions
storage
new
that
is the
At
12
is replicated.
regions,
base-types
Lines
specify
the
and
regions
Line
is a user-defined
a new
[South
the
from
be declared.
set.
dynamic
this
floating
can
index
discussion.
con-
is characteristic
how
indices,
2D
to the
.n)
5
14
codes
of ZPL.
5 shows
of array
case R is a square
statically
below
bacteria
features
ZPL.
use of the
[R]
of the
(I.
index
Thus,
synchronization.
fragment
and
elements
the
be applied.
4
as ba-
All
from
applications
9
prefix
serve
are not
)
of the
R specifies
is to
8
which
and scalars.
which
and
code
from
that
data
parallelism.
fibroblast
refers
displaced
13
scalar
programmer
ZPL
U@South
7
dimen-
are distributed
application
the
to all
the
recur-
(reductions
of communication
The
assign
are in the range
illustrates
region
statement
22 will
including
arrays
arrays
kernels
6
(e.g.
parallel
is derived
while
types
and the standard
parallel
parallel
two
elsewhere
operators
run-time
both
197 lines
3
features
these
of arbitrary
and
the
tails
224 lines
the
the
the
indices
1
constructs
described
etc.),
operators
by the
operations:
which
operator
compiled
in
or arrays,
while,
classes
produce
sizes for
255 lines
applications
that
logical
and scan),
processors,
do not
parallel
set of data
scalars
also has indexed
and
memory
powerful
arrays
concurrency
ZPL’S
demon-
needed
but
and
for,
parallel-prefix
physical
ing
(if,
has two
Specifically,
tar-
matching
ZPL
more
a typical
reduce
model,
C
the
has been
main
not
here
of computation:
ZPL
and
were
to either
structures
sion.
of the
arithmetic
can be applied
trol
Some
char),
usual
investigation.
Bacteria
461 lines
2
which
supports
1: Code
under
over
libraries
biology
the
are omitted
ZPL
on
evidence
describe
language,
Table
array
quality.
briefly
data
of the
ef-
current
ANSI
compiled
and nonshared
further
ZPL
and
Our
Performance
mathematical
se-
ran
process.
portability
C code
The
here
ZPL
machine-specific
provide
described
in
independent
then
parallel
shared
ZPL
simplifies
computers.
code.
explicitly
lines
sequential
debugging
machine
which
500
coded
which
across
machine
when
cent rol flow,
compiler
get
readable
ZPL
when
Fort
lines
only
have
goal
as object
but
Fibroblasts
Application
dynamics
5000
passing,
programs
program
A chief
to
message
addition,
ics and
fluid
is approximately
C plus
considerably
[16].
computational
BacteTa-
programs.
explicit
par-
/* declare
h:
The
actual
of
code consists
of the periodic
semantics
h
as
scalar
*/
double;
of a high-level
boundary
of the
implementation
conditions
Fortran
code,
and
the
applied
to
sequential
parallel
ar-
rays:
Figure
4:
for
Fibroblast
the
Data-dependences
and
boundary
conditions
[North
of
R]
[South
of
RI
wrap
thy
[R]
example.
theta;
:=
angle.
subtr(theta@South,
thetaQNorth)
/(2*h)
allelism
using
constructs
either
message-passing
would
conciseness
have
of ZPL
been
or shared
much
is attributed
more
memory
tedious.
mainly
The
last
ZPL:
The
to:
was
implicit
handling
munication
2. The
and
provision
of specific
of boundary
3. The
known
decomposition,
com-
support
for common
types
conditions;
provision
of
to have
such
of data
synchronization;
as
high-level
efficient
array
constructs
parallel
operations
that
are
implementations,
and
Reductions
boundary
cell-angles
the
do
conditions
and then
78
j=l,
cent
type
re-use
from
trates
how
parallel
to the same
status
(2*h
expression
of the
rays
statement.)
in the
= theta(n+2,
j)
the
11
i=2,
12
[9],
DEC-Alpha
and
KSR-2
j)
j)
subtr(theta(i-l,
=
, theta(i+l,
j))
/(2
12
cent
inue
11
cent
inue
A
of
description
and
ure 4. The
ZPL
the
of the
parallel
is being
carried
putation
the
boundary
corresponding
conditions
implementation
out,
upon
and
for
definitions
which
the
ried
the
dtrecttons
region
R=
North
[1. .n,l.
South
declare
theta,
based
uses the
in our
of
availability
of workstations
environment.
study
AXP’S
*/
371
the
133 MHz
measures
times
an assessment
numbers
of the
of
sequential
scalability
processors
PVM
comput-
protocols
makes
Our
on two
corresponding
is car-
of workstations.
execution
and
processors
distributed
of the
their
micropro-
network.
focuses
and
as the
double;
with
a hierarsys-
communication
development
Our
through
a 50 MHz
a networks
tive
of each
con-
for
high
3000/400
modules
phys-
is a message-lpassing
i860XP,
TCP/IP
KSR
with
sequential
interface
case,
rings.
The
hardware-managed
interconnection
passing
processors
space
enforces
between
a mesh
[13].
Archi-
packetized
is provided
Intel
the
PVM
custom
large
which
Communication
a comparison
arrays
ar-
Paragon
Memory
Memory
Paragon
the
connect,
;
[R]
on
ing that
.nl;
:
Intel
through
DEC
parallel
thy
The
running
Only
address
of a very
scheme
Intel
32 processors.
a shared
coherence
ory
for
var
/*
other
we performed
[6], the
40 MHz
memory,
com-
= [-1 ,01 ;
= [+1 ,0]
scalar
an array
platforms:
of slotted
contains
role
Cache
is a message
Fig-
dat~flow:
direction
ring
the
sistency.
data-
is given-in
includes
arrays
(The
to
codes
(Cache
with
provides
direct
cessor.
pictorial
region
. OdO*h)
tem
dependencies
illus-
elevated
as the
different
workstations
distributed
chical
ZPL
KSR-2
as a hierarchy
play
cache.
j)
Research
leaf-level
node
of the
multiprocessor
ically
n+l
angle_
types.
shape
also
been
promoted
on three
is a COMA
configured
n+l
thy(i,
is
in allowing
and
have
data
and
here
Results
of runs
Square
Each
j=2,
ZPL
trivial
size
efficiency
Kendall
tecture)
inue
do
same
but
programs,
in
of
aa parameters.
be significant
as the scalar
) is also
number
architecture
do
can
Performance
a large
j)
= theta(2,
parameters
sequential
arrays
feature
) function
theta@North
of promotion
expression
convenient
angle~ubtr(
scalar
and
code
To study
on the two-dimensional
array
of
calculates
angle derivatives
along
n+2
j)
accept
This
The
theta(n+2,
78
excerpt
periodic
y-axis:
theta(l,
to
The
thetaWcmth
6
a short
Fortran
that initializes
another
and
Scans.
As an example,
we present
from the Fibroblast
application
illustrates
promotion.
written
passed
1. The
line
scalar
;
this
to
The
an attrac-
workstations
are
clocks.
of performance:
for
the
Fortran
of the
is increased.
ZPL
codes
programs,
ZPL
The
codes
first
I
Fibroblasts
I Intel
I KSR
Fortran
.
2114.75
ZPL
3159.7
2914.17
—
Bacteria
I
I Fortran
I ZPL
Table
2:
times
metric
] KSR
I Intel
I1 20198.88
of
(FmgOIo
g
(RmgoIl)
Sqlm!hl
Fauml
C5&
I I 1675.54
Paraaon
I DEC
-
I 18946.03
Comparison
provides
ated
ZPL
Fortran
a measure
compiler.
by the
keeping
extra
ZPL
and
that
compiler
I _
1
I
I 9754.75
I
ZPL
execution
efficiency
the
code
of the
to exploit
however,
must
than
extra
not
be large
Fhroblast ZPL Code 2000 time steps, 100x1CKIgrid
the
book-
parallelism.
an alternative
Number of Rocessar$
gener-
Figure
This
if this
5: Running
(1 OOX1OO grid,
2 presents
last
and
the
KSR-2,
the
Bacteria
t ures.
Paragon,
ZPL
codes
under
PVMl.
Fibroblast
2000
ported
Bacteria
for
the
a 481
x 481
that
achieve
KSR-2
- huelPmgml
us-qmlid
I%m’mcO&(vamg.m)
codes
performance
4mm
worksta-
running
~
size of
times
E
GZmoo
re-
to runs
Table
on
processor
one
is competitive
3moo
for
time-steps.
running
that
Fort ran
of
reported
correspond
2000
with
2
that
10WO
of
o
programs.
2
1
Numbzr
Figures
5 and
instances
tions
on the
From
these
the ZPL
able
KSFt-2
plots
codes
speedup
about
in Figure
the
speedups
than
investigate
of the
monitoring
environment
instrument
the
compiler.
ment
[12]
In
the
to
program
visualize
diagrams
because
used
collect
performance.
showing
] We have
Alpha
and
we
no measurements
we have
XZpd
traces,
We
parallelism
the
of Fortran
compiler
[10]
spdlQ
--o. - BIuAcs.3c(kddFMwm)
--a-- Fikdiauc&(blPm8aI)
– e- BxxaiaCdc(lCSii.2)
in
more
perfor-
the
to
ZPL
instru-
Paragraph
for
for thk.
Bacteria
the
o~
detailed
processor
codes
Lin.
of the
iterations).
Paragon
by
and
Times
2000
Bacte-
the
produced
profiles,
no Fortran
the
Intel
produced
Running
grid,
code on
plots
to learn
we used
6:
—
is de-
with
for
and
16
number
the
achieved
on
C code
particular,
From
codes,
[19]
speedup
those
(481x481
of avail-
of the same
this
Figure
the relative
on a given
time
machine.
mance
to
7. Here,
time
8
32
Bacteria ZPL Code 2000 dme steps, 481x481 grid
applications
the number
same
are smaller
behavior
both
4
of Pmsewors
applica-
multiprocessors.
for
with
the running
To further
the
well
for the same
Bacteria
Paragon
of the running
of the
code
code.
Intel
times
the
can also be seen from
7 we see that
Fibroblast
and
very
shown
over
one processor
Figure
and
This
as the ratio
of processors
running
we can see that
scale
curves
the
Fibroblast
processors.
fined
ria
6 show
of the
application
Fibrob-
to a mesh
The
Fibroblast
architec-
times
correspond
and
ZPL
hand-written
DEC-Alpha
application
mesh
the
the
of the
iterations).
w
processor
DEC-Alpha
time-steps.
the
one
running
application
of
on
and
on
The
100 x 100 and
times
running
Intel
The
run
running
codes
Times
2000
par-
to sequential
ations.
Table
shows
because
is to represent
implement
sequential
expect
to be less efficient
code
is introduced
solution
tion
of the
We would
Fortran
overhead,
allel
the
-
I# 46919.16
I 47519.79
hand-written
on
I DEC
on one processor.
of the
the
Paraaon
m KSR-2
-Ir,el PuagOn
0 Sequdal mmul m
the
Number
of ITOCmm
uti-
speedups
DEC-
Figure
machine.
372
7: Relative
Speedup.
application
The
performance
Paragon
Fortran
8
.m. ..
Bra
Em
,—
m!.,m,.
-
,-
—
of the two codes
were
found
on the KSR, and Intel
to be competitive
with
sequential
programs.
Acknowledgments
It is a pleasure
to thank
for
with
their
help
Brad
ZPL.
Chamberlain
Julian
and Yi
Cook
Sun
first
suggested
and
developed
:
the
:
f
,
problem
of
a preliminary
:
tribution
.
would
:
modeling
model
is gratefully
also like
Fibroblasts
with
D.
Manoussaki.
acknowledged.
to thank
the
His
D.E.
Department
con-
Woodward
of Mathemat-
,-
l,,.
=.
,.,
.
WW’
mm
‘“
“
“
“
“
ics,
“
University
during
an
Figure
8:
Average
broblast
(top)
ning
Intel
on
and
the
of processors
Bacteria
Paragon.
the percentage
wait ing for
utilization
The
of running
communicant
codes
that
the
(bottom)
dark-grey
time
for
denotes
a processor
spends
files,
communication
and
overhead,
patterns
of
confirmed
that
lower
Fibroblast
application
cation
the
overhead
example,
average
Paragon.
This
exchanges
within
are due to the
23,870
messages
code
runs.
a small
of 2, 693, 600 bytes)
within
Alt.
Biased
and
Biol.,
of
bytes)
whereas
the
messages
a 14.33-second
[3]
(total
els of pattern
to
large
formation
scale
sented
new
biology
of fibroblast
available,
the
that
the
ZPL
able
in
of expressing
to
in the
complex
these
aspect
language
mathematical
and
to
chemoJ.
Bard.
Press,
Calvin
Lin,
Abstractions
programming.
Tech-
of Computer
University
IEEE
Systems,
Jonathan
H.C.
Berg.
E.O.
of Washing-
Trans.
on
Parallel
and
1993.
Morphogenesis.
Cambridge
Uni-
1991.
Random
basic
[7]
on
data
H.C.
cells
Biology.
Berg.
Princeton
edition,
Complex
of Escherichia
1993].
patterns
coli.
Nature,
1991.
Henry
Burkhardt.
system.
Overview
Technical
Square
S. Childress
and
J .K.
Math.
of the
Report
Research,
of chemotaxis.
mech-
[8]
as
J.
Cook
for
developmental
inary
shown
KSR1
February
Perkus.
1992.
Nonlinear
Biosci.,
com-
KSR-TR-9202001,
aspects
56:217-237,
1981.
and
Fibroblast
Ideas.
Mathematics,
D.
Manoussaki.
Pattern
Continuum
Formation:
Technical
report,
University
Some
Dept.
of Washington,
Model
Prelim-
of Applied
1994.
an ideal
models,
an ease-of-use
and
in
[Expanded
are
serve
we have
provides
walks
1983
by motile
Kendall
of the
focusing
should
Press,
Budrene
puter
formation
study
By
develop
Furthermore,
[6]
pre-
mathematical
of the
which
for
suited
have
experimental
more
programming
a performance
other
formation
systems.
two
bacteria.
models
Department
Engineering,
349:630-633,
mod-
ideally
We
study
detailed
been
paradigms
behavioral
means
have
are
solving
the
motile
where
pattern
and
from
of
we
for
important
in
structures,
applications
anisms
for
one
self-organization
biology
ARPA
monitoring
mathematical
programming.
algorithms
problems,
these
in
parallel
and
Snyder.
parallel
93–12–09,
and
formed
that
Dept.,
Griswold,
and Lawrence
submitted
versity
[5]
shown
the
1980.
scalabIe
Distributed
Conclusions
we have
by
from
approximations.
William
Notkin,
University
paper
partly
a grant
walk
diffusion
9:147-177,
Report
ton,
[4]
In this
NASA,
random
Alverson,
Science
of lntel’s
of 4, 138,240
Gail
nical
window.
7
hospitality
supported
Zoology
related
for portable,
of
application
window,
2,844
W.
David
For
number
partition
Fibroblast
only
[2]
communi-
in terms
(total
exchanges
higher
overhead
monitoring
from
of Washington
Math.
profiles
Fibroblast
of the
grant
was
kind
case of the
this
during
instance
the
the
on a 16-processor
a 3.59-second
Bacteria
by
utilization
running
in
work
its
NOO014-92-J-1824.
taxis
pro-
These
speedups
8 illustrates
processor
iterations
communication.
incurred
Figure
communication
for
References
ion.
[1]
lization,
This
visit.
HPCC/ESS
Grant
run-
color
her
University
Fi-
of Washington,
both
[9]
aspect.
Intel
Corporation.
overview.
373
Technical
Paragon
report,
1992.
XP/S
product
[10]
Intel
[11]
Guide,
Tom
Elsdale
ture
and
planar
tal
[12]
[14]
June
and
Heath
Etheridge.
September
Oak
National
Ridge
and Reference
Calvin
Lin
and
Programming,
Calvin
and
pages
96-114.
Calvin
Lin
Languages
Snyder.
Uptal
and
David
for
and
An
array
David
Gelern-
Padua,
editors,
Parallel
Computing,
Snyder.
ZPL.
In
Compilers
1991.
1993.
Lawrence
in
im-
Journa/
ZPL:
Banerjee,
Springer-Verlag,
results
A portable
20(5):363401,
Compilers
and
formance
3 User’s
1993.
International
Nicolau,
and
May
Snyder.
Lawrence
Alexandru
the
Software,
PVM
Manual,
of Parallel
Languages
Visualizing
IEEE
1991.
Lawrence
In
of Developmen-
Programs.
of SIMPLE.
Lin
of two
1976.
Laboratory.
Guide
sublanguage.
[16]
J.A.
of Parallel
Cul-
topology
Archives
121-147,
and
Tools
Fibroblast
The
pages
8(5):29-39,
ter,
Wasoff.
ROUX’s
180,
plementation
[15]
Frances
patterns.
Biology
M.T,
Application
1994.
Dermatoglyphics:
Performance
[13]
Paragon’s
Corporation.
User’s
for
SIMPLE
?h
per-
Workshop
Parallel
on
Computing,
1994.
[17]
J.D.
Murray.
Verlag,
[18]
G.F.
Oster
models
B.
Ries,
E.
Richards,
J.D.
R.
T.L.
Anderson,
and
Scribner
Biol.,
Segel
and
Exp.
Callaghan,
Paragon
In
E.H.
Per-
Supercom-
Rodgers.
and
of chemotactic
A nu-
propagation
bacteria.
of
J. theor.
1974.
Lawrence
Snyder.
Report
A
ZPL
programming
94-12–02,
Science
and
Department
Engineering,
guide,
of Com-
University
of
1994.
Wimpenny.
in space
J.
1994.
of the formation
Technical
[22] J. W.T.
formation
K.
The
Environment.
56:217-237,
puter
Auld,
Smith.
850-859,
bands
Washington,
Pattern
constraints.
W.
W.
L.A.
study
traveling
al
Springer1993].
1989.
Monitoring
merical
edition
Murray.
development
’94, pages
puting
[21]
and
Biology.
corrected
251:186-202,
formance
[20]
[2nd
and
200/.,
[19]
Mathematical
1989
and
Microbial
time.
Adv.
Micro.
systems:
Ecol.,
patterns
12:469–522,
1992.
[23]
D.E.
Woodward,
J .D.
Murray,
Spatio-temporal
typhimurium.
R.
E.O.
Tyson,
M.R.
Budrene,
patterns
Biophysicai
and
generated
J.,
1994
Myerscough,
H.C.
Berg.
by Salmonella
(submitted).
374