MPI
Adriano Cruz ©2003
NCE/UFRJ e IM/UFRJ
Summary







References
Introduction
Point-to-point communication
Collective communication
Groups and Communicators
MPI Implementations
Comparing MPI and PVM
@2001 Adriano Cruz
NCE e IM - UFRJ
MPI
References
Resources



www.mpi-forum.org
William Gropp, Ewing Lusk, Anthony
Skjellum. Using MPI – Portable Parallel
Programming with the Message-Passing
Interface. The MIT Press
Marc Snir, Steve Otto, Steven HussLederman, David Walker, Jack Dongarra.
MPI The Complete Reference. The MIT
Press. Available by ftp from ftp.netlib.org (cd
utk/papers/mpi-book; get mpi-book.ps; quit)
@2001 Adriano Cruz
NCE e IM - UFRJ
MPI
Introduction
Message-Passing Systems




PICL
PVM
Chamaleon
Express
@2001 Adriano Cruz
NCE e IM - UFRJ
History



April 1992 - The Center for Research in
Parallel Computation sponsored a
workshop in Distributed Environment
November 1992 - At Supercomputing
conference a committee was formed to
define a message-passing standard
1994 - MPI Standard completed
@2001 Adriano Cruz
NCE e IM - UFRJ
MPI Forum goals



Define a portable standard for message
passing. It would not be an official,
ANSI-like standard
Operate on a completely open way.
Anyone would be free to join the
discussions.
Be finished in one year
@2001 Adriano Cruz
NCE e IM - UFRJ
First ones



Parallel computer vendors: IBM, Intel,
Cray, NEC, Convex, etc
Portable libraries creators: PVM, p4,
Chamaleon, Express, etc.
A Number of parallel application
specialists
@2001 Adriano Cruz
NCE e IM - UFRJ
Primary Goal

The primary goal of the MPI
specification is to demonstrate that that
users need not to compromise among
efficiency, portability and functionality.
@2001 Adriano Cruz
NCE e IM - UFRJ
What is not new?

MPI is a library.

MPI is an attempt to collect the best
features of many message-passing
systems.

A computation is a collection of
processes communicating using
messages
@2001 Adriano Cruz
NCE e IM - UFRJ
Basic message concepts - send

send(address, lenght, destination,
tag)

address is the beginning of the buffer
containing the data to be sent

length is the length, in bytes, of the message

destination is the process identifier of the
process to which this message is sent

tag is an arbitrary nonnegative integer to
restrict receipt of the message
@2001 Adriano Cruz
NCE e IM - UFRJ
System requirements

Supply queuing capabilities so that a
receive operation such as
recv(address, maxlen, source,
tag, actlen);

will complete successfully only if a
message is received with a correct tag.
@2001 Adriano Cruz
NCE e IM - UFRJ
Receive

recv(address, maxlen, source,
tag, actlen);

address and maxlen define the
receiving buffer.

actlen is the actual number of bytes
received
@2001 Adriano Cruz
NCE e IM - UFRJ
Message Buffers Problems



The (address, length) specification was
good match for early hardware
In many situations the message is not
contiguous
In many heterogeneous systems the
data types vary
@2001 Adriano Cruz
NCE e IM - UFRJ
MPI solution



MPI specifies messages at a higher
level and in a more flexible way
MPI messages are defined as a triple
(address, cont, datatype)
MPI provides functions that give users
the power to construct their own
datatypes
@2001 Adriano Cruz
NCE e IM - UFRJ
Separating Families of Messages



Early systems provides a tag argument
so that users can deal with messages in
an organized way.
The problem is that users have to agree
to define tags in a predefined way
Libraries are a problem
@2001 Adriano Cruz
NCE e IM - UFRJ
MPI solution




Instead of tags use the concept of
context
Contexts are allocated at run time by
the system
No wild card is allowed
The message tag with wild cards
concept is retained
@2001 Adriano Cruz
NCE e IM - UFRJ
Naming Processes



Processes belong to groups
Processes are identified by ranks (0..n1), n is the number of processes.
There is an initial group to which all
process belong
@2001 Adriano Cruz
NCE e IM - UFRJ
Communicators


Context and groups are combined into
a object called communicator
The destination or source always refer
to the rank of the process in the group
identified with the given communicator
@2001 Adriano Cruz
NCE e IM - UFRJ
MPI send





MPI_Send (buf, count, datatype,
dest, tag, comm)
count occurrences of datatype starting at
buffer
Dest is the rank of the destination in the
group associated with the communicator
comm
tag is the usual tag
comm identifies a group of processes and a
communication context
@2001 Adriano Cruz
NCE e IM - UFRJ
MPI receive

MPI_Recv(buf, count, datatype, source,
tag, comm, status);
@2001 Adriano Cruz
NCE e IM - UFRJ
Other features



Collective Communications: broadcast,
scatter and gather
Collective Computations: maximum,
minimum, sum ,logical, etc
Mechanism for creating
@2001 Adriano Cruz
NCE e IM - UFRJ