CHAPTER 12
Parallel and
Distributed Algorithms
Algorithm 12.1.1 Maximum
This algorithm returns the maximum value in array a[0], ... , a[n - 1]. The
value of a.length is the number of elements in the array.
Input Parameters: a
Output Parameters: None
maximum(a) {
n = a.length
current_max = a[0]
for i = 1 to n - 1
if (a[i] > current_max)
current_max = a[i]
return current_max
}
Algorithm 12.2.2 Parallel Search
Given an array a of size n and a number x, this algorithm searches for x
in a and returns true if there is an i such that a[i] = x and false otherwise.
Input Parameters: a,x
Output Parameters: None
parallel_search(a,x) {
n = a.length
found = false
for i = 0 to n - 1 in parallel
if (a[i] == x)
found = true
return found
}
Algorithm 12.2.4 Locate
Given an array a and a value x, the algorithm returns an index i such that
a[i] = x if there is one, and -1 otherwise.
Input Parameters: a,x
Output Parameters: None
locate(a,x) {
n = a.length
location = -1
found = false
for i = 0 to n - 1 in parallel
if (a[i] == x)
location = i
return location
}
Cache Coherence Problem
What value does a[i] contain, after performing this code?
n = a.length
x = 0
for i = 0 to n in parallel
if (i == n)
x = 1
else
a[i] = x
Algorithm 12.2.5 ER Broadcast
This algorithm takes as input a value x and an array a and assigns x to all
elements of the array.
Input Parameters: a,x
Output Parameters: None
er_broadcast(a,x) {
n = a.length
a[0] = x
for i = 0 to lg n - 1
for j = 0 to 2i - 1 in parallel
if (2i + j < n)
a[2i + j ] = a[j]
}
Algorithm 12.2.11 Sequential -Sum
n 1
This algorithm computes  i  0 a[i ] given an array a as input.
Input Parameters: a,x
Output Parameters: None
sequential__sum(a) {
n = a.length
current_sum = a[0]
for i = 1 to n - 1
current_sum = current_sum  a[i]
return current_sum
}
Algorithm 12.2.12 Parallel -Sum
n 1
This EREW algorithm computes  i  0 a[i ] given an array a as input.
Input Parameters: a,x
Output Parameters: None
parallel__sum(a) {
n = a.length
for i = 0 to lg n - 1
for j = 0 to n/2i+1 - 1 in parallel
if (j * 2i+1 + 2i < n)
a[j * 2i+1 + 2i] =
a[j * 2i+1 + 2i]  a[j * 2i+1 + 2i]
return a[0]
}
Algorithm 12.2.16 Optimal -Sum
n 1
This algorithm computes  i  0 a[i ] given an array a as input.
Input Parameters: a
Output Parameters: None
optimal__sum(a) {
n = a.length
blocksize = 2lg lg n
// first, compute  sum of all blocks of
// size blocksize ≈ lg n
for i = 0 to n/blocksize in parallel {
base = i * blocksize
for j = 1 to blocksize - 1
if (base + j < n)
a[base] = a[base]  a[base + j]
}
// second, compute the  sum of the remaining
// n/blocksize elements
for i = lg blocksize to lg n - 1
for j = 0 to n/2i+1 - 1 in parallel
if (j * 2i+1 + 2i < n)
a[j * 2i+1 + 2i] =
a[j * 2i+1 + 2i]  a[j * 2i+1 + 2i]
return a[0]
}
Algorithm 12.2.20 Optimal ER
Broadcast
This algorithm takes as input a value x and an array a and assigns x to all
elements of the array.
Input Parameters: a,x
Output Parameters: None
optimal_er_broadcast(a,x) {
n = a.length
n = n/lg n
a[0] = x
// Use er_broadcast to fill the first n’ cells of a with x
for i = 0 to lg n’ - 1
for j = 0 to 2i - 1 in parallel
if (2i + j < n’)
a[2i + j] = a[j]
// Spread the contents using n processors
// each performing the sequential algorithm
for i = 1 to lg n
for j = 0 to n’ - 1 in parallel
if (i × n’ + j < n)
a[i × n’ + j] = a[j]
}
Algorithm 12.2.25 Sequential Prefix
k
This algorithm computes the prefixes i0 a[i ] for all 0 ≤ k < n given an
n-element array a as input.
Input Parameters: a
Output Parameters: None
sequential_prefix(a) {
n = a.length
sum[0] = a[0]
for i = 0 to n - 1
sum[i] = sum[i – 1]  a[i]
}
Algorithm 12.2.26 Parallel Prefix
k
This algorithm computes the prefixes i0 a[i ] for all 0 ≤ k < n given an
n-element array a as input.
Input Parameters: a
Output Parameters: None
parallel_prefix(a) {
n = a.length
for i = 0 to n - 1
sum[j] = a[j]
for i = 0 to lg n
for j = 0 to n - 1 in parallel
if (j + 2i < n)
sum[j + 2i] = sum[j]  sum[j + 2i]
}
Algorithm 12.2.31 Sequential List
Ranking
This algorithm takes as an input the first node u of a linked list. Each
node has fields next, the next node of the list or null for the last node in
the list, and rank, which is assigned the distance of the node from the
end of the list.
Input Parameters: u
Output Parameters: None
sequential_list_ranking(u) {
n = 0 // counter for total number of nodes
node = u
while (node ! = null) {
n = n + 1
node = node.next
}
node = u // start again at the beginning
while (node ! = null) {
node.rank = n
n = n - 1
node = node.next
}
}
Algorithm 12.2.32 List Ranking
This algorithm takes as an input the first node u of a linked list L(u). Each
node has fields next, the next node of the list or null for the last node in
the list, and rank, which is assigned the distance of the node from the
end of the list.
Input Parameters: u
Output Parameters: None
list_ranking(u) {
for each node in L(u) in parallel
node.rank = 1
done = false
do {
for each node in L(u) in parallel
if (node.next == null)
done = true
else {
node.rank = node.rank + node.next.rank
node.next = node.next.next
}
} while (!done)
}
Algorithm 12.4.11 Minimum Label
This algorithm takes as input a two-dimensional array a with indexes
ranging from 0 to n + 1. Every element of the array contains fields color
and label. The color field of elements in rows and columns with indexes 0
or n + 1 is assumed to be zero. The algorithm computes a component
labeling for a and stores it in the label field.
Input Parameters: a,n
Output Parameters: None
minimum_label(a,n) {
// initialize the label field of
// every black pixel to a unique value
for i,j  {1, ... , n} in parallel
if (a[i,j].color == 1)
a[i,j].label = i × n + j
else
a[i,j].label = 0
// perform the second step on
// every label n2 times
for k = 1 to n2
for i,j  {1, ... , n} in parallel
for x = i - 1 to i + 1
for y = j - 1 to j + 1
if (a[x,y].color == 1)
a[i,j].label =
min(a[i,j].label, a[x,y].label)
}
Algorithm 12.5.2 Ring Broadcast
This algorithm is run on a ring network. The initiator runs
init_ring_broadcast, and the noninitiators run ring_broadcast. All
processors terminate successfully after having received a message from the
initiator.
init_ring_broadcast() {
send token to successor
receive token from predecessor
terminate
}
ring_broadcast() {
receive token from predecessor
send token to successor
terminate
}
Algorithm 12.5.4 Broadcast
This algorithm works on any (fixed) connected network. The initiator runs
init_broadcast and the noninitiators broadcast. All processors terminate
successfully after having received a message from the initiator. In the
algorithms we assume that the current machine we are running the code on
is called p.
init_broadcast() {
N = {q | q is a neighbor of p}
for each q  N
send token to neighbor q
terminate
}
broadcast() {
receive token from neighbor q
N = {q | q is a neighbor of p}
for each q  N
send token to neighbor q
terminate
}
Algorithm 12.5.5 Echo
This algorithm works on any (fixed) connected network. The initiator runs
init_echo and the noninitiators echo. All processors terminate successfully
after having received a message from the initiator. The initiator terminates
after all processors have received its message. The machine on which we
are running the code is called p.
init_echo() {
N = {q | q is a neighbor of p}
for each q  N
send token to neighbor q
counter = 0
while (counter < |N|) {
receive token
counter++
}
terminate
}
echo() {
receive token from neighbor q
parent = q
N = {q | q is a neighbor of p} - {parent}
for each q  N
send token to neighbor q
counter = 0
while (counter < |N|) {
receive token
counter++
}
send token to neighbor parent
terminate
}
Algorithm 12.5.9 Leader Election
This algorithm runs on the unidirectional ring. The initiators run
init_election and the noninitiators run election. All processors terminate
successfully, and exactly one initiator has its leader attribute set to true. In
the algorithms, we assume that the current machine we are running the
code on is called p. Every processor has a next neighbor, p.next, to which it
can send messages.
init_election() {
send  token,p.ID  to p.next
min = p.ID
receive  token,I 
while (p.ID != I) {
if (I < min)
min = I
send  token,I  to p.next
receive  token,I 
}
if (p.ID == min)
p.leader = true
else
p.leader = false
terminate
}
election() {
p.leader = false // noninitiator is never chosen as leader
do {
receive  token,I 
send  token,I  to p.next
} while (true)
}