A PARALLEL BISECTION
ALGORITHM (WITHOUT
COMMUNICATION)
Rui Ralha
DMAT, CMAT
Univ. do Minho
Portugal
[email protected]
1
Acknowledgements
CMAT
FCT
POCTI (European Union contribution)
Prof. B. Parlett
2
Outline
• Counting eigenvalues of symmetric
tridiagonals
• The ScaLAPACK’s routine
• A parallel algorithm without communication
• An alternative algorithm
• Some conclusions
3
Counting eigenvalues
4
Nonmonotonicity of Count(x)
5
The ScaLAPACK’s implementation (1)
6
The ScaLAPACK’s implementation (2)
In [1] the authors wrote
“…Ideally, we would like a bracketing algorithm that was simultaneously
parallel, load balanced, devoid of communication, and correct in the face of
nonmonotonicity. We still do not know how to achieve this completely; in the
most general case, when different parallel processors do not even possess
the same floating point format, we do not know how to implement a correct
and reasonably fast algorithm at all. Even when floating point formats are
the same, we do not know how to avoid some global communication…”
and considered a bracketing algorithm to be correct if
(1) every eigenvalue is computed exactly once,
(2) the computed eigenvalues are correct to within the user
specified error tolerance,
(3) the computed eigenvalues are in sorted order.
7
The ScaLAPACK’s implementation (3)
8
The ScaLAPACK’s implementation (4)
9
Drawbacks of the ScaLAPACK’s implementation
10
A simple and incorrect parallel algorithm (without
communication)
To partition the initial Gerschgorin interval into p subintervals of equal width
and assign to processor i the task of finding all the eigenvalues in the
ith subinterval [ i 1 ,  i ] . But, even with processors with the same arithmetic
(nonmonotonic) the algorithm may be incorrect.
For example, with n=p=3, it may happen [1]
Processor 1 : [  0 , 1 ], n0  0, n1  2
Processor 2 : [ 1 ,  2 ], n1  2, n 2  1
Processor 3 : [  2 ,  3 ], n 2  1, n3  3
Therefore, the second eigenvalue will be computed twice (processors 1 and 3)
11
Parallel bisection for computing the
eigenvalues of [-1 2 -1] with 100 processors
12
Our proposal (1)
13
Our proposal (2)
14
Our proposal (3)
15
Our proposal (4)
16
Sorting eigenvalues
For the Wilkinson’s matrix of order 21 we have
16  8.03894111581428, 17  8.03894112282902
With single precision in Matlab we get
16  [ ,  ]  [8.0389404829687500, 8.038941383361816], n  15, n  17.
With double precision we get
17  [ , ]  [8.038941122829023, 8.038941122829025], n  16, n  17.
We assume that eigenvalues are to be gathered in a “master”
processor (this is a standard feature of ScaLAPACK). Supose that the
“master” receives 16   , 17   (out of order) and knows that the
processor that computed 17 has better accuracy. Then, it keeps 17
and, if required, it corrects 16 to be smaller than 17 .
17
An alternative algorithm (1)
• Phase 1(equal for every processor): carry out a
(not too large) number of bisection steps in a
breadth first search to get a “good picture” of the
spectrum. Produces a number of intervals (at
least p = number of processors).
• Phase 2: distributes intervals to processors
trying to achieve load- balance (the same
number of eigenvalues to each processor)
• Phase 3: each processor computes the assigned
eigenvalues to some prescribed accuracy
18
An alternative algorithm (2)
Speedup 
 n
B1  max B3( j )
1 j  p
 : average number of bisection steps per eigenvalue (40 - 50 in double precision)
B1 : number of bisection steps in Phase 1 (equal for all processors )
B3( j ) : number of bisection steps carried out by the jth processor in Phase 3
For B1  n, we get Speedup  
independen tly of the number p of processors .
We need B1    p with   n/p.




 n
1
 p
Speedup 
 1 
  n   p 
 n

 p
1

p
  p p  1 

n
For fixed values of  and  , the ratio  :
should not diminish as n and p grow (for scalabilit y).
p(p - 1)
19
An alternative algorithm (3)
• Preliminar implementation (in Matlab)
• Finishes Phase 1 when enough intervals have
been produced such that, for each k=1,…,p-1,
an end point x of one of those intervals satisfies

n
n 
Count(x)  k - 0.05 , k  0.05 
p
p

• This may affect the speedup by 10%.
• This termination criteria for Phase 1 may be
hard (i.e, take too many bisection steps) to
satisfy in some cases.
20
Parallel bisection for computing the
eigenvalues of [-1 2 -1] of order 10^4
21
Conclusions




Parallel bracketing in ScaLAPACK’s requires global
communication;
We have proposed an algorithm that is communication
free and is load balanced in the sense that each
processor computes the same number of eigenvalues
(if p divides n);
In homogeneous systems, our algorithm produces
sorted eigenvalues even when the arithmetic is
nonmonotonic;
In heterogeneous systems, eigenvalues may be
unsorted (they may be sorted by the “master” if
required);
22