Solution of eigenvalue problems on heterogeneous
computing architectures
Davor Davidović
Center for informatics and computing
Rudjer Boskovic Institute
Bijenička cesta 54, 10000 Zagreb
Email: [email protected]
Abstract—In this paper are presented current achievements
and the state-of-the-art algorithms and implementations for dense
linear algebra on traditional architectures such as single-core
machines or distributed memory parallel machines. Also, this
paper summarizes the current implementations and publicly
available libraries for basic linear algebra for multi-core and
many-core architectures (e.g. graphic processors), but also emphasizes the lack of advanced linear algebra implementation on
the multi- and many-core architectures. The problem that will
be solved during this research is to speed-up the execution of the
decomposition phase (i.e. Hessenberg reduction, tridiagonalization, bidiagonalization) for the symmetric eigenvalue problem by
utilizing the heterogeneous architectures (CPU + GPU). The final
aim of this research is to explore the use of external devices, in
particular, general purpose GPUs (GPGPUs), to accelerate the
solution of matrix eigenvalue problems. Also, interesting topic
will be to explore the applicability of the implementation results
for symmetric eigenvalue problem to the SVD decomposition on
the multi- and many-core architectures.
Index Terms—eigenvalue problem, dense linear algebra, Hessenberg reduction, tridiagonalization, graphic processors (GPUs),
hybrid computing
I. I NTRODUCTION
Computing the eigenvalues and eigenvectors of a matrix A
is one of the fundamental tasks in numerical linear algebra,
with applications in a large range of different scientific and
engineering problems. With the increased complexity of mathematical models and the availability of various high performance systems, such as multi-core and many-core systems (for
example general purpose graphic processor units - GPGPUs),
there is a growing demand to solve large-scale eigenvalue
problem.
The improvements in performance, functionality, and programmability of graphics processors in the last few years have
renewed the interest in this class of hardware for generalpurpose computations. Most of the activities were focused
on basic dense linear algebra, with important gains in the
performance delivered for basic linear algebra operations [1].
Combining CPUs with graphic processing units (GPUs),
allows moving well-structured, easily parallelizable tasks to
external devices while performing unstructured tasks on the
more flexible traditional (multi-core) processors. Despite its
popularity in recent years, the potential of solving advanced
linear algebra problems on heterogeneous architectures is still
largely unexplored. Earlier studies have evaluated the perfor-
mance of some advanced linear algebra operations on former
generations of graphic processing units that were produced
before the first CUDA release. Some of them, like in [2], were
specifically focused on the evaluation of different procedures
(like LU decomposition) for solving dense linear systems on
graphic hardware. The aim of this research is to explore the
use of external devices, in particular, general purpose GPUs
(GPGPUs), to accelerate the solution of matrix eigenvalue
problems.
There is variety of implementations and publicly available
libraries for advanced linear algebra on traditional architectures such as single-core machines or distributed memory
parallel machines. The most popular collections for the linear
algebra routines are LAPACK [3] and ScaLAPACK [4] libraries. LAPACK library includes BLAS [5] libraries that provide standard building block for performing basic vector and
matrix operations on single-core machines. The Level 1 BLAS
performs scalar, vector-scalar and vector-vector operations,
the Level 2 BLAS perform matrix-vector operations, while
the Level 3 BLAS perform matrix-matrix operations. The
ScaLAPACK library includes redesigned LAPACK libraries
for distributed memory parallel machines. The library and
research project called ATLAS (Automatically Tuned Linear
Algebra Software) [6] is focusing on applying empirical techniques in order to provide portable performance. At present,
it provides C and Fortran77 interfaces to a portably efficient
BLAS implementation, as well as a few routines from LAPACK. Also, some recent works has been done that describe
some specific implementations, like the implementation of the
One–Sided Jacobi method [7] for solving eigenvalue problem.
In the last few years a various implementations of basic
linear algebra algorithms for multi-core and many-core architectures has been published, while there is still a lack of
implementations for advanced linear algebra on the multi-core
and many-core architectures. The CUBLAS [8] is NVIDIAs
implementation of the basic linear algebra algorithms. It
includes the BLAS 1, 2 & 3 routines redesigned using CUDA
for execution on the NVIDIA graphic cards.
Some research has already been done in the implementation
of the different dense advanced linear algebra algorithms on
the multi-core and many-core architectures like systems of linear equations [9] [10] and the efficient implementation of the
LU, QR and Cholesky factorization using vector capabilities
of the graphic processors, described in [11]. The MAGMA
and PLASMA [12] are two projects that aim to achieve high
performance and portability across a wide range of multi-core
architectures and hybrid systems respectively. PLASMA is a
redesign of parts of LAPACK and ScaLAPACK for shared
memory machines based on multi-core processor architectures.
To achieve high performance on this type of architecture,
PLASMA relies on tile algorithms [13], which provide fine
granularity parallelism. The MAGMA [14] project aims to
develop a dense linear algebra library similar to LAPACK but
for heterogeneous/hybrid (CPU + GPU) architectures, starting
with current ”Multicore+GPU” systems. In MAGMA, several
important implementations of the linear algebra solutions are
developed, like QR factorization [15] using multiple GPUs,
Cholesky factorization [16] on multi-core systems with GPU
accelerators and Hessenberg reduction [17] for hybrid GPUbased systems.
The lack of the implementation of the eigenvalue solvers
on multi-core and many-core architectures is still present. The
current implementations of the algorithms that are utilized
to solve eigenvalue problems on the multi- and many-core
architectures will be described further in the article.
The rest of the paper is structured as follows: Section
II describes the mathematical problem that we are solving.
Section III introduces multi-core and many-core architectures.
Section IV summarizes the current state-of-the-art approaches
for solving eigenvalue problems and section V gives a short
description of the research that will be conducted.
II. M ATHEMATICAL P ROBLEM
Consider a square matrix A. If there is a scalar λ and a
nonzero vector x such that
Ax = λx,
(1)
then λ is called an eigenvalue and x is called an eigenvector
(associated with λ) of A. Counting multiple copies, every n×n
matrix A has n such eigenvalues λ. Typical applications in
science and engineering, for which it is desirable to compute
a large fraction of the eigenvalues for a potentially large matrix
A, include DFT (density functional theory) calculations in
computational chemistry, stability analyses in computational
fluid dynamics, multivariance statistics and linear-quadratic
optimal control problems. For an overview see the introduction
of [18].
Computing the eigenvalue information is sometimes the
most time-consuming stage in applications. When A is a
dense matrix, numerically stable methods for the solution
of this problem require O(n3 ) floating point arithmetic operations (flops). Given that for large-scale problems n is
usually more than 10000, large distributed-memory clusters
of computers have been employed for the solution of these
problems in the past. In addition, the need for computing
many eigenvalues renders traditional sparse iterative solvers
for eigenvalue problems, such as Arnoldi and Jacobi-Davidson
methods [19], inefficient, mainly because of the need for reorthogonalization to avoid re-convergence towards converged
eigenvalues. Hence, dense direct solvers need to be used to
address the computation of many eigenvalues.
For a general real matrix A, dense direct solvers usually
aim at computing a Schur decomposition of A:
QT AQ = S,
(2)
where Q is orthogonal and S is a quasi-upper triangular matrix
from which the eigenvalues can be directly read off. A simple
backward substitution procedure can be used to extract the
eigenvectors from S and Q [20].
If A is symmetric then (2) becomes a spectral decomposition:
QT AQ = D,
(3)
where Q is (again) orthogonal and D is a diagonal matrix with
the eigenvalues of A on the diagonal.
III. G ENERAL P URPOSE G RAPHIC P ROCESSING U NITS
(GPGPU S )
GPGPU stands for General-Purpose computing on GPU,
also known as GPU Computing. GPUs are capable of very
high computation and data throughput. Once specially designed for computer graphics and difficult to program, today
GPUs are general-purpose parallel processors with support
for accessible programming interfaces and industry-standard
languages such as C and FORTRAN. The speedups achieved
by using GPUs are of orders of magnitude vs. optimized CPU
implementations.
In 2006, with the first release of the NVIDIA CUDA [8]
programming model, a new generation of GPUs appeared,
with a completely different architectural design which is
capable to solve many of the restrictions related with general
purpose computation that were present in previous generations
of graphics processors. These new GPUs turn the massive
floating-point computational power of a modern graphics
accelerator’s shader pipeline into general-purpose computing
power. The GPU-based high performance computers are starting to play a significant role in large-scale computing. Three
of the 5 most powerful supercomputers in the world take
advantage of GPU acceleration. This includes the current
leader as of October 2010, Tianhe-1A, which uses the NVIDIA
Tesla platform.
There are several new characteristics in the new generation
of the GPUs:
1) The clock frequency of the unified shader is much higher
than that of the fragment processors present in previous
GPUs,
2) Every shader consists of a large collection of computation units, called Stream Processors (SP) which are
grouped in a multiprocessor,
3) The memory hierarchy is much more sophisticated, and
includes a L2 cache and small fast memories shared by
all SPs in the same multiprocessor.
Fig. 1. Difference between CPU and GPU architectures – picture taken from
NVIDIA CUDA C Programming Guide v3.1.1.
A. GPU architecture
Recently, starting from 2006, NVIDIA began releasing cards
supporting an API extension to the C programming language
called CUDA (Compute Unified Device Architecture), which
allows specified functions from a normal C program to run on
the GPUs stream processors. This makes C programs capable
of taking advantage of a GPUs ability to operate on large
matrices in parallel, while still making use of the CPU when
appropriate. A traditional C code that runs on the host (CPU)
is rearranged thus data parallel sections of the code, called
kernels, are moved to the device (GPU). CUDA is also the
first API to allow CPU-based applications to access directly
the resources of a GPU for more general purpose computing
without the limitations of using a graphics API.
The difference between CPU and GPU architecture is shown
in Fig. 1. One can see that every multiprocessor (green row)
on the GPU has its own control unit and cache, while on
the CPU, cores share common control and cache units. Furthermore, every multiprocessor is consisted of 8 − −16 cores
called stream processors. The GPU is especially well-suited
to address problems that can be expressed as data-parallel
computations (the same program is executed on many data
elements in parallel), with high arithmetic intensity (i.e. the
ratio of arithmetic operations to memory operations). Because
the same program is executed for each data element, there
is a lower requirement for sophisticated flow control, and
because it is executed on many data elements and has high
arithmetic intensity, the memory access latency can be hidden
with calculations instead of big data caches.
Despite the importance of arithmetic intensity, it is worth
noting that modern GPUs have very high peak bandwidth to
their on-board memory, on the order of 10× the CPU-memory
bandwidths on typical PC platforms. This is why GPUs can
outperform CPUs at tasks such as matrix multiplication and
other matrix-matrix computations, which have a low computation/bandwidth ratio.
In Fig. 2 is described the memory hierarchy of the GPU.
At the hardware level, for example the third generation of
NVIDIA graphic cards, the GTX 480 processor is a collection
of 15 multiprocessors, with 32 processors each. Each multiprocessor has its own shared memory which is common to all
32 processors inside it with the huge memory bandwidth but
small capacity. It also has a set of 32–bit registers, texture, and
constant memory caches. In any cycle, each processor of the
multiprocessor executes the same instruction on different data.
Fig. 2. GPU architecture – picture taken from NVIDIA CUDA C Programming Guide v3.1.1.
TABLE I
H ARDWARE DIFFERENCES : GPU VS. CPU
Processor
Intel Core
i7-970
GeForce
GTX 480
TESLA
M2070Q
Multiprocessors
1
15
14
Cores/threads
6/12
480
448
Clock (MHZ)
3200
1401
1150
Bandwidth (GB/s)
25.6
177.5
150.3
6144
Memory size (MB)
∅
1536
Single precision (GFLOPs) FMA
N/A
1344.96
1288
Double precision (GFLOPs) FMA
107.5
168
515.2
Communication between multiprocessors is done through the
device memory (slow), which is available to all the processors
of the multiprocessors.
Table I shows the differences between 2 NVIDIA GPUs,
GeForce GTX 480, TESLA M2070Q and the new generation
of 32–nm Intel Core i7-970 processor. One can see very high
memory bandwidth (bandwidth between multiprocessor and
device memory) on the both graphic cards in comparison to the
CPU. Although, the GTX 480 has better overall characteristics
than Tesla M2070Q, it shows considerably lower doubleprecision performance. This is because GTX aims at gaming
market community while Tesla graphic cards are specially
designed to perform high performance computing. The last
two table rows describe the peak processing power in single
and double precision, respectively. The FMA states for a fused
multiply-add that is a floating-point multiply-add operation
performed in one step, with a single rounding.
B. Hybrid GPU-based platform
Hybrid GPU-based multi-core platforms composed of both
homogeneous multi-core CPUs and many-core GPUs stand
out among a confluence of current hardware trends as they
provide an effective solution to these two needs: computational
power and decreasing the gap between computation and communication speeds. Indeed, as power consumption is typically
proportional to the cube of the frequency, GPUs have a clear
advantage against current homogeneous multi-cores, as GPUs
compute power is derived from many cores that are of low
frequency.
Older versions of NVIDIA GPUs (compute capability 1.x)
feature massive parallelism but serial kernel execution, i.e.
only one kernel is allowed to run at a time using the entire
GPU. This means that only large, highly parallelizable kernels can run efficiently on GPUs. Although, parallel kernel
execution is available with the newer versions of NVIDIA
GPUs (compute capability 2.x) this can not solve entirely solve
problems of load balancing between CPU and GPU. The idea
of using hybrid algorithms presents an opportunity to remedy
this situation and therefore enable the efficient use of GPUs
well beyond the case of data-parallel applications. Namely, the
solution and advice to developers is to use a hybrid coding
approach, where small, non-parallelizable kernels would be
executed on the CPU, and only large, data-parallel kernels on
the GPU. Although GPUs move towards supporting task-based
parallelism as well (e.g., advertised for the next generation
NVIDIA GPUs, code named Fermi), small tasks that arise in
dense linear algebra would still make sense to be executed on
the CPU, reusing existing software infrastructure.
Fig. 3. CUDA programming model – picture taken from NVIDIA CUDA
C Programming Guide v3.1.1.
IV. R ELATED WORK (C URRENT APPROACHES )
In the following, we briefly summarize state-of-the art
algorithms and software for computing spectral and Schur
decompositions that are basic steps in solving eigenvalue
problems and are good starting point for other researches.
A. Spectral decomposition
C. CUDA programming model
The CUDA programming model, for a programmer, is a
collection of threads running in parallel as shown by the
Fig. 3. A warp is a collection of threads that can be run
simultaneously on a multiprocessor. The warp size is fixed for
a specific GPU, and counts 32 threads for nowadays GPUs.
The programmer decides the number of threads to be executed.
If the number of threads is more than the warp size, they are
time-shared internally on the multiprocessor. A collection of
threads (called a block) is mapped to a multiprocessor at a
given time. Multiple blocks can be assigned to a multiprocessor and their execution is time-shared. A single computation
on a device (GPU) generates a number of blocks. A collection
of all the blocks in a single computation is called a grid. All
threads of the blocks mapped to a multiprocessor divide its
resources equally amongst themselves. Each thread and block
is given a unique ID that can be accessed within the thread
during its execution. Each thread executes a single instruction
at a time. GPU is a co-processor to the CPU and needs to be
initiated by the CPU.
Describe CUDA programming model - releases, compute
capabilities
Any symmetric matrix A can be reduced, in a finite number
of steps, as follows:
QT1 AQ1 = T,
(4)
where Q1 is orthogonal and T is a tridiagonal matrix. For
example, for n = 6 the matrix T takes the shape


x x
 x x x





x
x
x

,
T =

x
x
x



x x x 
x x
where x denotes an arbitrary, generically nonzero number.
There exist several approaches to achieve the reduction 4.
On a single-core processor, an approach based on (block)
Householder transformations described in [21] and implemented in LAPACK [3] is usually preferred. A similar approach is implemented in ScaLAPACK [4] for parallel distributed memory architectures.
On a multi-core processor, a two-stage approach described
in detail in [22] is preferred, as it avoids the need for large
matrix-vector multiplications (which perform quite poorly)
in the Householder-based approach mentioned above. The
two-stage approach first reduces A to block tridiagonal form
QT0 AQ0 = Tb with nb nonzero sub-diagonals. For example,
for n = 6 and nb = 3 the matrix Tb takes the shape


x x x x
 x x x x x



 x x x x x x 

.
Tb = 

 x x x x x x 

x x x x x 
x x x x
The second stage consists of reducing Tb further to tridiagonal form. If, nb n this process requires only a negligible
amount of operations. For single core processors, the approach
described above is implemented in the SBR toolbox [23].
Numerical experiments provide evidence that some vendorspecific variants of LAPACK, including Intel’s MKL, make
use of this approach. A prototype implementation documented
in [24] demonstrates the potential of the two-stage approach
to perform well in a heterogeneous computing environment,
by moving parts of the computation in Stage 1 to the external
device us as graphic processor.
Once A is reduced to tridiagonal form, the computation
of eigenvalues becomes a negligible operation. A GPGPU
implementation of the bisection algorithm is described in [25].
If also the eigenvectors are to be computed, more care needs
to be applied to devise an efficient implementation. The highly
sophisticated and successful MRRR algorithm [26] [27] is
currently the method of choice, of which an efficient GPGPU
implementation has been described in [28].
B. Schur decomposition
A general non-symmetric matrix A can be reduced, in a
finite number of steps, as follows:
QT1 AQ1
= H,
(5)
where Q1 is orthogonal and H is a Hessenberg matrix. For
example, for n = 6 the matrix H takes the shape


x x x x x x
 x x x x x x 



x x x x x 

.
H=
x x x x 



x x x 
x x
In contrast to the symmetric case, there exists so far only one
truly successful approach to achieve the reduction (refeq:5).
This approach is based on (block) Householder transformations, as described in [21] [29] and implemented in LAPACK [3]. A variant of this approach for parallel distributed
memory architectures is included in ScaLAPACK [4]. Preliminary experiments reported in [30] reveal that an extension
of the two-stage approach to Hessenberg reduction does not
perform very well. Even on multi-core processors the twostage approach was reported to perform significantly worse
than LAPACK (linked with multithreaded BLAS). This is
mainly due to the fact that the computational cost for the second stage becomes non-negligible for non-symmetric matrices.
Developing an efficient algorithm for reduction to Hessenberg
form on multi-core architectures remains an open problem.
However, a variant that is particularly efficient for heterogeneous architectures (single core + GPGPU) is described in [17]
and [31].
Once A has been reduced to a Hessenberg matrix H, the
QR algorithm can be used to compute Schur decomposition.
Several variants of the QR algorithm for parallel distributed
memory and traditionally shared memory architectures have
been developed, see [18] and the references therein. However,
there is no variant specifically tailored to multi-core and
heterogeneous architectures described in the literature.
V. R ESEARCH TO BE DONE
The algorithms and software to be developed in the thesis
aim at improving and complementing existing approaches to
solving eigenvalue problems on multi-core and heterogeneous
architectures. The ultimate goal is to have a complete highly
performing software package available. To achieve this goal,
we will make use of existing work, such as the GPGPU
implementation of the MRRR algorithm described in [28].
However, there are still a number of gaps to be filled.
The part of this research will be conducted at the University
Jaume I, Castellón, Spain as a part of PhD internship in the
period February–July 2011. The research will be done in the
research group of prof. Enrique Quintana-Ortı́ who is the
author of numerous scientific and technical papers in the ares
of numerical linear algebra and GPU-based hybrid computing.
In the following paragraphs are shortly described research
activities, divided by topics.
A. Reduction to tridiagonal form
For the reduction of a symmetric matrix to tridiagonal
form, two approaches appear to be promising: (I) a variant
of the two-stage approach as described in [24], and (II)
an extension of the work in [17] to symmetric matrices.
Intuitively, Approach (II) appears to be suitable if the CPU
has only a few, say 2 or 4, cores while Approach (I) appears
to offer greater advantages for many cores. However, a careful
numerical implementation and comparison of both approaches
needs to be performed before such a conclusion can be drawn.
The specific tasks are as follows.
1) Complement and improve the preliminary implementation described in [21] to accommodate multi-core CPUs
and accumulation of orthogonal transformation
2) Develop an algorithm and implementation that extends [17] to symmetric matrices.
3) Compare Approaches (I) and (II) for a variety of computing environments.
B. Reduction to Hessenberg form
As mentioned above, the efficient reduction to Hessenberg
form on multicore processors is an important open problem,
which will be approached in this thesis. The specific tasks are
as follows.
1) Develop a multi-core variant of Hessenberg reduction,
possibly by combining the LAPACK-like algorithm
based on (block) Householder transformations with
block storage schemes and dynamic scheduling.
2) Combine this multi-core variant with the ideas from [17]
to develop an efficient algorithm and implementation for
heterogeneous architectures.
3) Evaluate the efficiency of the developed implementations for a variety of computing environments.
C. Singular Value Decomposition
Interesting and important problem that is closely connected
to the eigenvalue problem in terms of employing the same
numerical tools (for example orthogonal transforms via, e.g.
Householder reflectors) is singular value decomposition problem. The scope of this work will also be to analyze the
applicability of the implementation results of the symmetric
eigenvalue problem to SVD decomposition. Some research in
implementation of the SVD on the multi-core and many-core
architectures have already been done in [32] and for hyperbolic
SVD in [33].
D. Iterative refinement of decomposition
Due to the poor performance of double precision (DP) operations on current GPGPUs, high performance can usually only
be attained in single precision (SP). In scientific computing,
SP is often not sufficient to warrant the desired accuracy
in the results and it is standard to use DP. To combine the
efficiency of GPGPUs with the accuracy of DP, it is necessary
to perform the major part of the computation in SP and attain
DP by a (hopefully cheap) refinement procedure. Algorithms
for refining entire spectral and Schur decompositions in a
mixed SP/DP environment are currently under investigations.
In the thesis these algorithms will be adapted to heterogeneous
computing environments.
VI. C ONCLUSION
With the omnipresence of multi-core processors and GPGPUs there is and will be an increasing demand to reformulate existing algorithms to perform well on heterogeneous
computing architectures. In this paper is shown that various
GPU/hybrid implementations of the basic linear algebra algorithms already exist, but there is a lack of advanced linear
algebra implementations on the many-core architectures, especially ones that compute the eigenvalue problems and SVD.
This research aims to perform the reformulation of the existing
algorithms for dense eigenvalue solvers. A software package
will be made available and it can be expected that it will have
a large impact on the scientific computing community as there
is currently a severe lack of eigenvalue solvers that can benefit
from many cores and/or the availability of a GPGPU.
ACKNOWLEDGMENT
I would like to express my special thanks to professor
Daniel Kressner from the ETH Zürich for his enormous help in
the preparation and correction of this paper without whom this
paper would not be possible. I would also like to thank to my
mentors, prof. Karolj Skala from the Rudjer Boskovic Institute,
and prof. Domagoj Jakobović from the Faculty of Electrical
Engineering and Computing for their help and support with
comments and leadership.
R EFERENCES
[1] S. Barrachina, M. Castillo, F. Igual, R. Mayo, and E. Quintana-Ortı́,
“Evaluation and tuning of the level 3 CUBLAS for graphics processors,” in Parallel and Distributed Processing, 2008. IPDPS 2008. IEEE
International Symposium on. IEEE, 2008, pp. 1–8.
[2] N. Galoppo, N. Govindaraju, M. Henson, and D. Manocha, “LUGPU: Efficient algorithms for solving dense linear systems on graphics
hardware,” in Proceedings of the 2005 ACM/IEEE conference on Supercomputing. IEEE Computer Society, 2005, p. 3.
[3] E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra,
J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney et al.,
“LAPAck Users’ Guide SIAM,” 1999.
[4] L. Blackford, A. Cleary, J. Choi, E. d’Azevedo d’Azevedo, J. Demmel,
I. Dhillon, J. Dongarra, S. Hammarling, G. Henry, A. Petitet et al.,
ScaLAPACK users’ guide. Society for Industrial Mathematics, 1997.
[5] C. Lawson, R. Hanson, D. Kincaid, and F. Krogh, “Basic linear algebra
subprograms for Fortran usage,” ACM Transactions on Mathematical
Software (TOMS), vol. 5, no. 3, pp. 308–323, 1979.
[6] R. Whaley and J. Dongarra, “Automatically tuned linear algebra software,” in Proceedings of the 1998 ACM/IEEE conference on Supercomputing (CDROM). IEEE Computer Society, 1998, pp. 1–27.
[7] S. Singer, S. Singer, V. Hari, K. Bokulić, D. Davidović, M. Jurešić, and
A. Ušćumlić, “Advances in Speedup of the Indefinite One-Sided Block
Jacobi Method,” in AIP Conference Proceedings, vol. 936, 2007, p. 519.
[8] C. NVIDIA, “CUBLAS Library,” NVIDIA Corporation, Santa Clara,
California, 2008.
[9] S. Tomov, R. Nath, H. Ltaief, and J. Dongarra, “Dense linear algebra
solvers for multicore with GPU accelerators,” in Parallel & Distributed
Processing, Workshops and Phd Forum (IPDPSW), 2010 IEEE International Symposium on. IEEE, 2010, pp. 1–8.
[10] S. Barrachina, M. Castillo, F. D. Igual, R. Mayo, E. S. Quintana-Ortı́,
and G. Quintana-Ortı́, “Exploiting the capabilities of modern GPUs for
dense matrix computations,” Concurrency and Computation: Practice
and Experience, vol. 21, no. 18, pp. 2457–2477, 2009. [Online].
Available: http://dx.doi.org/10.1002/cpe.1472
[11] V. Volkov and J. Demmel, “LU, QR and Cholesky factorizations
using vector capabilities of GPUs,” EECS Department, University of
California, Berkeley, Tech. Rep. UCB/EECS-2008-49, May, pp. 2008–
49, 2008.
[12] E. Agullo, J. Demmel, J. Dongarra, B. Hadri, J. Kurzak, J. Langou,
H. Ltaief, P. Luszczek, and S. Tomov, “Numerical linear algebra
on emerging architectures: The PLASMA and MAGMA projects,” in
Journal of Physics: Conference Series, vol. 180. IOP Publishing, 2009,
p. 012037.
[13] A. Buttari, J. Langou, J. Kurzak, and J. Dongarra, “A class of parallel
tiled linear algebra algorithms for multicore architectures,” Parallel
Computing, vol. 35, no. 1, pp. 38–53, 2009.
[14] S. Tomov, R. Nath, P. Du, and J. Dongarra, “MAGMA version 0.2 User
Guide,” 2009.
[15] E. Agullo, C. Augonnet, J. Dongarra, M. Faverge, H. Ltaief, S. Thibault,
and S. Tomov, “QR Factorization on a Multicore Node Enhanced with
Multiple GPU Accelerators,” University of Tennessee Computer Science,
Tech. Rep. ICL-UT-10-04, Oct. 2010.
[16] H. Ltaief, S. Tomov, R. Nath, P. Du, and J. Dongarra, “A Scalable High
Performant Cholesky Factorization for Multicore with GPU Accelerators,” Tech. report, LAPACK Working Note 223, Tech. Rep., 2009.
[17] J. Dongarra and S. Tomov, “Accelerating the reduction to upper hessenberg form through hybrid gpu-based computing,” LAPACK Working
Note 219, Tech. Rep., 2009.
[18] R. Granat, B. Kågström, and D. Kressner, “A novel parallel QR
algorithm for hybrid distributed memory HPC systems,” SIAM J. Sci.
Comput, vol. 32, no. 4, pp. 2345–2378, 2010.
[19] Z. Bai, J. Demmel, J. Dongarra, A. Ruhe, and H. van der Vorst,
“Templates for the solution of algebraic eigenvalue problems, volume
11 of Software, Environments, and Tools,” Society for Industrial and
Applied Mathematics (SIAM), Philadelphia, PA, 2000.
[20] G. Golub and C. Van Loan, Matrix computations. Johns Hopkins Univ
Pr, 1996.
[21] J. Dongarra, D. Sorensen, and S. Hammarling, “Block reduction of
matrices to condensed forms for eigenvalue computations,” Journal of
Computational and Applied Mathematics, vol. 27, no. 1-2, pp. 215–227,
1989.
[22] C. H. Bischof, B. Lang, and X. Sun, “A framework for symmetric
band reduction,” ACM Trans. Math. Softw, vol. 26, no. 4, pp. 581–601,
2000. [Online]. Available: http://doi.acm.org/10.1145/365723.365735
[23] C. H. Bischof, B. Lang, and X. Sun, “Algorithm 807: The SBR
toolbox - software for successive band reduction,” ACM Trans.
Math. Softw, vol. 26, no. 4, pp. 602–616, 2000. [Online]. Available:
http://doi.acm.org/10.1145/365723.365736
[24] P. Bientinesi, F. D. Igual, D. Kressner, and E. S. Quintana-Ortı́, “Reduction to condensed forms for symmetric eigenvalue problems on multicore architectures,” in PPAM (1), ser. Lecture Notes in Computer Science, R. Wyrzykowski, J. Dongarra, K. Karczewski, and J. Wasniewski,
Eds., vol. 6067. Springer, 2009, pp. 387–395.
[25] V. Volkov and J. W. Demmel, “Using GPUs to accelerate the bisection
algorithm for finding eigenvalues of symmetric tridiagonal matrices,”
Department of Computer Science, University of Tennessee, Knoxville,
inst-UT-CS:adr, LAPACK Working Note 197, Jan. 2008. [Online].
Available: http://www.netlib.org/lapack/lawnspdf/lawn197.pdf
[26] P. Bientinesi, I. S. Dhillon, and R. A. van de Geijn, “A parallel
eigensolver for dense symmetric matrices based on multiple relatively
robust representations.” SIAM J. Scientific Computing, vol. 27, no. 1, pp.
43–66, 2005. [Online]. Available: http://dx.doi.org/10.1137/030601107
[27] I. S. Dhillon, B. N. Parlett, and C. Voemel, “The design
and implementation of the mrrr algorithm.” ACM Trans. Math.
Softw., vol. 32, no. 4, pp. 533–560, 2006. [Online]. Available:
http://doi.acm.org/10.1145/1186785.1186788
[28] C. Lessig and P. Bientinesi, “On parallelizing the MRRR algorithm
for data-parallel coprocessors,” in PPAM (1), ser. Lecture Notes in
Computer Science, R. Wyrzykowski, J. Dongarra, K. Karczewski, and
J. Wasniewski, Eds., vol. 6067. Springer, 2009, pp. 396–402. [Online].
Available: http://dx.doi.org/10.1007/978-3-642-14390-8
[29] G. Quintana-Ortı́ and R. A. van de Geijn, “Improving the
performance of reduction to hessenberg form,” ACM Trans. Math.
Softw, vol. 32, no. 2, pp. 180–194, 2006. [Online]. Available:
http://doi.acm.org/10.1145/1141885.1141887
[30] H. Ltaief, J. Kurzak, and J. Dongarra, “Parallel block Hessenberg
reduction using algorithms-by-tiles for multicore architectures revisited,”
Department of Computer Science, University of Tennessee, Knoxville,
inst-UT-CS:adr, LAPACK Working Note 208, Aug. 2008. [Online].
Available: http://www.netlib.org/lapack/lawnspdf/lawn208.pdf
[31] S. Tomov, R. Nath, and J. Dongarra, “Accelerating the reduction to upper
Hessenberg, tridiagonal, and bidiagonal forms through hybrid GPUbased computing,” Parallel Computing, vol. 36, no. 12, pp. 645–654,
Dec. 2010.
[32] S. Lahabar and P. J. Narayanan, “Singular value decomposition on gpu
using cuda.” in IPDPS. IEEE, 2009, pp. 1–10. [Online]. Available:
http://dx.doi.org/10.1109/IPDPS.2009.5161058
[33] V. Novakovic and S. Singer, “A gpu-based hyperbolic svd algorithm,”
CoRR, vol. abs/1008.1371, 2010, informal publication. [Online].
Available: http://arxiv.org/abs/1008.1371