SIAM J. SCI. COMPUT.
Vol. 35, No. 2, pp. A696–A718
c 2013 Society for Industrial and Applied Mathematics
ABSOLUTE VALUE PRECONDITIONING FOR SYMMETRIC
INDEFINITE LINEAR SYSTEMS∗
EUGENE VECHARYNSKI† AND ANDREW V. KNYAZEV‡
Abstract. We introduce a novel strategy for constructing symmetric positive definite (SPD)
preconditioners for linear systems with symmetric indefinite matrices. The strategy, called absolute
value preconditioning, is motivated by the observation that the preconditioned minimal residual
method with the inverse of the absolute value of the matrix as a preconditioner converges to the
exact solution of the system in at most two steps. Neither the exact absolute value of the matrix
nor its exact inverse are computationally feasible to construct in general. However, we provide
a practical example of an SPD preconditioner that is based on the suggested approach. In this
example we consider a model problem with a shifted discrete negative Laplacian and suggest a
geometric multigrid (MG) preconditioner, where the inverse of the matrix absolute value appears
only on the coarse grid, while operations on finer grids are based on the Laplacian. Our numerical
tests demonstrate practical effectiveness of the new MG preconditioner, which leads to a robust
iterative scheme with minimalist memory requirements.
Key words. preconditioning, linear system, preconditioned minimal residual method, polar
decomposition, matrix absolute value, multigrid, polynomial filtering
AMS subject classifications. 15A06, 65F08, 65F10, 65N22, 65N55
DOI. 10.1137/120886686
1. Introduction. Large sparse symmetric and indefinite systems arise in a variety of applications. For example, in the form of saddle point problems, such systems
result from mixed finite element discretizations of underlying differential equations of
fluid and solid mechanics; see, e.g., [3] and references therein. In acoustics, large sparse
symmetric indefinite systems are obtained after discretizing the Helmholtz equation
for certain media types and boundary conditions. Often the need to solve symmetric
indefinite problems comes as an auxiliary task within other computational routines,
such as the inner step in interior point methods in linear and nonlinear optimization [3, 26] or solution of the correction equation in the Jacobi–Davidson method [36]
for a symmetric eigenvalue problem.
We consider an iterative solution of a linear system Ax = b, where the matrix
A is real nonsingular and symmetric indefinite, i.e., the spectrum of A contains both
positive and negative eigenvalues. In order to improve the convergence, we introduce a preconditioner T and formally replace Ax = b by the preconditioned system
T Ax = T b. If T is properly chosen, an iterative method for this system can exhibit
a better convergence behavior compared to a scheme applied to Ax = b. Neither the
preconditioner T nor the preconditioned matrix T A is normally explicitly computed.
If T is not symmetric positive definite (SPD), then T A, in general, is not symmetric with respect to any inner product [29, Theorem 15.2.1]. Thus, the introduction
∗ Submitted to the journal’s Methods and Algorithms for Scientific Computing section August 1,
2012; accepted for publication (in revised form) January 2, 2013; published electronically March 14,
2013. This work is partially supported by National Science Foundation grant 1115734.
http://www.siam.org/journals/sisc/35-2/88668.html
† Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, CA 94720
([email protected]).
‡ Department of Mathematical and Statistical Sciences, University of Colorado, Denver, CO
80217-3364, and Mitsubishi Electric Research Laboratories, Cambridge, MA 02139 (andrew.
[email protected], http://www.merl.com/people/?user=knyazev, http://math.ucdenver.edu/∼
aknyazevs/).
A696
ABSOLUTE VALUE PRECONDITIONING
A697
of a non-SPD preconditioner replaces the original symmetric problem Ax = b by a
generally nonsymmetric T Ax = T b. Specialized methods for symmetric linear systems are no longer applicable to the preconditioned problem and must be replaced by
iterative schemes for nonsymmetric linear systems, e.g., GMRES or GMRES(m) [35],
Bi-CGSTAB [41], and QMR [17].
The approach based on the choice of a non-SPD preconditioner, which leads to
solving a nonsymmetric problem, has several disadvantages. First, no short-term recurrent scheme that delivers an optimal Krylov subspace method is typically available
for a nonsymmetric linear system [15]. In practice, this means that implementations
of the optimal methods (e.g., GMRES) require an increasing amount of work and
storage at every new step and hence are often computationally expensive.
Second, the convergence behavior of iterative methods for nonsymmetric linear
systems is not completely understood. In particular, the convergence may not be
characterized in terms of reasonably accessible quantities, such as the spectrum of the
preconditioned matrix; see the corresponding results for GMRES and GMRES(m)
in [20, 43]. This makes it difficult to predict computational costs.
If T is chosen to be SPD, i.e., T = T ∗ > 0, then the matrix T A of the preconditioned linear system is symmetric with respect to the T −1 -inner product defined by
(u, v)T −1 = (u, T −1v) for any pair of vectors u and v. Here (·, ·) denotes the Euclidean
inner product (u, v) = v ∗ u, in which the matrices A and T are symmetric. Due to
this symmetry preservation, system T Ax = T b can be solved using an optimal Krylov
subspace method that admits a short-term recurrent implementation, such as preconditioned MINRES (PMINRES) [14, 28]. Moreover, the convergence of the method
can be fully estimated in terms of the spectrum of T A.
In light of the above discussion, the choice of an SPD preconditioner for a symmetric indefinite linear system can be regarded as natural and favorable, especially
if corresponding non-SPD preconditioning strategies fail to provide convergence in a
small number of iterations. We advocate the use of SPD preconditioning.
The question of constructing SPD preconditioners for symmetric indefinite systems has been widely studied in many applications. For saddle point problems, the
block-diagonal SPD preconditioning has been addressed, e.g., in [16, 37, 44]. In [2],
it was proposed to use an inverse of the negative Laplacian as an SPD preconditioner
for indefinite Helmholtz problems. This approach was further extended in [25] by
introducing a shift into the preconditioner. Another strategy was suggested in [18],
primarily in the context of linear systems arising in optimization. It is based on the
so-called Bunch–Parlett factorization [9].
We introduce here a different idea (originating from the PhD thesis [42]) of constructing SPD preconditioners that resemble the inverse of the absolute value of the
coefficient matrix. Throughout, the absolute value of A is defined as a matrix function
|A| = V |Λ| V ∗ , where A = V ΛV ∗ is the eigenvalue decomposition of A. We are motivated by the observation that PMINRES with |A|−1 as a preconditioner converges
to the exact solution in at most two steps. We refer to the new approach as the
absolute value (AV) preconditioning and call the corresponding preconditioners the
AV preconditioners.
The direct approach for constructing an AV preconditioner is to approximately
solve |A| z = r. However, |A| is generally not available, which makes the application
of standard techniques, such as incomplete factorizations and approximate inverses,
problematic. The vector |A|−1 r can also be found using matrix function computations, normally fulfilled by a Krylov subspace method [19, 23] or a polynomial
A698
EUGENE VECHARYNSKI AND ANDREW V. KNYAZEV
approximation [30, 31]. Our numerical experience shows that the convergence, with
respect to the outer iterations, of a linear solver can be significantly improved with
this approach, but the computational costs of approximating f (A)r = |A|−1 r may be
too high, i.e., much higher than the cost of matrix-vector multiplication with A.
Introduction of the general concept of the AV preconditioning is the main theoretical contribution of the present work. As a proof of concept example of the AV
preconditioning, we use a geometric multigrid (MG) framework. To investigate applicability and practical effectiveness of the proposed idea, we choose a model problem
resulting from discretization of a shifted Laplacian (Helmholtz operator) on a unit
square with Dirichlet boundary conditions. The obtained linear system is real symmetric indefinite. We construct an MG AV preconditioner that, used in the PMINRES
iteration, delivers an efficient computational scheme.
Let us remark that the same model problem has been considered in [4], where the
authors utilize the coarse grid approximation to reduce the indefinite problem to the
SPD system. Satisfactory results have been reported for small shifts, i.e., for slightly
indefinite systems. However, the limitation of the approach lies in the requirement
on the size of the coarse space, which should be chosen sufficiently large. As we show
below, the MG AV preconditioner presented in this paper allows keeping the coarsest
problem reasonably small, even if the shift is large.
The numerical solution of Helmholtz problems is an object of active research;
see, e.g., [1, 6, 12, 13, 21, 27, 40]. A typical Helmholtz problem is approximated
by a complex symmetric (non-Hermitian) system. The real symmetric case of the
Helmholtz equation, considered in this paper, is less common. However, methods for
complex problems are evidently applicable to our particular real case, which allows
us to make numerical comparisons with known Helmholtz solvers.
We test several solvers, based on the inverted Laplacian and the standard MG
preconditioning, to compare with the proposed AV preconditioning. In fact, the inverted (shifted) Laplacian preconditioning [2, 25] for real Helmholtz problems can be
viewed as a special case of our AV preconditioning. In contrast to preconditioners
in [18] relying on the Bunch–Parlett factorization, we show that the AV preconditioners can be constructed without any decompositions of the matrix, which is crucial for
very large or matrix-free problems.
This paper is organized as follows. In section 2, we present and justify the general notion of an AV preconditioner. The rest of the paper deals with the question of
whether AV preconditioners can be efficiently constructed in practice. In section 3,
we give a positive answer by constructing an example of a geometric MG AV preconditioner for the model problem. The efficiency of this preconditioner is demonstrated
in our numerical tests in section 4. We conclude in section 5.
2. AV preconditioning for symmetric indefinite systems. Given an SPD
preconditioner T , we consider solving a linear system with the preconditioned minimal
residual method, implemented in the form of the PMINRES algorithm [14, 28]. In the
absence of round-off errors, at step i, the method constructs an approximation x(i) to
the solution of Ax = b of the form
(2.1)
x(i) ∈ x(0) + Ki T A, T r(0)
such that the residual vector r(i) = b − Ax(i) satisfies the optimality condition
(2.2)
r(i) T =
min
u∈AKi (T A,T r (0) )
r(0) − uT .
ABSOLUTE VALUE PRECONDITIONING
A699
Here, Ki T A, T r(0) = span T r(0) , (T A)T r(0) , . . . , (T A)i−1 T r(0) is the Krylov subspace generated by the matrix T A and the vector T r(0) , the T -norm is defined by
v2T = (v, v)T for any v, and x(0) is the initial guess. Scheme (2.1)–(2.2) represents
an optimal Krylov subspace method and the PMINRES implementation is based on
a short-term recurrence. The conventional convergence rate bound for (2.1)–(2.2) can
be found, e.g., in [14] and relies solely on the distribution of eigenvalues of T A.
−1
The following trivial, but important, theorem regards |A| as an SPD preconditioner for a symmetric indefinite system.
Theorem 2.1. The preconditioned minimal residual method (2.1)–(2.2) with
−1
preconditioner T = |A| converges to the solution of Ax = b in at most two steps.
−1
Theorem 2.1 implies that T = |A| is an ideal SPD preconditioner. Note that the
theorem holds not only for the preconditioned minimal residual method (2.1)–(2.2),
but for all methods where convergence is determined by the degree of the minimal
polynomial of T A.
−1
In practical situations, the computation of an ideal SPD preconditioner T = |A|
is prohibitively costly. However, we show that it is possible to construct inexpensive
−1
SPD preconditioners that resemble |A| and can significantly accelerate the convergence of an iterative method.
Definition 2.2. We call an SPD preconditioner T for a symmetric indefinite
linear system Ax = b an AV preconditioner if it satisfies
(2.3)
δ0 (v, T −1 v) ≤ (v, |A| v) ≤ δ1 (v, T −1 v) ∀v
with constants δ1 ≥ δ0 > 0 such that the ratio δ1 /δ0 ≥ 1 is reasonably small.
Let us remark that Definition 2.2 of the AV preconditioner is informal because
no precise assumption is made about how small the ratio δ1 /δ0 should be. It is clear
−1
from (2.3) that δ1 /δ0 measures how well the preconditioner T approximates |A| ,
up to a positive scaling. If A represents a hierarchy of mesh problems, then it is
desirable that δ1 /δ0 is independent of the problem size. In this case, if A is SPD,
Definition 2.2 of the AV preconditioner is consistent with the well-known concept of
spectrally equivalent preconditioning for SPD systems; see [10].
The following theorem provides bounds for eigenvalues of the preconditioned matrix T A in terms of the spectrum of T |A|. We note that T and A, and thus T A and
T |A|, do not in general commute. Therefore, our spectral analysis cannot be based
on a traditional matrix analysis tool, a basis of eigenvectors.
Theorem 2.3. Given a nonsingular symmetric indefinite A ∈ Rn×n and an
SPD T ∈ Rn×n , let μ1 ≤ μ2 ≤ · · · ≤ μn be the eigenvalues of T |A|. Then eigenvalues
λ1 ≤ · · · ≤ λp < 0 < λp+1 ≤ · · · ≤ λn of T A are located in intervals
(2.4)
−μn−j+1
μj−p
≤ λj
≤ λj
≤ −μp−j+1 ,
≤
μj ,
j = 1, . . . , p;
j = p + 1, . . . , n.
Proof. We start by observing that the absolute value of the Rayleigh quotient of
the generalized eigenvalue problem Av = λ |A| v is bounded by 1, i.e.,
(2.5)
|(v, Av)| ≤ (v, |A| v) ∀v ∈ Rn .
Now, we recall that the spectra of matrices T |A| and T A are given by the generalized eigenvalue problems |A| v = μT −1 v and Av = λT −1 v, respectively, and introduce
the corresponding Rayleigh quotients
A700
EUGENE VECHARYNSKI AND ANDREW V. KNYAZEV
ψ(v) ≡
(2.6)
(v, Av)
(v, |A| v)
, φ(v) ≡
, v ∈ Rn .
−1
(v, T v)
(v, T −1 v)
Let us fix any index j ∈ {1, 2, . . . , n} and denote by S an arbitrary subspace of
Rn such that dim(S) = j. Since inequality (2.5) also holds on S, using (2.6) we write
−ψ(v) ≤ φ(v) ≤ ψ(v), v ∈ S.
(2.7)
Moreover, taking the maxima in vectors v ∈ S, and after that the minima in subspaces
S ∈ S j = {S ⊆ Rn : dim(S) = j}, of all parts of (2.7) preserves the inequalities, so
(2.8)
min max(−ψ(v)) ≤ minj max φ(v) ≤ minj max ψ(v).
S∈S j v∈S
S∈S
v∈S
S∈S
v∈S
By the Courant–Fischer theorem (see, e.g., [24, 29]) for the Rayleigh quotients ±ψ(v)
and φ(v) defined in (2.6), we conclude from (2.8) that
−μn−j+1 ≤ λj ≤ μj .
Recalling that j has been arbitrarily chosen, we obtain the following bounds on the
eigenvalues of T A:
−μn−j+1
0
(2.9)
≤
<
λj
λj
<
≤
0,
μj ,
j = 1, . . . , p;
j = p + 1, . . . , n.
Next, in order to derive nontrivial upper and lower bounds for the p negative and
n − p positive eigenvalues λj in (2.9), we use the fact that eigenvalues ξj and ζj of the
−1
generalized eigenvalue problems |A| v = ξT v and A−1 v = ζT v are the reciprocals
of the eigenvalues of the problems |A| v = μT −1 v and Av = λT −1 v, respectively, i.e.,
0 < ξ1 =
(2.10)
1
1
1
≤ ξ2 =
≤ · · · ≤ ξn =
μn
μn−1
μ1
and
(2.11)
ζ1 =
1
1
1
1
≤ · · · ≤ ζp =
< 0 < ζp+1 =
≤ · · · ≤ ζn =
.
λp
λ1
λn
λp+1
Similar to (2.5),
(v, A−1 v) ≤ (v, |A|−1 v) ∀v ∈ Rn .
Thus, we can use the same arguments as those following (2.5) to show that relations (2.7) and (2.8), with a fixed j ∈ {1, 2, . . . , n}, also hold for
(2.12)
ψ(v) ≡
−1
(v, A−1 v)
(v, |A| v)
, φ(v) ≡
, v ∈ Rn ,
(v, T v)
(v, T v)
where ψ(v) and φ(v) are now the Rayleigh quotients of the generalized eigenvalue
−1
problems |A| v = ξT v and A−1 v = ζT v, respectively. The Courant–Fischer theorem
for ±ψ(v) and φ(v) in (2.12) allows us to conclude from (2.8) that
−ξn−j+1 ≤ ζj ≤ ξj .
ABSOLUTE VALUE PRECONDITIONING
A701
Given the arbitrary choice of j in the above inequality, by (2.10)–(2.11) we get the
following bounds on the eigenvalues of T A:
(2.13)
−1/μp−j+1
0
≤
<
1/λj
1/λj
<
≤
0,
1/μj−p ,
j = 1, . . . , p;
j = p + 1, . . . , n.
Combining (2.9) and (2.13), we obtain (2.4).
Theorem 2.3 suggests two useful implications given by the corresponding corollaries below. In particular, the following result describes Λ(T A), i.e., the spectrum of
the preconditioned matrix T A, in terms of δ0 and δ1 in (2.3).
Corollary 2.4. Given a nonsingular symmetric indefinite A ∈ Rn×n , an SPD
T ∈ Rn×n , and constants δ1 ≥ δ0 > 0 satisfying (2.3), we have
(2.14)
Λ(T A) ⊂ [−δ1 , −δ0 ] [δ0 , δ1 ] ,
where Λ(T A) is the spectrum of T A.
Proof. The proof follows directly from (2.3) and (2.4) with j = 1, p, p+1, n.
The next corollary shows that the presence of reasonably populated clusters of
eigenvalues in the spectrum of T |A| guarantees the occurrence of corresponding clusters in the spectrum of the preconditioned matrix T A.
Corollary 2.5. Given a nonsingular symmetric indefinite A ∈ Rn×n and an
SPD T ∈ Rn×n , let μl ≤ μl+1 ≤ · · · ≤ μl+k−1 be a sequence of k eigenvalues of
T |A|, where 1 ≤ l < l + k − 1 ≤ n and τ = |μl − μl+k−1 |. Then, if k ≥ p + 2,
the k − p positive eigenvalues λl+p ≤ λl+p+1 ≤ · · · ≤ λl+k−1 of T A are such that
|λl+p − λl+k−1 | ≤ τ . Also, if k ≥ (n − p) + 2, the k − (n − p) negative eigenvalues
λn−k−l+2 ≤ · · · ≤ λp−l ≤ λp−l+1 of T A are such that |λn−k−l+2 − λp−l+1 | ≤ τ .
Proof. The proof follows directly from bounds (2.4).
Corollary 2.4 implies that the ratio δ1 /δ0 ≥ 1 of the constants from (2.3) measures
the quality of the AV preconditioner T . Indeed, the convergence speed of the preconditioned minimal residual method is determined by the spectrum of T A, primarily
by the intervals of the right-hand side of inclusion (2.14). Additionally, Corollary 2.5
prompts that a “good” AV preconditioner should ensure clusters of eigenvalues in the
spectrum of T |A|. This implies the clustering of eigenvalues of the preconditioned
matrix T A, which has a favorable effect on the convergence behavior of a polynomial
iterative method, such as PMINRES.
In the next section, we construct an example of the AV preconditioner for a
particular model problem. We apply the MG techniques.
3. MG AV preconditioning for a model problem. Let us consider the real
boundary value problem
(3.1)
−Δu(x, y) − c2 u(x, y) = f (x, y), (x, y) ∈ Ω = (0, 1) × (0, 1), u|Γ = 0,
where Δ = ∂ 2 /∂x2 + ∂ 2 /∂y2 is the Laplace operator and Γ denotes the boundary of
Ω. Problem (3.1) is a particular instance of the Helmholtz equation with Dirichlet
boundary conditions, where c > 0 is a wave number.
After introducing a uniform grid of size h in both directions and using the standard
five-point finite-difference stencil to discretize continuous problem (3.1), one obtains
the corresponding discrete problem
(3.2)
(L − c2 I)x = b,
A702
EUGENE VECHARYNSKI AND ANDREW V. KNYAZEV
where A ≡ L − c2 I represents a discrete negative Laplacian L (later called “Laplacian”), satisfying the Dirichlet boundary condition shifted by a scalar c2 .
The common rule of thumb (see, e.g., [13, 22]) for discretizing (3.1) is
ch ≤ π/5.
(3.3)
Below, we call (3.2) the model problem. We assume that the shift c2 is different from
any eigenvalue of the Laplacian and is greater than the smallest but less than the
largest eigenvalue. Thus, the matrix L − c2 I is nonsingular symmetric indefinite. In
the following subsection, we apply the idea of the AV preconditioning to construct an
MG AV preconditioner for system (3.2).
While our main focus throughout the paper is on the two-dimensional (2D) problem (3.1), in order to simplify presentation of theoretical analysis, we also refer to the
one-dimensional (1D) analogue
−u (x) − c2 u(x) = f (x), u(0) = u(1) = 0.
(3.4)
The conclusions drawn from (3.4), however, remain qualitatively the same for the 2D
problem of interest, which we test numerically.
3.1. Two-grid AV preconditioner. Along with the fine grid of mesh size h
underlying problem (3.2), let us consider a coarse grid of mesh size H > h. We
denote the discretization of the Laplacian on this grid by LH , and IH represents the
identity operator
of the corresponding dimension. We assume that the exact fine-level
its inverse
absolute value L − c2 I and
are not computable, whereas the inverse of
the coarse-level operator LH − c2 IH can be efficiently constructed. In the two-grid
framework, we use the subscript H to refer to the quantities defined on the coarse
grid. No subscript
is used for denoting the fine grid quantities.
While L − c2 I is not available, let us assume that we have its SPD approximation
B, i.e., B ≈ |L − c2 I| and B = B ∗ > 0. The operator B can be given in the explicit
matrix form or through the action on a vector. We suggest the following general
scheme as a two-grid AV preconditioner for model problem (3.2).
Algorithm 3.1 (the two-grid AV preconditioner). Input: r, B ≈ |L − c2 I|.
Output: w.
1. Presmoothing. Apply ν smoothing steps, ν ≥ 1:
(3.5)
w(i+1) = w(i) + M −1 (r − Bw(i) ), i = 0, . . . , ν − 1, w(0) = 0,
where M defines a smoother. Set wpre = w(ν) .
2. Coarse grid correction. Restrict (R) r − Bwpre to the coarse grid, apply
LH − c2 IH −1 , and prolongate (P ) to the fine grid. This delivers the coarse
grid correction, which is added to wpre :
(3.6)
(3.7)
−1
wH = LH − c2 IH R (r − Bwpre ) ,
wcgc = wpre + P wH .
3. Postsmoothing. Apply ν smoothing steps:
(3.8)
w(i+1) = w(i) + M −∗ (r − Bw(i) ), i = 0, . . . , ν − 1, w(0) = wcgc ,
where M and ν are the same as in step 1. Return w = wpost = w(ν) .
ABSOLUTE VALUE PRECONDITIONING
A703
In (3.6) we assume that LH − c2 IH is nonsingular, i.e., c2 is different from
any eigenvalue of LH . The presmoother is defined by the nonsingular M , while the
postsmoother is delivered by M ∗ . Note that the (inverted) absolute value appears only
on the coarse grid, while the fine grid computations are based on the approximation B.
It is immediately seen that if B = |L − c2 I|, Algorithm 3.1 represents a formal
two-grid cycle [8, 39] for system
L − c2 I z = r.
(3.9)
Note that the introduced scheme is rather general in that different choices of approximations B and smoothers M lead to different preconditioners. We address these
choices in more detail in the following subsections.
It can be verified that the AV preconditioner given by Algorithm 3.1 implicitly
constructs a mapping r → w = Ttg r, where the operator is
−1 ν
ν (3.10)
Ttg = I − M −∗ B P LH − c2 IH R I − BM −1 + F
ν
ν
with F = B −1 − (I − M −∗ B) B −1 I − BM −1 . The fact that the constructed
preconditioner T = Ttg is SPD follows directly from the observation that the first
term in (3.10) is SPD provided that P = αR∗ for some nonzero scalar α, while the
second term F is SPD if the spectral radii of I − M −1 B and I − M −∗ B are less than 1.
The latter condition requires the pre- and postsmoothing iterations (3.5) and (3.8) to
represent convergent methods for By = r. Note that the above argument essentially
repeats the one used to justify symmetry and positive definiteness of a preconditioner
based on the standard two-grid cycle for an SPD system; see, e.g., [5, 38].
In this paper we consider two different choices of the approximation B. The
first choice is given by B = L, i.e., it is suggested to approximate the absolute value
|L − c2 I| by the Laplacian L. The second choice is delivered by B = pm (L − c2 I),
where pm is a polynomial of degree at most m such that pm (L − c2 I) ≈ |L − c2 I|.
3.2. Algorithm 3.1 with B = L. If B = L, Algorithm 3.1 can be regarded as
a step of a standard two-grid method [8, 39] applied to the Poisson equation
(3.11)
Ly = r,
modified by replacing the operator LH by |LH − c2 IH | on the coarse grid. The
question remains of whether the algorithm delivers a form of an approximate solve
for absolute value problem (3.9) and hence is suitable for AV preconditioning of (3.2).
To be able to answer this question, we analyze the propagation of the initial error
2 −1
r of (3.9) under the action of the algorithm.
eAV
0 = |L − c I|
We start by relating errors of (3.9) and (3.11).
Lemma 3.1. Given a vector w, consider errors eAV (w) = |L − c2 I|−1 r − w and
eP (w) = L−1 r − w for (3.9) and (3.11), respectively. Then
(3.12)
eAV (w) = eP (w) + (c2 I − Wp )L−1 |L − c2 I|−1 r,
where Wp = 2Vp |Λp |Vp∗ , Vp is the matrix of eigenvectors of L − c2 I corresponding to
the p negative eigenvalues λ1 ≤ . . . ≤ λp < 0, and |Λp | = diag {|λ1 |, . . . , |λp |}.
Proof. Observe that for any w,
eAV (w) = |L − c2 I|−1 r − w = |L − c2 I|−1 r + (eP (w) − L−1 r)
= eP (w) + |L − c2 I|−1 L−1 (L − |L − c2 I|)r.
Denoting A = L − c2 I, we use the expression |A| = A − 2Vp Λp Vp∗ to get (3.12).
A704
EUGENE VECHARYNSKI AND ANDREW V. KNYAZEV
Algorithm 3.1 transforms the initial error eP0 = L−1 r of (3.11) into
(3.13)
eP = S2ν KS1ν eP0 ,
where S1 = I − M −1 L and S2 = I − M −∗ L are pre- and postsmoothing operators,
K = I − P |LH − c2 IH |−1 RL corresponds to the coarse grid correction step, and
eP = L−1 r − wpost . Denoting the error of absolute value system (3.9) after applying
Algorithm 3.1 by eAV = |L − c2 I|−1 r − wpost and observing that eP0 = |L − c2 I|L−1 eAV
0 ,
by (3.12)–(3.13) we obtain
(3.14)
eAV = S2ν KS1ν |L − c2 I| + c2 I − Wp L−1 eAV
0 .
The last expression gives an explicit form of the desired error propagation operator,
which we denote by G:
(3.15)
G = S2ν KS1ν |L − c2 I| + c2 I − Wp L−1 .
Below, as a smoother, we use a simple Richardson’s iteration, i.e., S1 = S2 =
I − τ L, where τ is an iteration parameter. The restriction R is given by the full
weighting and the prolongation P by the standard piecewise linear interpolation;
see [8, 39].
At this point, in order to simplify further presentation, let us refer to the 1D
analogue (3.4)
of model problem (3.1). In this case, the matrix L is tridiagonal:
L = tridiag −1/h2 , 2/h2 , −1/h2 . We assume that n, the number of interior grid
nodes, is odd: h = 1/(n + 1). The coarse grid is then obtained by dropping the oddnumbered nodes. We denote the size of the coarse grid problem by N = (n + 1)/2 − 1;
H = 1/(N + 1) = 2h. The tridiagonal matrix LH denotes the discretization of the
1D Laplacian on the coarse level.
Recall that the eigenvalues of L are θj = h42 sin2 jπh
2 with corresponding eigenvec√
n
tors vj = 2h [sin ljπh]l=1 . Similarly, the eigenvalues of LH are θjH = H42 sin2 jπH
2 ,
√
N
H
and the coarse grid eigenvectors are denoted by vj = 2H [sin ljπH]l=1 . It is clear
that operators L − c2 I and LH − c2 IH have the same sets of eigenvectors as L and
H
2
LH with eigenvalues tj = θj − c2 and tH
j = θj − c , respectively.
n
Let eAV
j be the expansion of the initial error in the eigenbasis of L.
0 =
j=1 αj v
n
Since eAV = GeAV
=
0
j=1 αj (Gvj ), we are interested in the action of the error
propagation operator (3.15) on the eigenmodes vj .
The action of the operators R and P on vj and vjH , respectively, is well known;
see, e.g., [8, pp. 80–81]. Thus, it is easy to obtain the following expression for Kvj :
(3.16)
⎧
⎪
θj
⎪
4 θj
⎪
1 − cj H vj + s2j c2j H vn+1−j ,
j = 1, . . . , N,
⎪
⎪
⎪
|tj |
|tj |
⎨
Kvj = vj ,
j = N + 1,
⎪
⎪
⎪
θj
θj
⎪
4
H
⎪
vn+1−j
, j = N + 2, . . . , n.
vj + s2j c2j H
⎪
⎩ 1 − cj |tH
|
|t
|
n+1−j
n+1−j
jπh
Here, cj = cos jπh
2 and sj = sin 2 . Since vj are the eigenvectors of S1 = S2 = I −τ L,
2
−1
L − c I, L , and Wp , (3.15) leads to explicit expressions for Gvj .
ABSOLUTE VALUE PRECONDITIONING
A705
Theorem 3.2. Let c2 < θN +1 = 2/h2 . Then the error propagation operator G
in (3.15) acts on the eigenvectors vj of the 1D Laplacian as follows:
⎧
(11)
(12)
⎪
⎨gj vj + gj vn+1−j , j = 1, . . . , N,
j = N + 1,
(3.17)
Gvj = gj vj ,
⎪
⎩g (21) v + g (22) v
j
n+1−j , j = N + 2, . . . , n,
j
j
where
(3.18)
(11)
gj
(3.19)
gj
(12)
2ν
= (1 − τ θj )
= (1 − τ θj )ν s2j c2j
gj = (1 − τ θj )2ν
(3.20)
(3.21)
(21)
gj
(3.22)
gj
(22)
1−
2ν
= (1 − τ θj )
βj =
θj
|tH
j |
βj
|tj | c2
+
−
,
θj
θj
θj
|tj |
(1 − τ θn+1−j )ν ,
|tH
|
j
|tj | c2
+
,
θj
θj
1−
= (1 − τ θj )ν s2j c2j
and
c4j
c4j
θj
|tH
n+1−j |
|tj | c2
+
,
θj
θj
|tj |
(1 − τ θn+1−j )ν ;
|tH
|
n+1−j
2
2(c − θj ),
0,
θj < c 2 ,
.
θj > c 2
Theorem 3.2 implies that for relatively small shifts, Algorithm 3.1 with B = L
and a proper choice of τ and ν reduces the error of (3.9) in the directions of almost
all eigenvectors vj . In a few directions, however, the error may be amplified. These
directions are given by the smooth eigenmodes associated with θj that are close to c2
on the right, as well as with θj that are distant from c2 on the left. The number of
the latter, if any, is small if ch is sufficiently small and becomes larger as ch increases.
Indeed, let τ = h2 /3 so that |1 − τ θj | < 1 for all j and |1 − τ θj | < 1/3 for j > N .
This choice of the parameter provides the least uniform bound for |1 − τ θj | that
correspond to the oscillatory eigenmodes [34, p. 415]. It is then readily seen that (3.19)
and (3.22) can be made arbitrarily small within a reasonably small number ν of
smoothing steps. Similarly, (3.20) and (3.21) can be made arbitrarily close to c2 /θj <
1. If c2 << θN +1 , then c2 /θj in (3.20) and (3.21) is close to zero. Thus, Theorem 3.2
shows that for relatively small shifts, smoothing provides small values of (3.19)–(3.22)
and hence damps the oscillatory part of the error. Note that the damping occurs even
though the smoothing is performed with respect to (3.11), not (3.9).
Now let us consider (3.18). Theorem 3.2 shows that if c2 is close to an eigenvalue
H
θj of the coarse-level Laplacian, i.e., if tH
j ≈ 0, then the corresponding reduction coefficient (3.18) can be large. This means that Algorithm 3.1 with B = L has a potential
difficulty of amplifying the error in the directions of a few smooth eigenvectors.
A similar effect is known to appear for standard MG methods applied to Helmholtz
type problems; see [7, 13]. Below, we analyze (3.18) in more detail.
Let θj > c2 . Then, using the relation θjH = c2j θj , we can write (3.18) as
1
c2
c2
(11)
2ν
2
gj = (1 − τ θj )
.
1 − cj
+
1
−
H
2
θj
θj
|1 − c /θj |
A706
EUGENE VECHARYNSKI AND ANDREW V. KNYAZEV
(11)
Here, it is easy to see that as c2 /θjH → 0, gj → (1 − τ θj )2ν s2j < 1/2, meaning that
the smooth eigenmodes corresponding to θj away from c2 on the right are well damped.
If θj < c2 , then (3.18) takes the form
2
2
4
/θ
−
c
−
c
c
c2
c2
j
(11)
j
j
2ν
gj = (1 − τ θj )
−1 + 2−
.
c2 /θj − c2j
θj
θj
Since c2j ∈ (1/2, 1), for any c2 /θj > 1, we can obtain the bound
c2 /θj − c2j − c4j
c2 /θj − 2
c2 /θj − 3/4
≤
.
≤
c2 /θj − 1
c2 /θj − c2j
c2 /θj − 1/2
Additionally, 3−2ν < (1 − τ θj )2ν < 1. Thus,
(11)
lj < g j
<
3(c2 /θj ) − 1
,
4(c2 /θj ) − 2
where lj = 0 if 1 < c2 /θj ≤ 2 and lj = 2 − c2 /θj if c2 /θj > 2.
(11)
The inequality implies that |gj | < 1 for 1 < c2 /θj ≤ 3, i.e., the algorithm
reduces the error in the directions of several smooth eigenvectors associated with θj
(11)
to the left of c2 . At the same time, we note that as c2 /θj → ∞, gj → ∞, i.e., the
smooth eigenmodes corresponding to θj that are distant from c2 on the left can be
amplified. Clearly, if ch is sufficiently small, then the number of such error components
is not large (or is none) and grows as ch increases.
The above analysis shows that Algorithm 3.1 with B = L indeed represents a
solve for (3.9), where the solution is approximated everywhere, possibly except for a
subspace of a small dimension. In the context of preconditioning, this translates into
the fact that the preconditioned matrix has a spectrum clustered around 1 and −1
with a few outliers generated by the amplification of the smooth eigenmodes. If the
shift is sufficiently small, the number of such outliers is not large, which only slightly
delays the convergence of the outer PMINRES iterations and does not significantly
affect the efficiency of the overall scheme.
3.3. Algorithm 3.1 with B = pm (L − c2 I). The analysis of the previous
subsection suggests that the quality of Algorithm 3.1 with B = L may deteriorate
as ch increases. This result is not surprising, since for larger ch the relation L ≈
|L − c2 I| becomes no longer meaningful. Below we introduce a different approach for
approximating the fine grid absolute value. In particular, we consider constructing
polynomial approximations B = pm (L − c2 I), where pm (λ) is a polynomial of degree
at most m > 0 such that pm (L − c2 I) ≈ |L − c2 I|.
Let us first refer to the ideal particular case, where pm (L − c2 I) = |L − c2 I|. This
can happen, e.g., if pm (λ) is an interpolating polynomial of f (λ) = |λ| on the spectrum
of L − c2 I, m = n − 1. In such a situation, Algorithm 3.1 with B = pm (L − c2 I)
results in the following transformation of the initial error:
(3.23)
eAV = S̄2ν K̄ S̄1ν eAV
0 ,
where S̄1 = I − M −1 |L − c2 I| and S̄2 = I − M −∗ |L − c2 I| are pre- and postsmoothing
operators and K̄ = I − P |LH − c2 IH |−1 R|L − c2 I| corresponds to the coarse grid
correction step. The associated error propagation operator is further denoted by Ḡ,
(3.24)
Ḡ = S̄2ν K̄ S̄1ν .
ABSOLUTE VALUE PRECONDITIONING
A707
For clarity, we again consider the 1D counterpart (3.4) of the model problem.
As a smoother, we choose Richardson’s iteration with respect to absolute value system (3.9), i.e., S̄1 = S̄2 = I − τ |L − c2 I|. It is important to note here that the
eigenvalues |tj | of the absolute value operator are, in general, no longer ascendingly
ordered with respect to j as is the case for θj ’s and tj ’s. Moreover, in contrast to
L and L − c2 I, the top part of the spectrum of |L − c2 I| may be associated with
both smooth and oscillatory eigenmodes. In particular, this means that Richardson’s
iteration may fail to properly eliminate the oscillatory components of the error, which
is an undesirable outcome of the smoothing procedure. To avoid this, we require that
|t1 | < tN +1 . It is easy to verify that the latter condition is fulfilled if
ch < 1.
(3.25)
Note that (3.25) automatically holds if discretization rule (3.3) is enforced. Repeating
the above argument for the 2D case also leads to (3.25).
Let the restriction and prolongation operators R and P be the same as in the
previous subsection. Similar to (3.16), we obtain an explicit expression for the action
of the coarse grid correction operator K̄ on eigenvectors vj :
⎧
⎪
|t
|
|tj |
⎪
j
⎪
1 − c4j H vj + s2j c2j H vn+1−j ,
j = 1, . . . , N,
⎪
⎪
⎪
|tj |
|tj |
⎨
j = N + 1,
vj ,
(3.26) K̄vj = ⎪
⎪
⎪
|tj |
|tj |
⎪
4
H
⎪
, j = N + 2, . . . , n.
vj + s2j c2j H
vn+1−j
⎪
⎩ 1 − cj |tH
|
|t
|
n+1−j
n+1−j
The following theorem is the analogue of Theorem 3.2.
Theorem 3.3. The error propagation operator Ḡ in (3.24) acts on the eigenvectors vj of the 1D Laplacian as follows:
⎧
(11)
(12)
⎪
⎨ḡj vj + ḡj vn+1−j , j = 1, . . . , N,
j = N + 1,
(3.27)
Ḡvj = ḡj vj ,
⎪
⎩ḡ (21) v + ḡ (22) v
j
n+1−j , j = N + 2, . . . , n,
j
j
where
(3.28)
(11)
ḡj
(3.29)
ḡj
(12)
2ν
= (1 − τ |tj |)
1−
= (1 − τ |tj |)ν s2j c2j
c4j
(3.31)
(21)
ḡj
(3.32)
ḡj
(22)
2ν
= (1 − τ |tj |)
1−
= (1 − τ |tj |)ν s2j c2j
,
|tj |
(1 − τ |tn+1−j |)ν ,
|tH
j |
ḡj = (1 − τ |tj |)2ν ,
(3.30)
|tj |
|tH
j |
c4j
|tj |
|tH
n+1−j |
,
|tj |
(1 − τ |tn+1−j |)ν .
|tH
n+1−j |
We conclude from Theorem 3.3 that in the ideal case where pm (L−c2 I) = |L−c2 I|,
Algorithm 3.1 with B = pm (L − c2 I) and a proper choice of τ and ν reduces the error
A708
EUGENE VECHARYNSKI AND ANDREW V. KNYAZEV
of system (3.9) in the directions of all eigenvectors vj , possibly except for a few that
correspond to θj close to the shift c2 . Unlike in the case of Algorithm 3.1 with B = L,
as ch grows, no amplified error components appear in the directions of eigenvectors
associated with θj distant from c2 on the left. This suggests that Algorithm 3.1 with
B = pm (L − c2 I) = |L − c2 I| provides a more accurate solve for (3.9) with larger ch.
To see this, let us first assume that τ = h2 /(3 − c2 h2 ). Since (3.25) implies that
|tj | = tj for j > N , this choice is known to give the smallest uniform bound on |1−τ |tj ||
corresponding to the oscillatory eigenmodes vj , which is |1 − τ |tj || < 1/(3 − ch) < 1/2
with the last inequality resulting from (3.25). Hence, coefficients (3.29)–(3.32) can be
reduced within a reasonably small number ν of smoothing steps.
Next, we note that (3.28), which is not substantially affected by smoothing, can
be large if c2 is close to θjH , i.e., if tH
j ≈ 0. At the same time, we can write (3.28) as
2 2
c
s
(11)
j
gj = (1 − τ |tj |)2ν 1 − c2j 1 + H ,
tj (11)
which shows that |gj | approaches (1−τ |tj |)2ν s2j < 1/2 as |tH
j | increases, i.e., smooth
error components associated with θj away from c2 are well damped.
Thus, if used as a preconditioner, Algorithm 3.1 with B = pm (L−c2 I) = |L−c2 I|
aims at clustering the spectrum of the preconditioned matrix around 1 and −1, with
a few possible outliers that result from the amplification of the smooth eigenmodes
associated with θj close to c2 . Unlike in the case where B = L, the increase of ch
does not additionally amplify the smooth error components distant from c2 on the
left. Therefore, Algorithm 3.1 with B = pm (L − c2 I) = |L − c2 I| can be expected to
provide a more accurate preconditioner for larger shifts.
Although our analysis targets the ideal but barely feasible case where pm (L −
c2 I) = |L − c2 I|, it motivates the use of polynomial approximations pm (L − c2 I) ≈
|L−c2 I| and provides a theoretical insight into the superior behavior of such an option
for larger ch. In the rest of this subsection we describe a method for constructing such
polynomial approximations. Our approach is based on the finding that the problem
is easily reduced to constructing polynomial filters.
We start by introducing the step function
1, λ ≥ α,
hα (λ) =
0, λ < α,
where α is a real number, and noting that sign(λ) = 2h0 (λ) − 1, so that
(3.33)
|L − c2 I| = 2h0 (L − c2 I) − I L − c2 I .
Here h0 (L − c2 I) = V h0 (Λ)V ∗ , where V is the matrix of eigenvectors of L − c2 I and
h0 (Λ) = diag{0, . . . , 0, 1, . . . , 1} is obtained by applying the step function h0 (λ) to the
diagonal entries of the matrix Λ of the associated eigenvalues. Clearly the number of
zeros on the diagonal of h0 (Λ) equals the number of negative eigenvalues of L − c2 I.
Let qm−1 (λ) be a polynomial of degree at most (m − 1) such that qm−1 (λ) approximates h0 (λ) on the interval [a, b], where a and b are the lower and upper bounds
on the spectrum of L − c2 I, respectively. In order to construct an approximation
pm (L − c2 I) of |L − c2 I|, we replace the step function h0 (L − c2 I) in (3.33) by the
polynomial qm−1 (L − c2 I). Thus,
(3.34)
|L − c2 I| ≈ pm (L − c2 I) = 2qm−1 (L − c2 I) − I L − c2 I .
ABSOLUTE VALUE PRECONDITIONING
A709
The matrix L − c2 I is readily available on the fine grid. Therefore, we have
reduced the problem of evaluating the polynomial approximation pm of the absolute value operator to constructing a polynomial qm−1 that approximates the step
function h0 . More specifically, since Algorithm 3.1 can be implemented without the
explicit knowledge of the matrix B, i.e., B can be accessed only through its action
on a vector, we need to construct approximations of the form qm−1 (L − c2 I)v to
h0 (L − c2 I)v, where v is a given vector.
The task of constructing qm−1 (L − c2 I)v ≈ h0 (L − c2 I)v represents an instance of
polynomial filtering, which is well known; see, e.g., [11, 32, 45]. In this context, due to
the property of filtering out certain undesirable eigencomponents, the step function
h0 is called a filter function. The approximating polynomial qm−1 is referred to as a
polynomial filter.
State-of-the-art polynomial filtering techniques such as [32] would first replace
the discontinuous step function h0 (λ) by a smooth approximation on [a, b] and then
approximate the latter by a polynomial in the least-squares sense. In this paper, we
follow a simpler approach based on the direct approximation of h0 (λ) using Chebyshev
polynomials [30, 31]. The constructed polynomial qm−1 allows defining qm−1 (L −
c2 I)v ≈ h0 (L−c2 I)v and hence pm (L−c2 I)v ≈ |L−c2 I|v. Thus, the entire procedure
provides means to replace a matrix-vector product with the unavailable |L − c2 I| by,
essentially, a few multiplications with L − c2 I. As we further show, the degree m
of the approximating polynomial can be kept reasonably low. Moreover, in the MG
framework discussed in the next subsection, the algorithm has to be invoked only on
sufficiently coarse grids.
3.4. The MG AV preconditioner. Now let us consider a hierarchy of s + 1
grids numbered by l = s, s − 1, . . . , 0, with the corresponding mesh sizes {hl } in
decreasing order (hs = h corresponds to the finest grid and h0 to the coarsest). For
each level l we define the discretization Ll − c2 Il of the differential operator in (3.1),
where Ll is the Laplacian on grid l and Il is the identity of the same size.
In order to extend the two-grid AV preconditioner
given by Algorithm 3.1 to the
multigrid, instead of inverting the absolute value LH − c2 IH in (3.6), we recursively
apply the algorithm to the restricted vector R(r − Bwpre ). This pattern is then
followed in the V-cycle fashion on all levels with the inversion of the absolute value
of the shifted Laplacian on the coarsest grid. The matrix B on level l is denoted by
Bl . Each Bl is assumed to be SPD and is expected to approximate |Ll − c2 Il |.
In the previous subsections we have considered two choices of B for the two-grid
preconditioner in Algorithm 3.1. In the MG framework, these choices give Bl = Ll
and Bl = pml (Ll − c2 Il ), where pml is a polynomial of degree at most ml on level l.
The advantage of the first option, Bl = Ll , is that it can be easily constructed
and the application of Bl to a vector is inexpensive even if the size of the operator
is very large. According to our analysis for the 1D model problem in subsection 3.2,
the approach is suitable for chl sufficiently small. Typically this is a case for l corresponding to finer grids. However, chl increases with every new level. This may result
in the deterioration of accuracy of the overall MG preconditioning scheme, unless the
size of the coarsest level is kept sufficiently large.
The situation is different for the second option Bl = pml (Ll − c2 Il ). In this case,
applications of Bl may be expensive on finer grids because they require a sequence
of matrix-vector multiplications with large shifted Laplacian operators. However, on
coarser levels, i.e., for larger chl , this is not restrictive because the involved operators are significantly decreased in size compared to the finest level. Additionally,
A710
EUGENE VECHARYNSKI AND ANDREW V. KNYAZEV
as suggested by the analysis in subsection 3.3, if pml (Ll − c2 Il ) represent reasonable
approximations of |Ll − c2 Il | on levels l, one can expect a higher accuracy of the whole
preconditioning scheme compared to the choice Bl = Ll .
Our idea is to combine the two options. Let δ ∈ (0, 1) be a “switching” parameter,
where for finer grids chl < δ. We choose
Ll ,
chl < δ,
(3.35)
Bl =
pml (Ll − c2 Il ), chl ≥ δ.
The polynomials pml (Ll − c2 Il ) are accessed through their action on a vector.
Summarizing our discussion, if started from the finest grid l = s, the following
scheme gives the multilevel extension of the two-grid AV preconditioner defined by
Algorithm 3.1. The subscript l is introduced to match quantities to the corresponding
grid. We assume that the parameters δ, ml , νl and the smoothers Ml are prespecified.
Algorithm 3.2 (AV-MG(rl ): the MG AV preconditioner). Input rl . Output wl .
1. Set Bl by (3.35).
2. Presmoothing. Apply νl smoothing steps, νl ≥ 1:
(i+1)
wl
(3.36)
(i)
(i)
(0)
= wl + Ml−1 (rl − Bl wl ), i = 0, . . . , νl − 1, wl
= 0,
(ν)
where Ml defines a smoother on level l. Set wlpre = wl .
3. Coarse grid correction. Restrict (Rl−1 ) rl −Bl wlpre to the grid l−1, recursively
apply AV-MG, and prolongate (Pl ) back to the fine grid. This delivers the
coarse grid correction added to wlpre :
wl−1 =
(3.37)
L0 − c2 I0 −1 R0 (r1 − B1 wpre ) , l = 1,
1
AV-MG (Rl−1 (rl − Bl wlpre )) ,
l > 1;
wlcgc = wlpre + Pl wl−1 .
(3.38)
4. Postsmoothing. Apply νl smoothing steps:
(3.39)
(i+1)
wl
(i)
(i)
(0)
= wl + Ml−∗ (rl − Bl wl ), i = 0, . . . , νl − 1, wl
= wlcgc ,
(ν )
where Ml and νl are the same as in step 2. Return wl = wlpost = wl l .
The described MG AV preconditioner implicitly constructs a mapping denoted
by r → w = Tmg r, where the operator T = Tmg has the structure
(3.40)
ν
ν
(s−1)
Tmg = I − M −∗ B P Tmg
R I − BM −1 + F
(s−1)
with F as in (3.10) and Tmg
(l)
(3.41)
Tmg
(0)
Tmg
=
=
defined according to the recursion
νl
νl
(l−1)
Il − Ml−∗ Bl Pl Tmg Rl−1 Il − Bl Ml−1 + Fl ,
L0 − c2 I0 −1 , l = 1, . . . , s − 1,
νl
νl
where Fl = Bl−1 − Il − Ml−∗ Bl Bl−1 Il − Bl Ml−1 .
The structure of the multilevel preconditioner T = Tmg in (3.40) is the same as
−1
that of the two-grid preconditioner T = Ttg in (3.10) with LH − c2 IH replaced by
(m−1)
in (3.41). Thus, the symmetry and positive
the recursively defined operator Tmg
ABSOLUTE VALUE PRECONDITIONING
A711
definiteness of T = Tmg follow from the same property of the two-grid operator
∗
through relations (3.41), provided that Pl = αl Rl−1
and the spectral radii of Il −
−1
−∗
Ml Bl and Il − Ml Bl are less than 1 throughout the coarser levels. We remark
that preconditioner (3.40)–(3.41) is nonvariable, i.e., it preserves the global optimality
of PMINRES.
The simplest possible approach for computing w0 in (3.37) is to explicitly construct |L0 − c2 I0 |−1 through the full eigendecomposition of the coarse-level Laplacian
and then apply it to R0 (r1 − B1 w1pre ). An alternative approach is to determine w0 as
a solution of the linear system (L0 − c2 I0 + 2V0 |Λ0 |V0∗ )w0 = R0 (r1 − B1 w1pre ), where
V0 is the matrix of eigenvectors associated with the negative eigenvalues of L0 − c2 I0
contained in the corresponding diagonal matrix Λ0 . In the latter case, the full eigendecomposition of L0 is replaced by the partial eigendecomposition targeting negative
eigenpairs, followed by a linear solve.
Since we use Richardson’s iteration with respect to pml (Ll − c2 Il ) as a smoother
on coarser grids, as motivated by the discussion in subsection 3.3, the guidance for
the choice of the coarsest grid is given by condition (3.25). More specifically, in the
context of the standard coarsening procedure (hl−1 = 2hl ), we select hierarchies of
grids satisfying chl < 1 for l = s, . . . , 1 and ch0 > 1. As shown in the next section,
even for reasonably large c2 , the coarsest-level problems are small.
The parameter δ in (3.35) should be chosen to ensure the balance between computational costs and the quality of the MG preconditioner. In particular, if δ is
reasonably large, then the choices of Bl are dominated by the option Bl = Ll ,
which is inexpensive but may not be suitable for larger shifts on coarser levels.
On the other extreme, if δ is close to zero, then the common choice corresponds to
Bl = pml (Ll − c2 Il ), which provides a better preconditioning accuracy for larger shifts
but may be too computationally intense on finer levels. In our numerical experiments,
we keep δ ∈ [1/3, 3/4].
As we demonstrate in the next section, the degrees ml of the occurring polynomials pml should not be large, i.e., only a few matrix-vector multiplications with Ll −c2 Il
are required to obtain satisfactory approximations of absolute value operators. For
properly chosen δ, these additional multiplications need to be performed on grids that
are significantly coarser than the finest grid, i.e., the involved matrices Ll − c2 Il are
orders of magnitude smaller than the original fine grid operator. As confirmed by
our numerical experiments, the overhead caused by the polynomial approximations
appears to be marginal and does not much affect the computational cost of the overall
preconditioning scheme.
4. Numerical experiments. This section presents a numerical study of the
MG preconditioner in Algorithm 3.2. Our goal here is twofold. On the one hand, the
reported numerical experiments serve as a proof of concept of the AV preconditioning
described in section 2. In particular, we show that the AV preconditioners can be
constructed at essentially the same cost as the standard preconditioning methods (MG
in our case). On the other hand, we demonstrate that the MG AV preconditioner
in Algorithm 3.2 combined with the optimal PMINRES iteration, in fact, leads to
an efficient and economical computational scheme, further called MINRES-AV-MG,
which outperforms several known competitive approaches for the model problem.
Let us briefly describe the alternative preconditioners used for our comparisons.
Throughout, we use MATLAB for our numerical examples.
The inverted Laplacian preconditioner. This strategy, introduced in [2],
is a representative of an SPD preconditioning for model problem (3.2), where the
A712
EUGENE VECHARYNSKI AND ANDREW V. KNYAZEV
Table 4.1
The largest problem sizes satisfying chl ≥ δ for different values of the shift c2 , “switching”
parameters δ, and the standard coarsening scheme hl−1 = 2hl . The last row (δ = 1) corresponds to
the sizes of the coarsest problems for different c2 .
δ = 1/3
δ = 1/2
δ = 3/4
δ=1
c2 = 300
961
961
225
225
c2 = 400
961
961
225
225
c2 = 1500
3969
3969
961
961
c2 = 3000
16129
3969
3969
961
c2 = 4000
16129
3969
3969
961
preconditioner is applied through solving systems Lw = r, i.e., T = L−1 . As has
been previously discussed, for relatively small shifts c2 , the Laplacian L constitutes
a good SPD approximation of |L − c2 I|. In this sense, the choice T = L−1 perfectly
fits, as a special case, into the general concept of the AV preconditioning presented in
section 2. We refer to PMINRES with T = L−1 as MINRES-Laplace.
Usually, one wants to solve the system Lw = r only approximately, i.e., use
T ≈ L−1 . This can be efficiently done, e.g., by applying the V-cycle of a standard
MG method [8, 39]. In our tests, however, we perform the exact solves using the
MATLAB “backslash” so that the reported results reflect the best possible convergence with the inverted Laplacian type preconditioning.
The indefinite MG preconditioner. We consider a standard V-cycle for problem (3.2). Formally, it can be obtained from Algorithm 3.2 by setting Bl = Ll − c2 Il
on all levels and replacing the first equality in (3.37) by the linear solve with L0 −c2 I0 .
The resulting MG scheme is used as a preconditioner for restarted GMRES and for
Bi-CGSTAB. We refer to these methods as GMRES(k)-MG and Bi-CGSTAB-MG,
respectively; k denotes the restart parameter. A thorough discussion of the indefinite
MG preconditioning for Helmholtz problems can be found, e.g., in [13].
In our tests, we consider the 2D model problem (3.2) corresponding to (3.1)
discretized on the grid of size h = 2−8 (the fine problem size n = 65025). The
exact solution x∗ and the initial guess x0 are randomly chosen. The right-hand-side
b = (L − c2 I)x∗ , which allows evaluating the actual errors along the steps of an
iterative method. All the occurring MG preconditioners are built upon the standard
coarsening scheme (i.e., hl−1 = 2hl ), restriction is based on the full weighting, and
prolongation is bases on piecewise multilinear interpolation [8, 39].
Let us recall that Algorithm 3.2 requires setting a parameter δ to switch between
Bl = L and pml (L−c2 I) on different levels; see (3.35). Assuming standard coarsening,
Table 4.1 presents the largest problem sizes corresponding to the condition chl ≥ δ for
a few values of δ and c2 . In other words, given δ and c2 , each cell of Table 4.1 contains
the largest problem size for which the polynomial approximation of |Ll − c2 Il | is
constructed. Unless otherwise explicitly stated, we set δ = 1/3. Note that according to
the discussion in subsection 3.4 (condition (3.25)), the row of Table 4.1 corresponding
to δ = 1 delivers the sizes n0 of the coarsest problems for different shift values.
Table 4.1 shows that the coarsest problems remain relatively small even for large
shifts. The polynomial approximations are constructed for coarser problems of significantly reduced dimensions, which in practical applications are negligibly small
compared to the original problem size.
As a smoother on all levels of Algorithm 3.2 we use Richardson’s iteration, i.e.,
Ml−1 ≡ τl Il . On the finer levels, where Bl = Ll , we choose τl = h2l /5 and νl = 1. On
the coarser levels, where Bl = pml (Ll − c2 Il ), we set τl = h2l /(5 − c2 h2l ) and νl = 5.
A713
ABSOLUTE VALUE PRECONDITIONING
2
2
Bi−CGSTAB−MG
GMRES(5)−MG
MINRES−AV−MG
MINRES−Laplace
Euclidean norm of error
0
10
−5
10
Shift value c = 400 (26 negative eigenvalues)
Euclidean norm of error
Shift value c = 300 (19 negative eigenvalues)
5
10
Bi−CGSTAB−MG
GMRES(10)−MG
MINRES−AV−MG
MINRES−Laplace
0
10
−5
10
−10
−10
10
10
0
10
20
30
40
50
60
0
40
60
80
Iteration number
Shift value c = 1500 (108 negative eigenvalues)
Shift value c = 3000 (222 negative eigenvalues)
Bi−CGSTAB−MG
GMRES(20)−MG
MINRES−AV−MG
MINRES−Laplace
10
0
10
−5
10
2
Euclidean norm of error
2
5
Euclidean norm of error
20
Iteration number
0
10
−5
10
Bi−CGSTAB−MG
GMRES(150)−MG
MINRES−AV−MG
MINRES−Laplace
−10
10
0
100
200
300
Iteration number
400
500
0
100
200
300
400
500
600
700
Iteration number
Fig. 4.1. Comparison of several preconditioned schemes; n = 65025.
Similar to the 1D case considered in subsections 3.2 and 3.3, both choices of
τl correspond to the optimal smoothing of oscillatory eigenmodes with respect to
the 2D operators Ll and |Ll − c2 Il |, respectively [8, 39]. Since pml (Ll − c2 Il ) only
approximates |Ll − c2 Il | in practice, the choice of τl on the coarser grids is likely to be
not optimal. Therefore, for chl ≥ δ, we have increased the number of smoothing steps
to 5. In all tests, the degrees ml of polynomials are set to 10. The intervals containing
Λ(Ll − c2 Il ), required for evaluating pml , are [−c2 , 8h−2
− c2 ]. The inverted coarse
l
grid absolute value |L0 − c2 I0 |−1 is constructed by the full eigendecomposition.
In Figure 4.1, we compare MINRES-AV-MG with the above introduced alternative preconditioned schemes for the model problem. Each plot corresponds to a
different shift value. The restart parameter k varies for all runs of GMRES(k)-MG,
increasing (left to right and top to bottom) as c2 grows from 300 to 3000. In our
tests, the size n0 of the coarsest problem in Algorithm 3.2 is 225 (Figure 4.1, top) and
961 (Figure 4.1, bottom); see Table 4.1 with δ = 1. The same n0 is used for the MG
preconditioner in the corresponding runs of GMRES(k)-MG and Bi-CGSTAB-MG.
Figure 4.1 shows that MINRES-AV-MG noticeably outperforms PMINRES with
the inverted Laplacian preconditioner. For smaller shifts (c2 = 300, 400), MINRESAV-MG is comparable, in terms of the iteration count, to GMRES(k)-MG and
Bi-CGSTAB-MG; k = 5, 10. For larger shifts (c2 = 1500, 3000); however, MINRESAV-MG provides a superior convergence behavior. In particular, the scheme exhibits
faster convergence than GMRES(k)-MG under less demanding storage requirements,
while Bi-CGSTAB-MG fails to converge (k = 50, 150).
If the polynomial approximations in Algorithm 3.2 appear only on sufficiently
coarse grids and the size n0 of the coarsest problem is relatively small, then the additional costs introduced by the coarser grid computations of the AV MG preconditioner
A714
EUGENE VECHARYNSKI AND ANDREW V. KNYAZEV
Table 4.2
Time comparison of the MG preconditioned schemes in Figure 4.1.
tB /tAV
tG /tAV
c2 = 300
1.1
1.4
c2 = 400
1.1
1.3
c2 = 1500
−
2.6
c2 = 3000
−
1.9
are negligible relative to the costs of operations on the finer grids, which are the same
as in the standard V-cycle for the indefinite problem. This means that the complexity
of Algorithm 3.2 is similar to that of the MG preconditioner in GMRES(k)-MG and
Bi-CGSTAB-MG.
To be more precise, in the tests reported in Figure 4.1, bottom, a single application of Algorithm 3.2 has required 15–20% more time than the indefinite MG
preconditioner, even though the polynomial approximations in Algorithm 3.2 have
been constructed for problem sizes as large as 16129 if c2 = 3000, which is of the same
order as the original problem size n. For larger problem sizes, the time difference
becomes negligible. For example, if h = 2−9 (n = 261121), Algorithm 3.2 results in
only 5% time increase, and the relative time difference becomes indistinguishable for
smaller h. The application of all the MG preconditioners in Figure 4.1, top, required
essentially the same time.
The above discussion suggests that our numerical comparison, based on the number of iterations, is representative. Additionally, in Table 4.2 we provide the time
comparisons for the MG preconditioned schemes. In particular, we measure the actual time required by the runs of MINRES-AV-MG (tAV ), GMRES(k)-MG (tG ), and
Bi-CGSTAB-MG (tB ) in Figure 4.1 and report the speed-ups.
We have observed that the performance of GMRES(k)-MG can be improved by
increasing the restart parameter. In Figure 4.1, however, the values k have been chosen
to balance storage and convergence behavior. In particular, we set k to be sufficiently
small, so that the storage required for GMRES(k)-MG is as close as possible to that
of MINRES-AV-MG, while the convergence of the method is not lost. Since BiCGSTAB-MG is based on a short-term recurrence, its storage expenses are similar to
that of MINRES-AV-MG.
The unsatisfactory performance of GMRES(k)-MG in Figure 4.1 can in part be
attributed to the observation that smoothing based on Richardson’s (or, more generally, Jacobi) iteration becomes increasingly unstable as grids coarsen. In particular,
as shown in [13], on the intermediate levels with chl ≥ 1/2 this smoothing scheme
strongly amplifies the smooth error eigenmodes. A straightforward remedy is to invoke
the coarse grid solve on the largest grid that fails to satisfy chl < 1/2.
In Table 4.3, we compare MINRES-AV-MG and GMRES(k)-MG with different
values of the restart parameter. We report the iteration counts required to reduce
Table 4.3
Number of iterations of MINRES-AV-MG (MINR) and GMRES(k)-MG (GMR(k)) required to
reduce the initial error by 10−8 ; n = 65025. The preconditioner in GMRES(k)-MG uses Richardson’s smoothing on levels chl < 1/2 and invokes the coarse grid solve on the level that follows.
Numbers in parentheses correspond to the right preconditioned GMRES(k)-MG. Dash denotes that
the method failed to converge within 1000 steps.
c2 = 1500
c2 = 3000
c2 = 4000
MINR
89
282
310
GMR(5)
29(44)
−(−)
−(−)
GMR(10)
20(22)
−(−)
−(−)
GMR(20)
16(18)
−(−)
−(−)
GMR(25)
16(18)
223(269)
−(−)
GMR(35)
16(18)
69(69)
395(471)
A715
ABSOLUTE VALUE PRECONDITIONING
the initial error by 10−8 for systems with c2 = 1500, 3000, and 4000. The indefinite
MG preconditioner in GMRES(k)-MG is configured to run Richardson’s smoothing
on grids chl < 1/2 and perform the coarse grid solve on the level that follows. We
test the indefinite MG preconditioner in the left (used so far) and right preconditioned
versions of GMRES(k).
The above described setting of the right preconditioned GMRES(k)-MG represents a special case of the Helmholtz solver introduced in [13]. In this paper, instead
of the “early” coarse grid solve, the MG preconditioning scheme performs further
coarsening on levels chl ≥ 1/2 with GMRES used as a smoother. Since the resulting
preconditioner is nonlinear, (the full) FGMRES [33] is used for outer iterations.
It is clear that the results in Table 4.3 provide an insight into the best possible outer iteration counts that can be expected from the restarted version of the
method in [13]. In fact, the same observation is used by the authors of [13] for
hand-tuning the smoothing schedule of their preconditioner to study its best-case
performance. Table 4.3 then demonstrates that regardless of the smoothing schedule
and the smoother’s stopping criterion, the (F)GMRES based solvers, such as the one
in [13], require increasing storage to maintain the convergence as c2 grows, whereas
the robustness of MINRES-AV-MG is not lost under the minimalist memory costs.
Thus, if the shift is large and the amount of storage is limited, so that k is forced
to be sufficiently small, the (F)GMRES(k) outer iterations may fail to converge within
a reasonable number of steps, even if the coarse grid solve in the MG preconditioner
is performed “early.” We note, however, that if storage is available or the shifts are
not too large, the (F)GMRES based methods may represent a valid option.
We have also tested the Bunch–Parlett factorization [18] as a coarse grid solve
in the MG framework. In particular, as a preconditioner for MINRES, we have used
Algorithm 3.2 with Bl = Ll on all levels and the coarsest grid absolute value in (3.37)
replaced by the application of the “perfect” Bunch–Parlett factorization based preconditioner [18]. We have obtained results that are inferior to the schemes considered
in this paper for shifts not too small, e.g., for c2 > 200 if h = 2−8 . The unsatisfactory behavior may be related to the fact that the inverted Laplacian T = L−1 and
−1
the ideal absolute value T = L − c2 I preconditioners share the same eigenvectors
with A = L − c2 I, while the preconditioner from [18] does not.
The standard MG preconditioners, as in GMRES(k)-MG and Bi-CGSTAB-MG,
are known to have optimal costs, linearly proportional to n. As discussed above, the
same is true for the AV preconditioner in Algorithm 3.2. Therefore if in addition the
number of iterations in the iterative solver preconditioned with Algorithm 3.2 does
not depend on the problem size, the overall scheme is optimal.
We verify this optimality in Table 4.4, which shows the mesh-independence of the
convergence of PMINRES with the MG AV preconditioner. The rows of the table
Table 4.4
Mesh-independent convergence of PMINRES with the MG AV preconditioner. The numbers in
parentheses correspond to δ = 3/4. The default value of δ is 1/3.
c2
= 300
c2 = 400
c2 = 1500
c2 = 3000
h = 2−6
31(31)
37(40)
67(97)
228(229)
h = 2−7
31(31)
38(40)
97(119)
222(284)
h = 2−8
30(32)
37(40)
89(109)
279(332)
h = 2−9
30(32)
37(40)
88(108)
256(298)
h = 2−10
30(32)
37(40)
89(106)
257(296)
h = 2−11
30(30)
37(39)
90(107)
256(298)
A716
EUGENE VECHARYNSKI AND ANDREW V. KNYAZEV
Eigenvalues of T|L − c2 I| and T(L − c2 I) for different shifts
3000
3000
Shift value c2
3000
1500
1500
1500
400
400
400
300
0
1
10 10
2
Λ(T|L − c I|)
300
0
−10
300
−1
0
−10
10
2
Λ(T(L − c I))<0
1
10
2
Λ(T(L − c I))>0
Fig. 4.2. Spectrum of T L − c2 I (left), negative eigenvalues of T L − c2 I (center), and
2
positive eigenvalues of T L − c I (right); n = 16129.
Table 4.5
Number of eigenvalues λj that equal the upper/lower bound in (2.4) up to machine precision.
Upper
Lower
c2 = 300
0
0
c2 = 400
15
0
c2 = 1500
1
10
c2 = 3000
115
0
correspond to the shift values c2 , while the columns match the mesh size h. The cell
in the intersection contains the numbers of steps performed to achieve the decrease
by the factor 10−8 in the error 2-norm with the choices of the “switching” parameter
δ = 1/3 and δ = 3/4.
As previously, the size of the coarsest grid has been set according to Table 4.1 with
δ = 1. We conclude that the convergence does not slow with the decrease of h; thus,
PMINRES preconditioned by Algorithm 3.2 is optimal. Note that for larger shifts,
c2 = 1500 and c2 = 3000, mesh-independent convergence occurs for h sufficiently
small when the “switching” pattern is stabilized, i.e., Bl = Ll on a few finer grids and
Bl = pml (Ll − c2 Il ) on the coarser grids that follow.
Table 4.4 shows that as c2 grows, the increase in the iteration count is mild and
essentially linear. As expected, the smaller value of δ, which leads to the construction
of the polynomial approximations earlier on finer levels, results in a higher accuracy
of the AV preconditioner.
Finally, in Figure 4.2 we plot the eigenvalues of T L − c2 I and T (L − c2 I) for
different shift values using the logarithmic
scale;
n = 16129. As suggested by Corol
lary 2.5, clusters of eigenvalues of T L − c2 I are preserved in the spectrum of the
preconditioned matrix T (L − c2 I). Almost all eigenvalues of T (L − c2 I) are clustered
around −1 and 1, with only a few falling outside the clusters. We note that the
clustering and the condition number κ(T |A|) deteriorate as c2 increases from 300 to
3000, which is compatible with the results in Table 4.4.
The spectra computed in Figure 4.2 allow validating numerically the tightness of
bounds (2.4) in Theorem 2.3 for the MG AV preconditioner. In Table 4.5, we report
the number of eigenvalues λj of T (L − c2 I) that satisfy either the upper or the lower
bound up to machine precision. The table shows that the bound is numerically sharp.
5. Conclusions. We propose a new approach for SPD preconditioning for symmetric indefinite systems based on the idea of implicitly constructing approximations to the inverse of the system matrix absolute value. An MG example of such
a preconditioner is presented for a real-valued Helmholtz problem. Our experiments
demonstrate that PMINRES with the new MG AV preconditioner leads to an efficient
ABSOLUTE VALUE PRECONDITIONING
A717
iterative scheme, which has modest memory requirements and outperforms traditional
GMRES based methods if available memory is tight.
Acknowledgments. The authors thank Michele Benzi, Yvan Notay, and Joe
Pasciak for their comments on the draft of the manuscript.
REFERENCES
[1] T. Airaksinen, E. Heikkola, and A. Pennanen, An algebraic multigrid based shiftedLaplacian preconditioner for the Helmholtz equation, J. Comput. Phys., 226 (2007),
pp. 1196–1210.
[2] A. Bayliss, C. I. Goldstein, and E. Turkel, An iterative method for the Helmholtz equation,
J. Comput. Phys., 49 (1983), pp. 443–457.
[3] M. Benzi, G. H. Golub, and J. Liesen, Numerical solution of saddle point problems, Acta
Numer., 14 (2005), pp. 1–137.
[4] J. H. Bramble, Z. Leyk, and J. E. Pasciak, Iterative schemes for nonsymmetric and indefinite elliptic boundary value problems, Math. Comp., 60 (1993), pp. 1–22.
[5] J. H. Bramble and X. Zhang, The analysis of multigrid methods, in Solution of Equation in
Rn (Part 3), Techniques of Scientific Computing (Part 3), Handb. Numer. Anal. 7, Elsevier,
Amsterdam, 2000, pp. 173–415.
[6] M. Bollhöfer, M. J. Grote, and O. Schenk, Algebraic multilevel preconditioner for
the Helmholtz equation in heterogeneous media, SIAM J. Sci. Comput., 31 (2009),
pp. 3781–3805.
[7] A. Brandt and S. Ta’asan, Multigrid method for nearly singular and slightly indefinite
problems, in Multigrid Methods II, Lecture Notes in Math. 1228, Springer, Berlin, 1986,
pp. 99–121.
[8] W. L. Briggs, V. E. Henson, and S. F. McCormick, A Multigrid Tutorial, SIAM,
Philadelphia, 2000.
[9] J. R. Bunch and B. N. Parlett, Direct methods for solving symmetric indefinite systems of
linear equations, SIAM J. Numer. Anal., 8 (1971), pp. 639–655.
[10] E. G. D’yakonov, Optimization in Solving Elliptic Problems, CRC Press, Boca Raton, FL,
1996.
[11] J. Erhel, F. Guyomarc, and Y. Saad, Least-Squares Polynomial Filters for Ill-Conditioned
Linear Systems, Report UMSI-2001-32, Minnesota Supercomputer Institute, University of
Minnesota, Minneapolis, 2001.
[12] Y. A. Erlangga, C. Vuik, and C. W. Oosterlee, On a class of preconditioners for solving
the Helmholtz equation, Appl. Numer. Math, 50 (2004), pp. 409–425.
[13] H. C. Elman, O. G. Ernst, and D. O’Leary, A multigrid method enhanced by Krylov subspace
iteration for discrete Helmholtz equations, SIAM J. Sci. Comput., 23 (2001), pp. 1291–1315.
[14] H. C. Elman, D. J. Silvester, and A. J. Wathen, Finite Elements and Fast Iterative Solvers
with Applications in Incompressible Fluid Dynamics, Oxford University Press, Oxford,
UK, 2005.
[15] V. Faber and T. Manteuffel, Necessary and sufficient conditions for the existence of a
conjugate gradient method, SIAM J. Numer. Anal., 21 (1984), pp. 352–362.
[16] B. Fischer, A. Ramage, D. J. Silvester, and A. J. Wathen, Minimum residual methods for
augmented systems, BIT, 38 (1998), pp. 527–543.
[17] R. W. Freund and N. M. Nachtigal, QMR: A quasi-minimal residual method for nonHermitian linear systems, Numer. Math., 60 (1991), pp. 315–339.
[18] P. E. Gill, W. Murray, D. B. Ponceleón, and M. A. Saunders, Preconditioners for indefinite systems arising in optimization, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 292–311.
[19] G. H. Golub and C. F. Van Loan, Matrix Computations, Johns Hopkins University Press,
Baltimore, MD, 1996.
[20] A. Greenbaum, V. Pták, and Z. Strakoš, Any nonincreasing convergence curve is possible
for GMRES, SIAM J. Matrix Anal. Appl., 17 (1996), pp. 465–469.
[21] E. Haber and S. MacLachlan, A fast method for the solution of the Helmholtz equation,
J. Comp. Phys., 230 (2011), pp. 4403–4418.
[22] I. Harari and T. J. R. Hughes, Finite element method for the Helmholtz equation in an
exterior domain, Comput. Methods Appl. Mech. Engrg., 87 (1991), pp. 59–96.
[23] N. J. Higham, Functions of Matrices: Theory and Computation, SIAM, Philadelphia, 2008.
[24] R. A. Horn and C. R. Johnson, Topics in Matrix Analysis, Cambridge University Press, New
York, 1994.
A718
EUGENE VECHARYNSKI AND ANDREW V. KNYAZEV
[25] A. L. Laird and M. B. Giles, Preconditioned Iterative Solution of the 2D Helmholtz Equation,
Oxford University Computing Laboratory, Oxford, UK, 2002.
[26] J. Nocedal and S. Wright, Numerical Optimization, Springer, Berlin, 1999.
[27] D. Osei-Kuffuor and Y. Saad, Preconditioning Helmholtz linear systems, Appl. Numer.
Math., 60 (2010), pp. 420–431.
[28] C. C. Paige and M. A. Saunders, Solution of sparse indefinite systems of linear equations,
SIAM J. Numer. Anal., 12 (1975), pp. 617–629.
[29] B. N. Parlett, The Symmetric Eigenvalue Problem, SIAM, Philadelphia, 1998.
[30] M. J. D. Powell, Approximation Theory and Methods, Cambridge University Press, New York,
1981.
[31] T. J. Rivlin, An Introduction to the Approximation of Functions, Dover, New York, 1981.
[32] Y. Saad, Filtered conjugate residual-type algorithms with applications, SIAM J. Matrix Anal.
Appl., 28 (2006), pp. 845–870.
[33] Y. Saad, A flexible inner-outer preconditioned GMRES algorithm, SIAM J. Sci. Comput., 14
(1993), pp. 461–469.
[34] Y. Saad, Iterative Methods for Sparse Linear Systems, SIAM, Philadelphia, 2003.
[35] Y. Saad and M. H. Schultz, GMRES: A generalized minimal residual algorithm for solving
nonsymmetric linear systems, SIAM J. Sci. Statist. Comput., 7 (1986), pp. 856–869.
[36] G. L. G. Sleijpen and H. A. van der Vorst, A Jacobi–Davidson iteration method for linear
eigenvalue problems, SIAM J. Matrix Anal. Appl., 17 (1996), pp. 401–425.
[37] D. J. Silvester and A. J. Wathen, Fast iterative solution of stabilised Stokes systems II:
Using general block preconditioners, SIAM J. Numer. Anal., 31 (1994), pp. 1352–1367.
[38] O. Tatebe, The multigrid preconditioned conjugate gradient method, in Proceedings of the
Sixth Copper Mountain Conference on Multigrid Methods, 1993, pp. 621–634.
[39] U. Trottenberg, C. W. Oosterlee, and A. Schüller, Multigrid, Academic Press,
New York, 2001.
[40] M. B. van Gijzen, Y. A. Erlangga, and C. Vuik, Spectral analysis of the discrete Helmholtz
operator preconditioned with a shifted Laplacian, SIAM J. Sci. Comput., 29 (2007),
pp. 1942–1958.
[41] H. A. van der Vorst, Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for
the solution of nonsymmetric linear systems, SIAM J. Sci. Statist. Comput., 13 (1992),
pp. 631–644.
[42] E. Vecharynski, Preconditioned Iterative Methods for Linear Systems, Eigenvalue and Singular Value Problems, Ph.D. thesis, University of Colorado, Denver, 2010.
[43] E. Vecharynski and J. Langou, Any admissible cycle-convergence behavior is possible
for restarted GMRES at its initial cycles, Numer. Linear Algebra Appl., 18 (2011),
pp. 499–511.
[44] A. J. Wathen and D. J. Silvester, Fast iterative solution of stabilized Stokes systems I: Using
simple diagonal preconditioners, SIAM J. Numer. Anal., 30 (1993), pp. 630–649.
[45] Y. Zhou, Y. Saad, M. L. Tiago, and J. R. Chelikowsky, Self-consistent-field calculation
using Chebyshev-filtered subspace iteration, J. Comput. Phys., 219 (2006), pp. 172–184.