COMPUTATIONAL METHODS IN ENGINEERING AND SCIENCE
EPMESC X, Aug. 21-23, 2006, Sanya, Hainan, China
©2006 Tsinghua University Press & Springer
A Note on the Complexity of the PCG Algorithm for Solving Toeplitz
Systems with a Fisher-Hartwig Singularity
Seak-Weng Vong *, Wei Wang, Xiao-Qing Jin
Department of Mathematics, University of Macau, Macao, China
Email: [email protected], [email protected]
Abstract Recently, Y. Lu and C. Hurvich showed that the complexity of T. Chan’s preconditioned conjugate gradient
algorithm for solving the Toeplitz system Tn( f )x = b is O(n log3 n) where the generating function f is given by
f(ω) = |1 − e−iω|−2dh(ω) with d ∈(−1/2, 1/2)\{0} and h(ω) being positive continuous on [−π, π] and differentiable on
[−π, π]\{0}. Although their results are interesting, there exist some improper expressions in their proofs needed to be
corrected. In this paper, we try to improve those improper expressions and demonstrate these important results by
some numerical tests.
Key words: PCG algorithm, Toeplitz matrix, circulant matrix, Fisher-Hartwig singularity, T. Chan’s circulant
preconditioner
INTRODUCTION
In this paper, we consider to use the preconditioned conjugate gradient (PCG) algorithm to solve some special Toeplitz
systems. The PCG algorithm has been a popular and effective iterative method for solving Toeplitz systems Tn x = b
since 1986 [11]. It is well known that Toeplitz systems arise in a variety of applications in science and engineering [2,
6, 7, 8, 10]. The Toeplitz matrix is defined as follows:
(1)
where the entries are given by
for k = 0, ±1, ±2, …. The function f is called the generating function of Tn. In the following, we assume that Tn is
symmetric, i.e., tk = t− k. The complexity of the PCG algorithm with some suitable preconditioners, for instance, T.
Chan’s circulant preconditioner and superoptimal circulant preconditioner, is known to be O(n log n) for Toeplitz
systems where the generating function f(ω) is positive continuous on [−π, π], see [2, 6, 10].
Recently Y. Lu and C. Hurvich [9] considered the CG algorithm with T. Chan’s pre-conditioner for solving Tn x = b
with the generating function f(ω) satisfying the following assumption:
Assumption 1 [4, 9] The generating function f is given by
where d ∈ (−1/2, 1/2) and h(ω) is even positive continuous on [−π, π] and differentiable on [−π, π]\{0}. Also log h is
Riemann integrable on [−π, π], i.e.,
⎯ 903 ⎯
and there exists c ∈ (0, ∞) such that
Moreover, h(ω) is assumed to be in
We recall that the Besov space
and all conditions for h(ω) are also applied to 1/h(ω) .
includes all functions g such that g ∈ L1 and
Now let us give the definition of T. Chan's circulant preconditioner proposed in [3]:
Definition 1 [2, 3] For Tn given as in (1), the diagonals of T. Chan's circulant preconditioner cF (Tn) are given by
Under Assumption [2], Tn has a Fisher-Hartwig singularity caused by the term |1 − e−iω|−2d in the generating function f
and the condition number of Tn approaches ∞ as n increases, see [4]. Y. Lu and C. Hurvich showed in ([9]) that the
complexity of the CG algorithm for solving Tn x = b without any preconditioning grows asymptotically as n1+|d| log n.
With T. Chan’s optimal circulant preconditioner cF (Tn), the complexity of the PCG algorithm is O(n log3 n). But there
exist some improper expressions in their proofs needed to be corrected or revised. In this paper, we try to improve their
proofs and demonstrate these important results by some numerical examples.
SPECTRAL ANALYSIS
We first introduce the following lemma and the proof can be found in [9].
Lemma 1 [9] If f satisfies Assumption 1, we then have
where Ci, i = 1, 2, represent positive constants independent of n, λmin(M) and λmax(M) denote the smallest and the
largest eigenvalues of M respectively.
Lemma 2 [9] If f satisfies Assumption 1, then Tn(f)−
(f −1) is positive semidefinite.
Proof: Let
and
Then
and
Note that for any α(ω) = (α 1(ω),α 2(ω), …, α n(ω))T , the matrix
⎯ 904 ⎯
is positive semidefinite since
for any x ∈ Cn. Therefore, for any n × n matrix A, we have
(2)
In particular, if we take A = −T−1(f −1) in (2), we can conclude that Tn(f) −
(f −1) is positive semidefinite.
Theorem 1 [9] If f satisfies Assumption 1, then we have
(3)
where C is a positive constant independent of n.
Proof: We can factorize
as
where
By using Lemmas 1 and 2, we notice that B1, B2, B3 are all Hermitian positive definite and B4 is Hermitian positive
semidefinite. Therefore, we have
Thus, the condition number of T. Chan’s preconditioned matrix satisfies
Therefore, the number of iterations in the PCG algorithm is bounded by O(log2 n), see [5, 7]. Since each iteration needs
O(n log n) operations using the FFT, the total complexity of the PCG algorithm is still O(n log3 n).
NUMERICAL TESTS
In this section, we verify Theorem 1 by the following problem.
Problem 1 ([1]) The generating function is given by
⎯ 905 ⎯
Then we obtain
Also, we choose b = (t1, t2, …, tn)T as the right-hand side of Tn x = b.
Figure 1: Two examples of iteration numbers
All the experiments were performed in MATLAB.We used the MATLAB-provided M-file
to solve the system.
−10
< 10 where rj is the residual
In our tests, the zero vector is the initial guess and the stopping criterion is
after the j-th iteration. We consider the following two cases in Problem 1:
Case 1: Choose d = 0.37 and
Case 2: Choose d = 0.1 and
= 0.27.
= 0.27.
In Table 1, we give the number of iterations for convergence. In the table, n is the matrix size. We compare the
preconditioned system by using T. Chan’s preconditoner with the system with no preconditioner. We note that when n
increases, the number of iterations of the preconditioned system is much less than that of the system with no
preconditioner.
Table 1: Number of iterations for PCG algorithm
Acknowledgments
The authors would like to thank Prof. Clifford M. Hurvich for his useful suggestions on this topic. The support of the
Grant 050/2005/A from FDCT is gratefully acknowledged.
REFERENCES
1. Brockwell P, Davis R. Time Series: Theory and Methods. Springer-Verlag, New York, USA, 1991.
2. Chan R, Ng M. Conjugate Gradient Methods for Toeplitz Systems. SIAM Review, 1996; 38: 427-482.
⎯ 906 ⎯
3. Chan T. An Optimal Circulant Preconditioner for Toeplitz Systems. SIAM J. Sci. Statist. Comput., 1988; 9:
766-771.
4. Fisher M, Hartwig R. Toeplitz determinants ⎯ some applications, theorems, and conjectures. Adv. Chem. Phys.,
1968; 32: 190-225.
5. Golub G, Loan C. Matrix Computations. 3rd edition, Johns Hopkins University Press, Baltimore, USA, 1996.
6. Jin X. Developments and Applications of Block Toeplitz Iterative Solvers. Kluwer Academic Publishers,
Dordrecht; and Science Press, Beijing, China, 2002.
7. Jin X, Wei Y. Numerical Linear Algebra and Its Applications. Science Press, Beijing, China, 2004.
8. King R, Ahmadi M, Gorgui-Naguib R, Kwabwe A, Azimi-Sadjadi M. Digital Filtering in One and Two
Dimensions: Design and Applications. Plenum, New York, USA, 1989.
9. Lu Y, Hurvich C. On the complexity of the preconditioned conjugate gradient algorithm for solving Toeplitz
systems with a Fisher-Hartwig singularity. SIAM J. Matrix Anal. Appl., 2005; 27: 638-653.
10. Ng M. Iterative Methods for Toeplitz Systems. Oxford University Press, Oxford, UK, 2004.
11. Strang G. A proposal for Toeplitz matrix calculations. Stud. Appl. Math., 1986; 74: 171-176.
⎯ 907 ⎯