Introduction to Krylov Subspace Methods
Ax  b
A  R nn , b  R n
DEF:
2
b, Ab, A b,
Example:
A
Krylov sequence
10 -1 2 0
-1 11 -1 3
2 -1 10 -1
0 3 -1 8
Krylov sequence
b Ab A2b A3b A4b
1
1
1
1
11
12
10
10
118
141
100
106
1239
1651
989
1171
12717
19446
9546
13332
Introduction to Krylov Subspace Methods
DEF:
Krylov subspace
 m ( A, b)  span{ b, Ab,, A
Example:
A
b}
Krylov subspace
10 -1 2 0
-1 11 -1 3
2 -1 10 -1
0 3 -1 8
Remark:
m 1


 3 ( A, b)  span 


A3b  A( A( Ab))
1
1 ,
11

11
12 ,
10

10
118
141
100

106





WHY: Krylov Subspace Methods
Example: Solve:






 x1 
 6 
10 -1 2 0 
x 
 25 
2
-1 11 -1 3 
  

  11
2 -1 10 -1   x3 




0 3 -1 8   x4 
15


>> A=[10 -1 2 0; -1 11 -1 3;2 -1 10 -1; 0 3 -1 8]
1
39
552
A1   7395
A3  7395
A2  7395
A  3357
I
7395
>> poly(A)
1
-39
552
-3357
7395
Characteristic poly
p( x)  x 4  39 x3  552 x 2  3357 x  7395
the Cayley-Hamilton theorem a matrix
satisfies its characteristic polynomial, p(A) =
0. That is,
A4  39 A3  552 A2  3357 A  7395I  0
Multiplying with inv(A) & rearrange
Hence,
1
39
552
A1b   7395
A3b  7395
A2b  7395
Ab  3357
b
7395
The key observation here is that the
solution x to Ax = b is a linear
combination of the vectors b and Ab,..
which make up the Krylov subspace
the solution to Ax = b has a
natural representation as a
member of a Krylov space,
Krylov subspace Methods
Krylov
subspace
Conjugate
Gradient
others
A SPD
min e
Kn
A
MINRES
A symm.
min r
Kn
GMRES
A general
min r
Kn
MATLAB commands
Conjugate
Gradient
A symm. &  definite
x = pcg(A, b, tol, maxit)
MINRES
A symm.
x = minres(A, b, tol, maxit)
GMRES
A general
x = gmres(A,b,[],tol,maxit)
Conjugate Gradient Method
We want to solve the following linear system
Ax  b
Conjugate Gradient Method
r0  b  Ax0
A  R nn , b  R n
p0  r0
A SPD (symmetric positive definite)
for k  0 ,1,2 ,..
αk  rkT rk /p kT Apk
x T Ax  0
xk 1  xk  αk pk
x  0
rk 1  rk  αk Apk
βk 1  rkT1rk 1 /rkT rk
pk 1  rk 1  βk 1 pk
end
Conjugate Gradient Method
Conjugate Gradient Method
Example: Solve:






r0  b  Ax0
 x1 
 6 
10 -1 2 0 
x 


-1 11 -1 3  2    25 
  11
2 -1 10 -1   x3 




0 3 -1 8   x4 
15


p0  r0
for k  0 ,1,2 ,..
αk  rkT rk /p kT Apk
xk 1  xk  αk pk
rk 1  rk  αk Apk
βk 1  rkT1rk 1 /rkT rk
pk 1  rk 1  βk 1 pk
end
0
x1
x2
x3
X4
r (k )
K=1
K=2
0 0.4716 0.9964
0 1.9651 1.9766
0 -0.8646 -0.9098
0 1.1791 1.0976
K=3
K=4
1.0015
1.9833
-1.0099
1.0197
1.0000
2.0000
-1.0000
1.0000
31.7 5.1503 1.0433 0.1929
0.0000
1
2
x*   
 1
 
1