Solving Scalar Linear Systems
A Little Theory For Jacobi Iteration
Lecture 15
MA/CS 471
Fall 2003
Review:
• Using Kirchoff’s second law we build the
loop current circuit matrix.
3
1W
I1  2  1  I 5 2  V1
I 2  7  6   I5 7
I 3  4  1  I 5 4
4W
V4
+
-
I 4  3  4   I 5 3  V4
4
4W
3W
5
6W
7W
2W
+
V1
-
1
1W
I 5  2  7  4  3  r5   I1 2  I 2 7  I 3 4  I 4 3
Note we have boosted the center cell to ensure diagonal dominance (hack)
2
System:
3I1  I 5 2  V1
13I 2  I 5 7
5I3  I5 4
7 I 4  I 5 3  V4
16  r5  I 5  I1 2  I 2 7  I 3 4  I 4 3
2  I1   V1 
 3 0 0 0
 0 13 0 0
7  I 2   0 

   
 0 0 5 0
4  I 3    0 

   
0
0
0
7

3

 I 4   V4 
 2 7 4 3 16  r  I   0 
 
5  5 

3I1n1  3I 5n  V1
Jacobi iterative approach
13I 2n1  7 I 5n
5 I 3n1  4 I 5n
7 I 4n1  3I 5n  V4
n 1
n
n
n
n
16

r
I

2
I

7
I

4
I

3
I

5 5
1
2
3
4
Jacobi v. Gauss-Seidel
3I1n1  3I 5n  V1
Jacobi
13I 2n1  7 I 5n
5 I 3n1  4 I 5n
7 I 4n1  3I 5n  V4
n 1
n
n
n
n
16

r
I

2
I

7
I

4
I

3
I

5 5
1
2
3
4
3I1n1  3I 5n  V1
13I 2n1  7 I 5n
Gauss-Seidel
5 I 3n1  4 I 5n
7 I 4n1  3I 5n  V4
16  r5  I 5n1  2 I1n1  7 I 2n1  4 I 3n1  3I 4n1
Convergence Proof
Definition of Spectral Radius
• We define the spectral radius of a matrix A
as:
  A   max   such that Ax   x for some nonzero x
Stage 1: Unique Limit
• The equation we wish to solve is: Ax  b
• We consider the following iterator:
Qxn1   Q  A  x n  b
• For some easily invertible matrix Q
• Suppose this iteration does indeed converge
n
as n   , i.e. n  , x  x
• Then xtilde will satisfy: Qx   Q  A  x  b
cont
• And Qx   Q  A  x  b yields Ax  b
• So if the iteration converges, then the limit
vector will be a solution to the original
system.
Second Stage of
Convergence Proof
• We first prove that the Jacobi (and GaussSeidel) methods converge if and only if the
spectral radius of Q  A  Q 
1
Theorem
• Suppose b 
n
,A
nn
,Q 
nn
and that both
Q and A are non-singular. If the spectral radius
1
A
:

Q
 Q  A  is strictly bounded above by
of
n
x
1 then the iterates
defined by:
Qxn1   Q  A  x n  b
1
0
x

A
b
x
converge to
for any starting vector
Proof
1) Let e  x  x denote the error in the n’th
iterate. We combine the following
relationships: Qx n1   Q  A  x n  b
n
n
Qx   Q  A  x  b
2) to obtain:
3) simplifying:
Qen1   Q  A  en
e
n 1
Q
1
Q  A e
 Aen
n 1 0
A e
n
Proof cont
• The iteration en1  A n1e0 converges with
increasing n if and only if   A   1 (proof
omitted).
Third Stage of Convergence
• Now we are left with the task of proving
that for the choice of Q the A  Q  A  Q 
matrix has   A   1
• i.e. we have to prove that the absolute
value of all of the eigenvalues of the A
matrix is less than one.
• So we use Gershgorin’s theorem to find
the range of the eigenvalues of A
1
Recall: Gershgorin’s Circle
Theorem
Let A be a square NxN matrix. Around every element
aii on the diagonal of the matrix draw a circle with
jN
radius
ri 

j 1, j i
aij
Such circles are known as Gershgorin’s disks.
Theorem: every eigenvalue of A lies in one of these
Gershgorin’s disks.
Jacobi Iteration
• Recall the generic iterative scheme required
a Q matrix which is “easily” invertible.
• Let’s take Q=diag(A) (i.e. a matrix with zeros
everywhere, apart from the diagonal entries
which are the same as those of A)
D
• Let’s write: A=L+D+U
U
L
Cont.
n 1
n
Dx

D

A
x

 b
• Then the scheme becomes:
 ( U  L)x n  b
n 1
1
1
• i.e. x  D (U  L)x  D b
• We can determine conditions under which
this scheme will converge.
• Recall the necessary and sufficient
condition that   A     D1  L  U    1
n
Cont.
• We can use Gershgorin’s theorem after we
note that
1
AD
L  U
• has zero entries on the diagonal – so all
the Gershgorin disks will be centered at
zero and have maximum radius:
 D
1
jN
aij
j 1, j i
aii
 L  U    r  imax

1,..., N
cont
• So if
 D
1
jN
aij
j 1, j i
aii
 L  U    imax

1,..., N
• Then we are done.
• Note, a matrix which satisfies:
max
i 1,..., N
jN
aij
j 1, j i
aii

1
• Is called diagonally dominant.
1
Summary
1) The first stage of the convergence proof
showed that the unique possible convergent
limit of the scheme is the actual solution to
the linear system.
2) Secondly, we showed that the scheme
converges if and only if   Q  A  Q    1
3) Thirdly, we showed that for the choice of
Q = diagonal of A, that (2) was satisfied.
4) i.e. Jacobi iteration converges for any initial
guess for x if A is diagonally dominant.
1