Special Matrices and Gauss-Seidel
Chapter 11
Credit: Prof. Lale Yurttas, Chemical Eng., Texas A&M University
1
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
• Certain matrices have particular structures
that can be exploited to develop efficient
solution schemes (e.g. banded, symmetric)
• A banded matrix is a square matrix that has
all elements equal to zero, with the exception
of a band centered on the main diagonal.
• Standard Gauss Elimination is inefficient in
solving banded equations because
unnecessary space and time would be
expended on the storage and manipulation of
zeros.
• There is no need to store or process the zeros
(off of the band)
2
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Solving Tridiagonal Systems
(Thomas Algorithm)
A tridiagonal system has a bandwidth of 3
 f1
e
 2



g1
f2
g2
e3
f3
1
e'
A  L U   2
0

0
e4
  x1   r1 
  x  r 
  2    2 
g 3   x3  r3 
   
f 4   x4  r4 
0
0
1
0
e'3
1
0
e'4
0   f1
0 
0 

1 
DECOMPOSITION
DO
ek = ek / fk-1
fk = fk - ek gk-1
g1
f 2'
k = 2, n
g2
f 3'
 END DO


g3  Time Complexity?

O(n)
f 4' 
vs. O(n3)
3
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Tridiagonal Systems (cont.)
{d}
1
e '
 2
0

0
1
e'
 2
0

0
0
0
1
0
e'3
1
0
e'4
0
0
1
0
e'3
1
0
e'4
0   f1
0 
0 

1 
0  d1   r1 
0 d 2  r2 




 r3 
0 d3
   
1 d 4  r4 
Forward Substitution
d1 = r1
DO
k = 2, n
f 2'
g2
f 3'
 f1





  x1   r1 
  x  r 
 2    2 
g3   x3   r3 
   
f 4'   x 4  r4 
g1
f 2'
g2
f 3'
  x1   d1 
  x  d 
 2    2 
g3   x3   d 3 
   
f 4'   x 4  d 4 
Back Substitution
xn = dn /f’n
DO
dk = rk – e’k dk-1
END DO
g1
k = n-1, 1, -1
xk = (dk - gk . xk+1 )/f’k
END DO
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
4
Cholesky Decomposition
(for Symmetric Positive Definite† Matrices)
†
A positive definite matrix is one for which the product
{X}T[A]{X} is greater than zero for all nonzero vectors X
l11
l
l
T
T
[ A]  [ A]  L  L   21 22
l31 l32

l41 l42
l33
l43
 a11 a21
a
a22
[ A]  [ A]T   21
 a31 a32

a41 a42
 l11 l21 l31 l41 


l
l
l
22
32
42



l33 l43 


l44  
l44 
a31
a32
a33
a43
a41 
a42 
a43 

a44 
LT means Transpose of L
i 1
lki 
aki   lijlkj
j 1
lii
for k  1,2,, n i  1,2,, k  1
k 1
lkk  akk   lkj2
j 1
Time Complexity:
O(n3) but requires half the number of operations as standard Gaussian elimination.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Jacobi Iterative Method
Iterative methods provide an alternative to the elimination methods.
Ax  b
 a11 a12
A  a 21 a 22
a31 a32
[ D  ( A  D )] x  b

a13 
a 23 
a33 
a11 0
D   0 a 22
 0
0
Dx  b  ( A  D ) x
x  D 1[b  ( A  D ) x ]

0
0    b1   0
 x1 
1/ a11
x    0
 *  b    a
1/
a
0
22
 2

   2   21
 x3  k 1  0
0
1/ a33    b3   a31
k 1
1
x
b1  a12 x2k  a13 x3k

a11
k 1
2
x
b2  a21 x1k  a23 x3k

a22
0
0 
a33 
k 1
3
x
a12
0
a32
a13   x1  

a23   x2  
0   x3  k 
b3  a31 x1k  a32 x2k

a33
Choose an initial guess (i.e. all zeros) and Iterate until the equality is satisfied.
No guarantee for convergence! Each iteration takes O(n2) time!
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
6
Gauss-Seidel
• The Gauss-Seidel method is a commonly used iterative method.
• It is same as Jacobi technique except with one important difference:
A newly computed x value (say xk) is substituted in the subsequent
equations (equations k+1, k+2, …, n) in the same iteration.
Example: Consider the 3x3 system below:
x
b1  a12 x 2old  a13 x3old

a11
x 2new
b2  a 21 x1new  a 23 x3old

a 22
x3new
b3  a31 x1new  a32 x 2new

a33
new
1
{ X }old  { X }new
• First, choose initial guesses for the x’s.
• A simple way to obtain initial guesses is
to assume that they are all zero.
• Compute new x1 using the previous
iteration values.
• New x1 is substituted in the equations to
calculate x2 and x3
• The process is repeated for x2, x3, …
7
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Convergence Criterion for Gauss-Seidel Method
• Iterations are repeated until the convergence criterion is satisfied:
 a ,i
For all i, where j and j-1 are
xij  xij 1

100%   s the current and previous iterations.
j
xi
• As any other iterative method, the Gauss-Seidel method has problems:
– It may not converge or it converges very slowly.
• If the coefficient matrix A is Diagonally Dominant Gauss-Seidel is
guaranteed to converge.
For each equation i:
Diagonally Dominant 
n
aii   ai , j
j 1
j i
• Note that this is not a necessary condition, i.e. the system may still have a
chance to converge even if A is not diagonally dominant.
Time Complexity:
Each iteration takes O(n2)
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
8