The Islamic University of Gaza
Faculty of Engineering
Civil Engineering Department
Numerical Analysis
ECIV 3306
Chapter 11
Special Matrices and Gauss-Siedel
Symmetric Matrices
– Certain matrices have particular structures
that can be exploited to develop efficient
solution schemes.
• Symmetric matrix:
If aij = aji  [A]nxn is a
symmetric matrix
5
1
[ A]  
2

16
1 2 16 

3 7 39 
7 9 6

39 6 88 
Cholesky Decomposition

This method is suitable for only symmetric systems
where:
a ij  a ji and
A A
T
 l11 0
[L ]  l 21 l 22

 l 31 l 32
0
0

l 33 
A  L * LT
a11 a12 a13   l11 0
a
  l
a
a
l
 21 22 23   21 22
a31 a32 a33   l 31 l 32
0  l11 l 21 l 31 
0  *  0 l 22 l 32 
 

l 33   0 0 l 33 
3
Cholesky Decomposition
i 1
l ijl ij lkjl kj
aakiki 
i 1
ll kiki 
j 11
j
lliiii
for i i 1,2,
 1,2, , k ,k1 1
for
k k11
22
l kkkk  aakkkk 
l
l kjkj
j j 11
For 3x3
matrix,
L11  a11
L22  a22  L21
a21
L21 
L11
a32  L21L31
L32 
L22
L31 
a31
L11
2
L33  a33  L232  L231
Once the matrix L has been calculated, LT can easily be found and the
forward , backward substitution steps can be followed to find the
solution {X} for any right hand side {B}.
Pseudocode for Cholesky’s LU
Decomposition algorithm (cont’d)
Gauss-Siedel
• Iterative or approximate methods provide an
alternative to the elimination methods. The GaussSeidel method is the most commonly used iterative
method.
• The system [A]{X}={B} is reshaped by solving the
first equation for x1, the second equation for x2,
and the third for x3, …and nth equation for xn. We
will limit ourselves to a 3x3 set of equations.
Gauss-Siedel
a11 x1  a12 x2  a13 x3  b1
a21 x1  a22 x2  a23 x3  b2
a31 x1  a32 x2  a33 x3  b3

b1  a12 x2  a13 x3
x1 
a11
b2  a21 x1  a23 x3
x2 
a22
b3  a31 x1  a32 x2
x1 
a33
Now we can start the solution process by choosing
guesses for the x’s. A simple way to obtain initial
guesses is to assume that they are zero. These
zeros can be substituted into x1 equation to
calculate a new x1=b1/a11.
Gauss-Siedel
• New x1 is substituted to calculate x2 and x3. The
procedure is repeated until the convergence
criterion is satisfied:
 a ,i 
x
new
i
old
i
x
new
i
x
100%   s
Gauss SeidalGraphical Interpretation
Jacobi iteration Method
An alternative approach, called Jacobi iteration,
utilizes a somewhat different technique. This
technique includes computing a set of new x’s on the
basis of a set of old x’s. Thus, as the new values are
generated, they are not immediately used but are
retained for the next iteration.
Gauss-Siedel
The Gauss-Seidel method
The Jacobi iteration method
Convergence Criterion for Gauss-Seidel Method
•
•
The gauss-siedel method is similar to the technique of
fixed-point iteration.
The Gauss-Seidel method has two fundamental
problems as any iterative method:
1. It is sometimes non-convergent, and
2. If it converges, converges very slowly.
•
Sufficient conditions for convergence of two linear
equations, u(x,y) and v(x,y) are:
u u

1
x y
v v

1
x y
Convergence Criterion for Gauss-Seidel
Method (cont’d)
• Similarly, in case of two simultaneous equations, the
Gauss-Seidel algorithm can be expressed as:
b1 a12
u (x 1 , x 2 ) 

x2
a11 a11
b 2 a21
v (x 1 , x 2 ) 

x1
a22 a22
u
0
x 1
a12
u

x 2
a11
a21
v

x 1
a22
v
0
x 2
Convergence Criterion for Gauss-Seidel
Method (cont’d)
Substitution into convergence criterion of two linear
equations yield:
a12
a21
a11
 1,
a22
1
In other words, the absolute values of the slopes must be
less than unity for convergence:
a11  a12
For n equations
a22  a21
aii 
n
a
j 1
j i
i, j
That is, the diagonal element must be greater than the sum
of off-diagonal element for each row.
Gauss-Siedel Method- Example 1
 3  0.1 0.2   x1   7.85 
  

0.1

7
 0.3  x2    19.3

0.3  0.2 10   x3   71.4 
7.85  0.1x2  0.2 x3
3
x1 
 19.3  0.1x1  0.3 x3
 x2 
7
x3  71.4  0.3 x1  0.2 x2
10
• Guess x1, x2, x3= zero for the first guess
Iter.
0
1
2
x1
x2
0
0
2.6167
-2.7945
2.990557 -2.499625
x3
0
7.005610
7.000291
|a,1|(%)
|a,2| (%)
100
12.5
100
11.8
|a,3| (%)
100
0.076
Improvement of Convergence Using
Relaxation
x
new
i
  x
new
i
 1     x
old
i
• Where  is a weighting factor that is assigned a
value between [0, 2]
• If  = 1 the method is unmodified.
• If  is between 0 and 1 (under relaxation) this is
employed to make a non convergent system to
converge.
• If  is between 1 and 2 (over relaxation) this is
employed to accelerate the convergence.
Gauss-Siedel Method- Example 2
The system can be re-arranged to the following
6x1-x2-x3 = 3
6x1+9x2+x3=40
-3x1+x2+12x3=50
Gauss-Siedel Method- Example 2
3  x2  x3
x1 
6
40  6 x1  x3
x2 
9
50  3 x1  x2
x3 
12
Without Relaxation
Gauss-Siedel Method- Example 2