Lecture 4
Iterative methods: Jacobi method, Gauss-Seidel method, General schemes.
The Sufficient Condition of convergence of the Iterative Process. Estimated
of Error
We consider linear algebraic equations system
Ax  b ,
(1)
 x1 
b1 
a1n 
 
x 
 , x   2  , b  b2  , a  0 i  1,2,..., n .


ii

 
 
ann 
 
 
bn 
 xn 
The general idea of an iterative method for solving a linear system of the form
Ax  b is to formulate a scheme that calculates an approximate solution x ( k 1) at
the  k  1 th step based on the approximation x( k ) obtained at the k th iteration.
 a11
where A  
 an1
Starting with an initial guess x(0) , we then hope that the iterations produce better
and approximations. The method is said to be convergent if
(2)
lim x ( k )  x  0 ,
k 
where x is the solution of the system. Because of the equivalence of norms for
finite-dimensional vector space, any convenient vector norm can be used in the
limit (2). In possession of a convergent method, we obviously don’t sit around
and wait for an infinite number of steps until we reach the solution x . Instead,
we would like to stop the iterations once the error (vector)
(3)
e( k )  x  x ( k )
becomes sufficiently small (according to a given norm). But since x is not
known, we can stop the iterations once the residual
(4)
r ( k )  Ax( k )  b
becomes sufficiently small.
Jacobi method
Given an approximate solution x( k ) , the Jacobi method proceeds to calculate the
first component of x( k 1) from the first equation in the system, the second
component of x( k 1) from the second equation, and so on. We have
 x1( k 1)  1  12 x2( k )  13 x3( k )   1n xn( k )
 ( k 1)
(k )
(k )
(k)
 x2   2   21 x1   23 x3    2 n xn


 x ( k 1)     x ( k )   x( k )  ...   x( k )
n
n1 1
n2 2
nn 1 n 1
 n
1
(5)
where i 
bi
,
aii
 ij  
aij
aii
,
(i  j ) and  ij  0 if i  j . Or in matrix form
x( k 1)     x( k ) , k  0,1, ,
where k is iteration number and x(0)   is initial guess.
(6)
Theorem 1 If the limit x  lim x( k ) it is then the system (1) given unique
k 
solution.
Proof.
lim x( k 1)     lim x( k )
Really, it follows from (6) that
k 
k 
or
x     x that is to say х is solution of system (1).
Let ( k )  x( k )  x( k 1) , k  1,2, .
(7)
x( k 1)     x( k ) ,
(8)
x( k )     x( k 1) .
It follows from (7) and (8) that x( k 1)  x( k )  ( k 1)   ( x (k )  x k(  1))   k( )or
if 
 
( k 1)
(k  0,1,...) and 
(k )
(0)
x
(0)
then x
 m
m
    .
k
k 0

Let
(0)
(k )
x
(k )
 
s 0
  . We have 
 
(k )
( k 1)
 
k  1, 2 , . . .
k
and
k
(s)
  s  .
s 0

Let
x
(0)
(0)
 x(0)  
   x   x    x     x
1
then
1
k
0
0
k
0
and
( k )  ( k 1)   k 1  , (k  1,2,...) , and x( k )   ( s )  х    s 1  .
1
s 0
0
1
s 1
The Sufficient Condition of convergence of the Iterative Process
We consider linear algebraic equations system
x x  
(1)
Theorem 2 If canonical norm of matrix  it is less 1 (i.e. 
con
 1 ) then
for  х  the iteration process x      x ( k 1) , k  1,2,... it is converged to
unique solution of the system (1). The condition
 1
(2)
0
k
is the sufficient condition of convergence of iteration process x      x ( k 1) ,
k  1,2,... .
Proof. We calculate
1
0
x      x  ,
k
2
x      x  ,
1
0
x      x
k
x   E     2  ...   
k
From here
 1
if

k 1
 k  0,
then
lim( E      ....  
2
k 1
k 
.
k 1
k  .
    x  ,
k
0
(3)
l
Then

)   k  ( E   ) 1 .
k 
i k m
and 0
Now that in (3) k   we
k 0
have
x  lim x   ( E   ) 1  .
k
(4)
k 
It follows from (4) that ( E   ) x   or x   x    The vector х is
solution of system (1) and because the matrix  E    is non-singular matrix the
solution х is unique.
Corollary 1 If a)  m  1 , b)  l  1 , c)  k  1 then for system (1) the
iteration process x      x ( k 1) , k  1,2,... it is converged.
1
k
In particular, if  ij 
then the iteration process x      x ( k 1) ,
n
k  1,2,... it is converged, where n is number of unknowns.
k
n
n
2) a jj   aij ,
i  1, 2 , .n. . and
,
If 1) aii   aij ,
Corollary 2
j 1
i j
j  1,2,..., n then for system
n
a x
j 1
ij
i 1
i j
 bi
j
(i  1, 2,...n ;) ( Ax  b) the iteration
process x      x ( k 1) , k  1,2,... it is converged.
k
The Estimation of the Error Vector of Iteration Process
Let x
, x( k ) , (k  1) is two approximation of the system x   x   .
Let p  1 we have
( k 1)
x
By x
k  p
m 1
 x
k
 x
k 1
 x
  x    and x
m
We have x
m1
 x
m
m
k
 x
  x
m 1
  x   x
m
k  2
 x
k 1

   x
m1
 
mk
m1
 x
k p
 x

k  p 1
.
 x    x   x
 x
m
k 1
 x
k
m

pk 
 x
k
 x
k 1
 x
k
  x
k 1
 x
1
k 1
k
 x   x  .
1 
3
k

 
p 1
x
k 1
.
, m  k  1 . It
follows from (1) that
x
m1
(1)
 x
k

If the latter is the inequality p   then
xx
k  1 , or
x  x
k

x  x
In the general case, if
xi  xi 
k
k
particular,
x   x
1
0

k 1
 x
k
(2)
1 
 x    x
k1
k
x   x
k
k 1
and if x   x
k
1 q
  , then
q

k 1
x  x
k
  , then
  , where
  , (i  1,2,..., n) .
It follows from (2) that x  x
In
x

k
k 1
 x   x  .
1 
1
If   , then
2
k
x  x    .
q    1; and
k 


k
 x   x
1
1 
x    ,
0
if
     
k 
0
.
x     
1
then
and

xx
k 


k 1
1 
  .
(2').
The Necessary and Sufficient Conditions of convergence of Iterative
Process of the LAES
We consider LAES
x x  
(1)
Theorem 1 In order that for any initial vector х  and any free term  the
iteration process
k
k 1
x     x     , k  1,2,
(2)
to be convergent the condition  ( )  1 it is necessary and sufficient.
0
Proof.
(2)

x   ( E     2  ...   k 1 )   k x  .
k
0
(3)
It follows that for any initial vector х  and any free term  the iteration
process (2) to be convergent is equivalent to convergence the geometric series
0
E      ....   
2
k

  k .
(4)
k 0
If is correctly the condition
j 1
j  1,2, , n ,
4
(5)
where  j , j  1,2, , n j  1,2, , n is eigenvalues of matrix  , then by
Theorem of convergence geometric series the geometric series (4) it is
convergent. Then  k  0 k   . It follows from equality (3) that for any
0
initial vector х  and any free term  the iteration process is convergence or
lim x( k )  x , where х is solution of system (1).
k 
Corollary 3 In order that the iteration process (2) is convergence the
condition
(6)
 m,l ,k  1
it is sufficient conditions.
Remark 1 We consider the linear system
Ax  b ,
(7)
0 
 a11
b  (b1 ,..., bn ) . D  diagA  
  0.
ann 
0
In order that the system (7) to reduce to system (1) we transformed the matrix
A : A  D  ( A  D) . Then Dx  b  ( A  D) x and det D  a11   ann  0, then
where A  aij 
,
i , j 1
n
From here   D1 ( D  A) ,
x  D1b  D1 ( D  A) x .
  D1b . As it is
from (7) for b and x  in order that the iteration process is convergence
(8)
det( D1 ( D  A)   E )  0 or det(   E)  0
0
 det(D
the condition
1
( D  A)   E )  det  D 1 ( D  A)   D 

i  1, i  1,2, , n
is satisfied. The condition (9) is necessary and sufficient conditions.
It follows from (8) that
det D 1  det  ( D  A)   D   0 or det   D  ( A  D)   0 .
Or
a11   a12
a1n
a21 a22  
a2 n
an1 an 2
ann  
 0.
 det  D  A   D  det  1  D  ( A  D)   det   D  ( A  D) 
5
(9)