Systems of Non-Linear Equations
1
Objective
Finding the roots of a set of simultaneous
nonlinear equations (n equations, n unknowns).
f1 ( x1 , x2 ,..., xn )  0
Example,
f 2 ( x1 , x2 ,..., xn )  0
x 2  xy  10

f n ( x1 , x2 ,..., xn )  0
y  3 xy2  57
where each of fi(x1, … xn) cannot be expressed in
the form
a1x1  a2 x2  ...  a2 x2  b  0
2
Review of iterative methods
for finding unknowns
Finding x that satisfies f(x) = 0 (one equation, one
unknown)
- Fixed-Point Iteration
- Newton-Raphson
- Secant
Solving Ax = b or Ax – b = 0, or finding multiple x's
that simultaneously satisfy a system of linear
equations
- Gauss-Seidel
- Jacobi
Variation of Fixed-Point
Iteration
3
Fixed Point Iteration
x  xy  10
To solve
2
Ans : x  2, y  3
y  3xy  57
2
We can create updating formula as
10  x
xi 1 
yi
xi 1  10  xi yi
2
i
yi 1  57  3xi yi2
or
57  yi
yi 1 
3xi
Which formula will converge?
What initial points should we pick?
4
Fixed Point Iteration
x0  1.5, y0  3.5
10  xi2
xi 1 
yi
xi 1  10  xi yi
yi 1  57  3x y
57  yi
yi 1 
3xi 1
x1  2.21429
x1  2.17945
2
i 1 i
y1  24.37516
x2  0.20910
y2  429.709
Diverging
y1  2.86051
x2  1.94053
y2  3.04955
Converging
5
Fixed Point Iteration – Converging Criteria
For solving f(x) = 0 (one equation, one unknown),
we have the updating formula
xi 1  g ( xi )
Trough analysis, we derived the following
relationship
 i 1  g ' ( x) i
which tells us convergence is guaranteed if
g ' ( x)  1
6
Fixed Point Iteration – Converging Criteria
For solving two equations with two unknowns, we
have the updating formula
xi 1  u ( xi , yi )
yi 1  v( xi , yi )
Through similar reasoning, we can demonstrate that
convergence can be guaranteed if
u v

1
x x
u v

1
y y
For single equation
dg
g ' ( x ) or
1
dx
7
Fixed-Point Iteration – Summary
• Updating formula is easy to construct, but
updating formula that satisfy (guarantees
convergence)
f1 f 2
f n

 ... 
 1, for i  1, ..., n
xi xi
xi
is not easy to construct.
• Slow convergent rate
8
Newton-Raphson (one equation, one unknown)
Want to find the root of f(x) = 0.
From 1st-Order Taylor Series Approximation, we have
f ( xi 1 )  f ( xi )  f ' ( xi )( xi 1  xi )
Idea: use the slope at xi to predict the location of the root. If
xi+1 is the root, then f(xi+1) = 0. Thus we have
0  f ( xi )  f ' ( xi )( xi 1  xi )
  f ( xi )  f ' ( xi )( xi 1  xi )

f ( xi )
 xi 1  xi
f ' ( xi )
 xi 1
f ( xi )
 xi 
f ' ( xi )
Single-equation form
9
Newton-Raphson (two equations, two unknowns)
Want to find x and y that satisfy
u( x, y )  0
v ( x, y )  0
From 1st-Order Taylor Series Approximation, we have
ui
ui
ui 1  ui  ( xi 1  xi )
 ( yi 1  yi )
x
y
and
vi
vi
vi 1  vi  ( xi 1  xi )
 ( yi 1  yi )
x
y
Using similar reasoning, we have ui+1 = 0 and vi+1 = 0.
continue …
10
Newton-Raphson (two equations, two unknowns)
Replacing ui+1 = 0 and vi+1 = 0 in the equations yields
ui
ui
0  ui  ( xi 1  xi )
 ( yi 1  yi )
x
y
vi
vi
0  vi  ( xi 1  xi )
 ( yi 1  yi )
x
y

ui
u
u
u
xi 1  i yi 1  ui  xi i  yi i
x
y
x
y
vi
vi
vi
vi
xi 1 
yi 1  vi  xi
 yi
x
y
x
y
continue …
11
Newton-Raphson (two equations, two unknowns)
Solving the equations algebraically yields
xi 1
vi
ui
ui
 vi
y
y
 xi 
,
ui vi ui vi

x y y x
ui
v
 ui i
x
x
yi 1  yi 
ui vi ui vi

x y y x
vi
Alternatively, we may solve for xi+1 and yi+1 using well-known
methods for solving systems of linear equations
 ui
 x

 vi
 x
ui 
 ui
ui   x
y   xi 1 
      
vi   yi 1 
 vi   vi
y 
 x
ui 
y   xi 

vi   yi 
y 
12
Newton-Raphson Example
To solve
Ans : x  2, y  3
u( x, y)  x 2  xy  10  0
v( x, y)  y  3xy2  57  0
First evaluate u  2 x  y, u  x, v  3 y 2 , v  1  6 xy
x
y
x
y
With x0 = 1.5, y0 = 3.5, we have
u0
u0
 2(1.5)  3.5  6.5,
 1.5
x
y
v0
v0
 3(3.5) 2  36.75,
 1  6(1.5)( 3.5)  32.5
x
y
u0  (1.5) 2  1.5(3.5)  10  2.5
continue …
v0  3.5  3(1.5)( 3.5) 2  57  1.625
13
Newton-Raphson Example
vi
u
 vi i
y
y
 xi 
,
ui vi ui vi

x y y x
ui
xi 1
ui
vi
vi
 ui
x
x
yi 1  yi 
ui vi ui vi

x y y x
From these two formula, we can then calculate x1 and y1 as
 2.5(32.5)  1.625)(1.5)
x1  1.5 
 2.03603
6.5(32.5)  1.5(36.75)
1.625(6.5)  ( 2.5)( 36.75)
y1  3.5 
 2.84388
6.5(32.5)  1.5(36.75)
These process can be repeated until a "good enough"
approximation is obtained.
14
Newton-Raphson (n equations, n unknowns)
Want to find xi (i = 1, 2, …, n) that satisfy
f1 ( x1 , x2 ,..., xn )  0
f 2 ( x1 , x2 ,..., xn )  0

f n ( x1 , x2 ,..., xn )  0
From 1st-Order Taylor Series Approximation, we have
f k ,i
f k ,i
f k ,i 1 f k ,i  ( x1,i 1  x1,i )
 ...  ( xn,i 1  xn,i )
x1
xn
15
Newton-Raphson (n equations, n unknowns)
For each k = 0, 1, 2, …, n, setting fk,i+1 = 0 yields
f k ,i
f k ,i
 ...  ( xn ,i 1  xn ,i )
0  f k ,i  ( x1,i 1  x1,i )
xn
x1
f k ,i
f k ,i
f k ,i
f k ,i
xn ,i
x1,i  ... 
xn ,i 1   f k ,i 
x1,i 1  ... 

xn
x1
xn
x1
These equations can be expressed in matrix form as
Zx i1  f  Zx i
where
 fx1,i
 f 21,i
Z   x1
 
 f n ,i
 x1
f1,i
x2
f 2 ,i
x2

f n ,i
x2

 x1,i 1 
 x1,i 
 f1,i 

x 
x 
f 

 x   2,i 1  x   2,i  f   2,i 
i 1
i
i







  
 


 
 
f n ,i 
x
x
 xn 
 n ,i 1 
 n ,i 
 f n ,i 

f1,i
xn
f 2 ,i
xn
16
Newton-Raphson – Summary
• Updating formula is not convenient to construct.
• Excellent initial guesses are usually required to
ensure convergence.
• If the iteration converges, it converges quickly.
17