AMS 147 Computational Methods and Applications
Lecture 04
Copyright by Hongyun Wang, UCSC
Recap of Lecture 3:
Convergence analysis of xn +1 = g ( xn ) :
Case #1:
Case #2:
Case #3:
Suppose g( x
)
> 1 at a fixed point x*.
Then xn +1 = g ( xn ) cannot converge to x*.
There are 2 possibilities:
*) It may diverge to infinity
*) It may converge to a different fixed point.
Order of convergence:
Let en = xn - x*.
Linear convergence:
en +1 en
Quadratic convergence:
en +1 ( en )
2
Multiplicity of a root
Let x* be a root of ƒ(x) = 0.
Taylor expansion:
( )
( )( x x ) +
f ( x) = f x + f x
( )
f x =0
2!
(x x )
2
++
( )
f ( p) x
p!
(x x )
p
+
The multiplicity is defined based on the first non-zero term in Taylor expansion.
( )
x* is a simple root if and only if f x 0
Convergence and order of convergence of Newton’s method
We have studied the convergence and the order of convergence for Newton’s method,
respectively near a simple root and near a root of multiplicity p > 1.
We summarize the conclusions in the table below.
-1-
AMS 147 Computational Methods and Applications
Behavior of Newton’s method
Near a simple root
Near a root of multiplicity p > 1
It converges if we start sufficiently close to the solution
Convergence
Order of
convergence
Quadratic convergence
Linear convergence
Question: How to regain quadratic convergence near a root of multiplicity > 1?
Answer:
Schroder’s method:
Let x* be a root of f ( x ) = 0 with multiplicity p > 1.
Consider a modified version of Newton’s method
xn +1 = xn f ( xn )
f ( xn )
where is to be determined.
Goal: To regain quadratic convergence.
The iteration function is
g( x) = x f ( x)
f ( x)
To have quadratic convergence near x*, we need
( )
g x = 0
x* is a root of multiplicity p > 1.
( )
==>
f x = 0
==>
( f ( x )) f ( x ) f ( x )
g ( x ) = 1 ( f ( x ))
2
2
(the denominator will be zero at x ).
( )
We use Taylor expansion to calculate g x .
(skip the derivation in lecture).
f ( x) =
( )
f ( p) x
f ( x) = p
p!
(x x )
( )
f ( p) x
p!
p
+
(x x )
p 1
( )
is not good for calculating g x .
+
-2-
AMS 147 Computational Methods and Applications
f ( x) 1
=
x x + f ( x) p
(
)
We write g(x) as
f ( x)
f ( x)
g( x) = x = x (
)
1
x x + p
(
)
= x + x x (
)
1
x x + p
1
= x + 1 x x + p
(
)
Comparing with the Taylor expansion of g(x) around x g( x ) = g( x ) + g ( x
)( x x ) + we obtain that
1
g x = 1 p
( )
( )
To have g x = 0 , we set
=p
Schroder’s method:
f ( xn )
xn +1 = xn p
f ( xn )
Note: Schroder’s method requires that we know the multiplicity of the root.
Solving minimization problems
Definition:
If f ( x ) f ( x * ) for all x, then we say ƒ(x) attains the minimum at x*.
Notation:
min f ( x) = f ( x )
x
( min f ( x) means the minimum value of ƒ(x) )
x
argmin f ( x ) = x
x
-3-
AMS 147 Computational Methods and Applications
( argmin f ( x ) means the location where the minimum value of ƒ(x) is attained)
x
(Draw a graph of ƒ(x) to illustrate min f ( x) and argmin f ( x ) )
x
x
The golden search method
Suppose f ( x ) is concave up and attains the minimum in [a, b] .
(Draw graphs to illustrate a concave up function and a concave down function)
Goal: To find arg min f ( x ) (the location at which f ( x ) attains the minimum).
x
Strategy:
Start with [a, b] that contains arg min f ( x ) .
x
After each iteration, we obtain a smaller interval that contains arg min f ( x ) .
x
First we point out that bisection does not work.
a+b
, then it is not possible to determine which
2
sub-interval contains the location of minimum.
If we look only at the function value at c =
(Draw graphs of two functions to demonstrate that the location of minimum may be in
a+b
either half. Knowing the function value at c =
does not help us to determine which
2
half contains the location of minimum.)
So we consider two points in [a, b]
r1 = a + (b a)(1 g)
r2 = a + (b a)g
We require
0.5 < g < 1
So we have
a < r1 < r2 < b
(The value of g is to be determined later.)
(Draw the graph of a concave up f ( x ) with a, b, r1, r2.)
If f ( r1 ) < f (r2 ) ,
continue with [a, r2 ] .
Otherwise,
-4-
AMS 147 Computational Methods and Applications
continue with [r1 , b] .
If the size of interval < tol,
stop.
Now we look at the computational cost of the search method described above.
The number of function evaluations per iteration = 2.
Question: How to reduce it from 2 to 1?
Suppose we continue with [a, r2 ] .
b .
Let us denote the new interval [a, r2 ] by a,
a˜ = a ,
b˜ = r
2
b˜ a˜ = r2 a = (b a) g
b are
The two points for the new interval a,
˜r1 = a˜ + (b˜ a˜ )(1 g) = a + (b a) g(1 g) ,
˜r2 = a˜ + (b˜ a˜) g = a + (b a) g2
r , r )
b,
(Draw the real axis to show a, b, r1 , r2 , a,
1 2
To reduce the number of function evaluations, we like to have
r2 = r1
So we can use f ( r2 ) = f ( r1 ) to save one function evaluation.
This reduces the number of function evaluations per iteration to 1
(except for the very first iteration).
r2 = r1
==>
a + ( b a ) g 2 = a + ( b a ) (1 g )
==>
g2 = (1 g)
Solving the quadratic equation, we obtain
g=
5 1
0.618
2
This number is called the golden ratio.
The method using g =
5 1
is called the golden search method.
2
-5-
AMS 147 Computational Methods and Applications
Matlab code (golden search method for calculating arg min f ( x ) )
x
clear;
a = 0;
b = 2;
tol = 1.0e10;
n = 0;
%
g=(sqrt(5)1)/2;
r1=a+(ba)*(1g);
f1=f(r1);
r2=a+(ba)*g;
f2=f(r2);
%
while (b-a) > tol,
n = n+1;
if f1 < f2,
b=r2;
r2=r1;
f2=f1;
r1=a+(ba)*(1g);
f1=f(r1);
else
a=r1;
r1=r2;
f1=f2;
r2=a+(ba)*g;
f2=f(r2);
end
end
x0 = (a+b)/2;
-6-
AMS 147 Computational Methods and Applications
In a separate file named “f.m”
function [y]=f(x)
y=exp(x/4)sin(x);
Convergence of the golden search method:
The golden search method has guaranteed convergence.
After each iteration, we either continue with [a, r2] or with [r1, b].
r2 a = a + ( b a ) g a = ( b a ) g
b r1 = b a ( b a ) (1 g ) = ( b a ) g
It is always true that
(The size of new interval) = (the size of old interval) g
That is, after each iteration, the size of interval is multiplied by a factor of g 0.618.
Let N be the number of iterations needed to reach the error tolerance.
(b a) gN tol
tol
ba
==>
gN ==>
tol N log( g) log
b a
==>
tol log
b a
N
log( g)
Example: a = 0 , b = 2 , tol = 1010
==>
==>
tol log
b a
N
49.3
log( g)
N = 50
(Go through sample codes in assignment #1)
(Computer illustration of case #3 in convergence analysis)
(Computer illustration of order of convergence)
-7-