Solution 1 14/08/02
1
Solution 1: Solution of nonlinear equations
Find the first non-zero and positive root xr of the function f (x) = tan x − x depicted
in Figure 1 using Newton’s method, the secant method, and bisection with a tolerance of
Figure 1: The function f (x) = tan x − x, showing the first three non-zero positive roots.
= 10−7 . Use |x2 − x1 | as a measure of the error for Newton’s method, |x3 − x2 | for the
secant method, and |x2 − x1 |/2 for bisection.
Part 1
Present a plot comparing the convergence histories of the logarithm of these errors (log10 (Error))
as a function of the number of iterations n.
The convergence histories of each method are tabulated at the end of this solution set. The
starting values for each method are given by:
• Newton’s method: x1 = 3π/2 − ∆x, ∆x = 0.005.
• Secant method: x1 = 3π/2 − ∆x, x2 = 3π/2 − 2∆x, ∆x = 0.005.
• Bisection: x1 = 3π/2 − ∆x, x2 = π, ∆x = 0.005.
The most accurate root is given by the Newton iteration, for which xr = 4.493409. The error
history for all three methods is shown in Figure 2.
Solution 1 14/08/02
2
Figure 2: Convergence history for the three methods when solving for the first root of
tan x − x = 0.
Part 2
Comment on the convergence behavior of each method.
Newton’s method converges the fastest, followed by the secant method, and then the bisection
method. The convergence rate of the bisection method is independent of the function because
the error is simply reduced by a half at each time step, that is,
1
|en+1 | = |en | ,
2
(1)
where en is the error at the nth iteration and the initial error is e0 = |x1 − x2 |/2. For
this reason it is the slowest to converge, requiring 24 iterations. Newton’s method and the
secant method behave similarly because the secant method is the discrete analog of Newton’s
method. Newton’s method converges quadratically with
en+1 = ke2n ,
(2)
where k is dependent upon the function in which the roots are being found, and the Secant
method converges almost quadratically with
en+1 = ken−1 en .
(3)
Solution 1 14/08/02
3
Newton’s method converges in 11 iterations with an error of 2.34 × 10−11 and the secant
method converges in 15 iterations with an error of 2.91 × 10−9 . These two methods converge
to an error that is much less than the specified tolerance of = 10−7 because both methods
reduce their errors by a significant amount on the last iteration. The error increases initially
for both methods because the distance between successive approximates to the root increases
initially, due to the nature of the function, but then decreases again as the methods converge
upon the root. The error in some senses is misleading, because it is a measure of the distance
between successive approximates, rather than the distance of the approximates to the exact
value of the root.
Part 3
How sensitive is each method to the starting values?
The convergence of the bisection method is independent of the starting values as long as
they bracket the desired root. We can find out exactly which starting values will converge
for Newton’s method. Newton’s method is given by
x2 = x1 −
f (x1 )
,
f 0 (x1 )
(4)
for which, when f (x) = tan x − x, becomes
x1 sec2 x1 − tan x1
.
(5)
tan2 x1
Since we must have π/2 < x2 < 3π/2 for the solution to converge to the first root (otherwise
it will converge to some other root), then we must have
x2 =
π
x1 sec2 x1 − tan x1
<
,
2
tan2 x1
and
(6)
x1 sec2 x1 − tan x1
3π
<
.
(7)
2
tan x1
2
For the first part (6) to be satisfied, we must have x1 > π/2. For the second part (7)
to be satisfied as an equality, we have one of three possible roots, namely, xs1 = 2.2696,
xs2 = 4.2877 and xs3 = 3π/2. From this we see that if the starting value is given by
π/2 < x1 < 2.2696, the iteration will not converge, but if we have a starting value of
4.2877 < x1 < 3π/2, it will converge to the correct root, even though if the initial guess is
very close to, but less than, 3π/2, it may take a long time to converge. Any other values will
cause the Newton iteration either to converge to another root or to not converge at all.
The secant method behaves similarly to Newton’s method, only that now there are two
starting values. If the first starting value x1 satisfies the same requirements as Newton’s
method and the second starting value is closer to the root than x1 , then the secant method
will converge. If x2 is further from the root than x1 but still in the range specified by the
Newton solver, it will still converge, but convergence will be slower. There are still an infinite
number of other starting values for the secant method that will converge on the correct root,
and they do not all necessarily have to lie in the range π/2 < x1 , x2 < 3π/2. An analysis to
determine what these are is beyond the scope of this assignment.
Solution 1 14/08/02
4
Part 4
What happens if you use a fixed-point iteration with g(x) = tan(x) to find the root xr ?
Show graphically that if the iteration starts slightly to the left of the root then it diverges
to the left, while if it starts to the right of the root then it diverges to the right. You only
need to show two steps of the iteration.
Figure 3 shows the fixed-point iteration track of what happens when you try to find the
root starting slightly to the right of it, while Figure 4 shows what happens when you use
fixed-point iteration when starting slightly to the left of the root. Evidently, the iteration
does not converge. This occurs because there are no values in the vicinity of the root xr for
which |g 0 (x)| = | sec2 x| < 1, since | sec2 x| ≥ 1 for all x.
Figure 3: Fixed point iteration with g(x) = tan x, showing the divergence of the iteration
when starting slightly to the right of the root.
Solution 1 14/08/02
5
Figure 4: Fixed point iteration with g(x) = tan x, showing the divergence of the iteration
when starting slightly to the left of the root.
Solution 1 14/08/02
6
Newton’s Method
n
1
2
3
4
5
6
7
8
9
10
11
x1
f (x1 )
f 0 (x1 )
x2
|x1 − x2 |
4.707385 195.131858 39935.723148 4.702499 4.89E-03
4.702499 96.405125 10222.751663 4.693068 9.43E-03
4.693068 47.058782 2678.254007 4.675498 1.76E-02
4.675498 22.418888
734.105731 4.644959 3.05E-02
4.644959 10.162683
219.266242 4.598610 4.63E-02
4.598610 4.152406
76.580277
4.544387 5.42E-02
4.544387 1.351823
34.765293
4.505503 3.89E-02
4.505503 0.258915
22.699680
4.494097 1.14E-02
4.494097 0.013922
20.322238
4.493412 6.85E-04
4.493412 0.000045
20.191153
4.493409 2.23E-06
4.493409 0.000000
20.190729
4.493409 2.34E-11
Secant Method
n
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
x1
4.711385
4.706385
4.705409
4.699603
4.693042
4.681417
4.664875
4.640767
4.609055
4.571615
4.535113
4.508554
4.496364
4.493620
4.493412
x2
f (x1)
f (x2 )
x3
|x3 − x2 |
4.706385 991.323676 161.847787 4.705409 9.76E-04
4.705409 161.847787 138.567297 4.699603 5.81E-03
4.699603 138.567297 73.503946 4.693042 6.56E-03
4.693042 73.503946 46.989102 4.681417 1.16E-02
4.681417 46.989102 27.595009 4.664875 1.65E-02
4.664875 27.595009 16.365562 4.640767 2.41E-02
4.640767 16.365562
9.297514 4.609055 3.17E-02
4.609055 9.297514
5.033811 4.571615 3.74E-02
4.571615 5.033811
2.484975 4.535113 3.65E-02
4.535113 2.484975
1.046592 4.508554 2.66E-02
4.508554 1.046592
0.329232 4.496364 1.22E-02
4.496364 0.329232
0.060501 4.493620 2.74E-03
4.493620 0.060501
0.004254 4.493412 2.08E-04
4.493412 0.004254
0.000059 4.493409 2.93E-06
4.493409 0.000059
0.000000 4.493409 2.91E-09
Solution 1 14/08/02
Bisection Method
n
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
x1
4.711385
4.711385
4.711385
4.515161
4.515161
4.515161
4.515161
4.502897
4.496765
4.493699
4.493699
4.493699
4.493699
4.493507
4.493411
4.493411
4.493411
4.493411
4.493411
4.493411
4.493410
4.493410
4.493410
4.493409
x2
f (x1 )
f (x2 )
x3
f (x3 )
|x2 − x1 |/2
3.141590 991.323676 -3.141593 3.926488 -2.927494 7.85E-01
3.926488 991.323676 -2.927494 4.318936 -1.909860 3.92E-01
4.318936 991.323676 -1.909860 4.515161 0.489190
1.96E-01
4.318936 0.489190 -1.909860 4.417048 -1.130151 9.81E-02
4.417048 0.489190 -1.130151 4.466105 -0.488188 4.91E-02
4.466105 0.489190 -0.488188 4.490633 -0.055342 2.45E-02
4.490633 0.489190 -0.055342 4.502897 0.200511
1.23E-02
4.490633 0.200511 -0.055342 4.496765 0.068831
6.13E-03
4.490633 0.068831 -0.055342 4.493699 0.005846
3.07E-03
4.490633 0.005846 -0.055342 4.492166 -0.024968 1.53E-03
4.492166 0.005846 -0.024968 4.492932 -0.009617 7.67E-04
4.492932 0.005846 -0.009617 4.493315 -0.001900 3.83E-04
4.493315 0.005846 -0.001900 4.493507 0.001970
1.92E-04
4.493315 0.001970 -0.001900 4.493411 0.000034
9.58E-05
4.493315 0.000034 -0.001900 4.493363 -0.000933 4.79E-05
4.493363 0.000034 -0.000933 4.493387 -0.000450 2.40E-05
4.493387 0.000034 -0.000450 4.493399 -0.000208 1.20E-05
4.493399 0.000034 -0.000208 4.493405 -0.000087 5.99E-06
4.493405 0.000034 -0.000087 4.493408 -0.000026 2.99E-06
4.493408 0.000034 -0.000026 4.493410 0.000004
1.50E-06
4.493408 0.000004 -0.000026 4.493409 -0.000011 7.49E-07
4.493409 0.000004 -0.000011 4.493409 -0.000004 3.74E-07
4.493409 0.000004 -0.000004 4.493409 0.000000
1.87E-07
4.493409 0.000000 -0.000004 4.493409 -0.000002 9.36E-08
7