Revista de la
U nión Matemática Argentina
53
Volumen 39, 1994.
ACCELERATING SIM PLIFIED MONOTONE N EWT O N ITERAT I O N S
S . A B D E L MASIH A N D J . P . MILASZEWICZ
ABSTRACT. Simplified Newton iterations are analyzed in the context given by the
hypotheses of the monotone Newton theorem; it is known that they produce upper and
lower seqllences with mOIlotone convergence to a root of the nonliIlear system Fy = O. It
is proved here that accurate partial elimination accelerates convergenc e in both sequences,
while retaining monotoIlicity.
1 . INTRODUCTION.
The monotone Newton theorem ( See [6] ) and some of its generalizat ion s give conditiollS
that generate two mOllotonl" seqllences, OIle decreasing and OIle ill<Tl"asing, both convf'r gi n g
at least quadratical1y to the same root of the nonlinear system Fy = O . This resu l t
has three main advantages , when compared with the usual Newton cOllvergence tlll'orenl .
Firstly, no a priori knowledge of the existence of a root is assumed: sf'rondly, a rel i able
bound for the error in thc determinatioll of the exact root is providf'd hy the vector
di fference of any two terms. one from the up p er sequellce ami allotlwr from tlw lower (me ;
final1y, if t h e s e two terms corres p oll d to thl" s ame iterate. the hOUllds on thl" error t hns
obtained converge to O at least quadrat ic al 1y. On the o t her hand, a c t u al app l i ca tioll of tIl('
result implies illcreased work per i t erat i on, because two linear systems per s tep , inst ead of
one. mu st be solved with the saml" Jacobian matrix. This obj ect ion may loose p a rt of its
value, if, for installce, a two-processor p ar al lel computer is available.
The hyp ot h e se s of the monutone Newtoll theorem will be ass ume cl thronghout t h i s notl'. In
t hi s context, a gain in convergenCf' spe{'cl can be o h t ai ned by p ar tial rl"<luction of tllf' ¡!litial
systern, if ap pli ecl ac cur at ely ; w e s ay that an unknown has bef'll accurat el>, eliminatp(l, if
the equation used for the elimillation has the same inclpx ( See [4] ) ; whe ther thl" acct>leratioIl
t h us obtained can hf' compleIllented by t h e reclur t ion of computa tional work dep e n cls OIl the
p rob l em dealt with and t he choice of u n k now n s to b(' el i m i n at e cl . Anot h('1" usefll l possihility
is given by the simplifiecl monotone Newton iterations; wi t h t hem, ll p c l a t in [!; of thl" J acohiaIl
rnatrix i s don e e ver y p i tera t ions ( we c on s i de r he1'e only fixe¡l u p d a t ill g steplength p. fol'
th(' sake of sirnplicity ) : in this case, an extension of t he m o no tone Xt'wton t heorC ll l says
that p ai r wi s e rnonotone convergellce is o bt ai ned with ronve1'genCf' orcler grpa t er o � eq1 \al
than p + 1 ( See t h eorem 4 in [í] ) .
The main objectivp i n this paper is t o prove that accurate p art i al rl"cluction i n t he ori gin al
system, via p art i al eliminatioll of unknowns, yi elcls improvement in convergenct' of the
simplified Newton iterates , as in the ( n on s i mp lified ) monotone Newton context o
54
2 . THE
SIMPLIFIED NEWTON�FOURIER ITERATIONS.
Consider a continuously differentiable function F : D e 3?n ----+ 3?n , for which we seek
solution to the equation
Fy =
°
.
a
(2. 1 )
It will b e assurned that we have x O ::::: yO , i.e. x? ::::: y? , for
1
::::: i :::::
n,
are such that
and
We also suppose that F is order convex on ( XO , yO ) , i.e.
F( >.x + ( 1 - '\ )y) ::::: '\ F x + ( 1 - '\)Fy
whenever
x
:::::
y
or y ::::: x
and ,\ E
(0, 1 ) .
The following result is known as the rnonotone N ewton theorern .
2 . 1 . Suppose that for each x E ( XO , yO ) , F '( x ) is a nonsingular M-rnatrix (i. e.
(F ' ( x ) ) i , j ::::: 0 , for i -=f. j , and F '(x ) - 1 is nonnegative). Then the Newton iterates
THEOREM
k
=
0, 1 , . . .
( 2.2)
satisfy yk 1 y* E ( xO , yO ) . as k -+ 00 and y* is the unique solution of (2. 1) in ( XO , yO ) .
Moreover, if F ' is isotone on ( :rO , yO ) (i. e. :r ::::: y irpplies F ' ( x ) ::::: F ' ( y ) ), then the
Néwton-Fourier (N-F) iterates
k = 0, 1 , . . .
satisfy :rk i y* as k
-+ oo .
(2.3 )
vVe also have
,,�
= 0, 1 , . . .
( 2 .4 )
Pinally, if for sorn e norm.
1I F ' ( x )
-
then there exists a constant
F ' ( y ) 1 1 ::::: , 11 :r
e
-
yll
(2.5)
such that
k = 0, 1 , . . .
PROO F :
See [6] .
(2.6)
55
REMARK 2 . 2 . Apparently, it has been Baluev (See [2) ) , who first realized that the iterat es
in ( 2.3 ) generate a complementary sequence to the one given by (2.2); we shall call thc
couple (x k , yk ) the Newton-Fourier (N-F) iterates. Other possible formulations of the
theorem aboye are obtained by interch angin g the roles of x O and y O , and also by supposing
that F is order concave (See Table 1 3 . 1 in [6) ) . Note that isotonocity of F I implies order
convexity of Fj however, sorne partial resul ts do not need the isotonicity hypothesis.
In the context given by the theorem aboye, it is possible to introduce simplified monotone
Newton iterations with a fixed steplength p :::: 1 as follows:
For k
yk , O :
x k,o
=
=
: =
0, 1 , . . .
yk
xk
For i = 1 , . . .
yk ,i : = y k,i- l
xk ,i
yk+ l :
x k+ l :
=
=
:=
yk,p
xk,i- l
,p
_
F ' ( yk ) - l Fyk , i - l
_
F ' ( y k )- l Fx k , i- l
x k,p
( 2.7)
These simplified i t erations can be useful in two basic situations, i.e. when the J acobian
matrix F / ( y ) is difficult to calculate, and when F ' ( y ) becomes ill conditioned for y near
the root ; thus making it impractical to further apply the Newton mcthod. Thc following
theorem extends the monotone Newton theorem to the simplified N-F iterations .
THEOREM 2 . 3 . Tbe sequen ces ( x k ) and ( y k ) satisfy:
(i) Fxk ,i :::; ° :::; Fy k , i V I :::; i :::; p
(ii) x k :::; xk , i - l :::; x k ,i :::; x k + l :::; yk+ l :::; y k , i :::; y k , i - l :::; yk
(iii) ( x k ) and ( y k ) botb converge to y * .
(iv) Tbere exists bp sucb tbat
k
V
=
I :::; i :::;
0, 1 , . . .
p
( 2.8 )
PROOF : (i) and (ii) can be proved inductively, as the correspondi ng statements in Theorem
2 . 1 ; ( i ii) then also follows as in Theorem 2 . 1 .
( iv)Not e first that , sinee ( xO , yO ) is compact and F ' i s eontinuous, there exists b such that
66
It is not difficult to see that the following inequalities hold
I l y k + l - X k + l ll = I l y k,p X k, p 11 = ll y k, p- l - X k, p- l - F ' ( y k )-l (F y k, p -l Fx k,P-l ) 1 I
X k, P -l ) Fy k , p -l + Fx k, P - 1 1 1
� I I F ' ( y k )- l I I II F '(y k ) ( y k, p-l
1
( F '( yk ) F '(x k,p- l + t ( y k, P � 1 x k , P - l ) ) , y k ,P - l X k ,P - I ) dt
� b
_
_
_
_
l
11
(11 I I F ' ( yk ) - F ' (X k ,P-I + t( y k ,P- l X k ,p-l » 1 1 dt) I l yk,p-l ,rk,P- I II
(1 1 Il y k - x k ,P � l - t(yk ,p- l X k,p-l ) 1 1 dt) I l y k ,p - l x k 'P � l l l
( 1 l y k - ;r k,P- 1 1 1 + � I I ( yk,p-l - Xk ,p-l ) 11 ) I l y k ,p-l - x k ,P - 1 11
(� rs) Il y k - xk 1 1 I l yk ,p-l - xk,P-1 1 1
_
� b
� b
� b
� b
I
_
I
I
11 �
rl l
11 00
;= h,( �rs ), we have
= 1 , we e ssent i ally have ( 2.6 ) ;
REMA RK
_
_
11
Thus, if we set b1
_
_
where
If p
_
and
11
1 1 00 �
sil
11
.
if p '¡' 1 , we finally get
2 .4 . Theorem 2 . 3 sli ghtly improves a result stated by Wolfe ( See Theorem
4
in
[7) ) . Wolfe ' s proof of ( 2 . 8 ) is based on S a t z 4 in [1] , which assumes F " to be continuous:
the proof aboye
assumes
the somewhat weaker hypothesis ( 2 . 5 ) . Also. the inequalit ies
stated here for the inner loops iterates will be neede d in the following s{'ction.
TH EOREM 2 . 5 . Consider two steplengtbs p and q wi tb p
N-F seq uences by
tben
( x� , y; )
and
x s ,j <
q
P ROOF : Notice that
( x� , y : ) .
xk,i
- p
If kp +
í
=
" q
<
+
q , and denote tbe corresp onding
j , wi tb O � i < p and O � j < q .
< y* < y k , i < y S . J
-
yl = yO,P.
thus
q
'
p
An induction argument completes the proof.
- p - q
3. ELIMINATION AND NEWTON-FOURIER ITERATIO N S .
The main result will be proved in this section, i.e. that accurate partial elimination im­
proves convergence oí the simplified Newton iterations. In this way we extend the corre­
sponding result in the mntext oí Theorem 2 . 1 ( See Theorem 3.9 in [ 4] ) . Some additional
material is taken almost verbatim from [4] .
Since F ' ( y* ) is a nonsi ngul ar M-matrix, it íollows that 8dI (Y* ) i:- O; the implicit fundion
theorem yiel d s the existence oí neighborhoods U oí y * , V oí y* =
: (Y2 " ' " Y� ) and a
funct ion 9 : V ---+ �, such that h ( g( y) , y)= O ; if moreover y E U is such that h (y) = O ,
then VI = g(y) (y : = ( Y 2, ' " , Yn ) ) . We as sume throughout that ( X O , y O ) C U and that
( X O , ll)
e
V; unnecessary distinction between row and column vectors will be avoided.
It is possi ble now to eliminate YI in ( 1 . 1 ) , by means of g , and get the reduced syst.em
(3.1 )
F := (Ji )
where
THEOREM
i
=
2,
..
.
)n
,
: fi ( g( Y) , y)
y E V , and fl ff) =
3 . 1 . The following propositions hold:
(i ) g is isotone on
(
.1' 0 ,
yo
)
and
(ii) F ' (y) is an M-matrix for
yE
x � � g(x k ) � y ; � g ( y k ) � y f
( X O , yo )
( F ' ) ¡ / ( y ) = ( F ' r ; / ( g (fi), y)
, k= 0 , 1 , . . .
and
, for
( i:- 1 i:- j
(3.2 )
(
0\
(III' ) F " 18 180tone on 0
.1' , Y / .
'
' '
(iv)
PROOF :
F .rú
.
� O s:
F yo
See Lemmas 3.2, 3 . 3 , 3.5 and 3 . 7 in [4] .
)
REMA RK 3 . 2 . Theorem 3 . 1 gives the possibility of applying Theorem 2 . 1 to the reduced
y o ; it has been proved in [4] that under appropriate
system ( 3. 1 ) with starting interval
conditions the N F iterates arising from the reduced system ( 3 . 1 ) converge better than those
generated by the original system ( See Theorem 3.9 in [4] ). Our aim is to extend this result
to the simplified Newton sequences; the conclusions in Theorem 3 . 1 , especially (3.2 ) , and
Theorem 2.3 as well, will be used in the prooí without explicit mention; the reasoning is
(:1' 0,
-
basically s i mi l ar to that in [4] . We need the notation corresponding to algorithm (2.7),
when ap p lied to the reduced system; let u s set
68
and define
FOT
y k ,O
X- k ,o
k
=
: =
: =
O, 1 , . . .
yk
X- k
F OT i = 1 , . . . , p
y k ,i : = y k ,i - l F ' (y k ) - l F y k , i - l
X- k, i : = X- k ,i - l F '(y k ) - l F x k ,i -l
: = y k,p
_
_
y k+ l
X- k+ 1
THEOREM 3 . 3 .
: =
X- k ,p
The following inequalities hold for k
x kl
----r;I
x J·
,i
<
<
X- k , i
.1
k
Xg( ,i )
<
<
y*
*
Yl
=
0 , 1 , . . . and 1 :::;
_ k,i
<
Yj
<
g(
y k,i )
<
i
:::;
p :
p
-
l.
----r;I
<
Yj
k,i
YI
P ROO F : Suppose that the condusions hold for a ('ertain k and O :::;
For 2 :::; j :::; n , we have
i
:::;
n
n
1=2
=
and
thus
!
n
y}<.i - L ( F / ) j) ( g (y k ), y k ) * fl( g('iJ k. i ), y k .i )
1= 1
!
+
y , i l :::; y , i
n
_
L ( F / r ;:J ( y k )
1= 1
*
i
f¡ ( g ( y k , i ) , y k , )
( 3. 3 )
&9
The inductive hypotheses, in cluding
Fy".i
_
the order convexity of F ,
F( g ( y " . i ) , y ".i) :::; F ' ( y " .i )(y".i
:::; F ' (y " )( y k. i
(g(y " ,t y " .i )
Thus,
for 2 :::; j :::;
_
F '( y k ) - 1 F(g(y "·i). y ". i ) :::; y k . i
n,
"
.
�
.
i
n
_
" ( F ' ) -:- I (
�
J.I Y
1= 1
"
,
,
F '(y " ) - 1 Fy",i
(3.4) meROS that
YJ
_
_
imply that
(g(y k,i), y . i ))
(y i
(g " ) y " i ) )
_
_
which yields
also
) * f1 ( 9 ( -Y " . i ) , -Y " i )
,
<
-
- � . i+ t
Y1
=
,
y",i+ l
( 3. 4 )
'
which combined with ( 3. 3 ) yields
¡¡ " .i+ l :::;
l
Since we haye y. :::; y " . i+ , it follows that
On the
ot
yk
.i+
1
her hand. it is now clear that
Ir i+ l
and thus
=
n
Iri ¿ (F ' )¡; (g(¡¡k ). ¡¡") fl(g ( Yk.i), X ", i )
-
·i
Ir i + 1 2:: If ·
We also have
F( g(I k.i ) . z: Ir.i )
_
*
1=1
-
n
2:( F ' ) j} ( y k ) * f¡ ( g ( y k , i ) , X k , i )
and we finally obtain
( 3. 5 )
1= 1
FX "·i :::; F ' ( g( ;¡ k ,i ) , z: " , i )« g( X " , i ) , X k , i )
k
ki
:::; F ' ( yK ) « g ( k i ) X i ) X , )
whence
Thus, by taking account
,
of ( 3.5), it follows that
X ,
,
,
_
_
,
X "·i )
80
R EMARK 3 . 4 . If instead of applying accurate elimi n ation, !ln unknown is eliminated by
means of a differently numbered equation, Theorem 3.3 is no longer true (see Remark
3 . 1 0 in [4] and the counterexample given there ) j nevertheless an extension of Theorem
3. 1 1 in [4] ' comparing the generated sequences for such elimination, can be proved for the
simplified N-F iterations in case the elimination satisfies the hypotheses in that theorem.
Theorem 3 . 3 provides a useful too1 for the reduction of non1inear systems that satisfy its
hypotheses, in either of the two basic situations mentioned following ( 2 . 7 ) j such reduction
yie1ds a better convergencej however, its application should take account of the computa­
tional cost invo1ved, as in the linear case ( See [3] ) . Thus , theorem 3 . 3 implies that if the
computationa1 cost of the elimination is neg1igib1e and the computationa1 cost per itera­
tion is not increased, then a net gain is to be expected with the simp1ified N-F iterations
app1ied to ( 3 . 1 ) . These considerations will be illustrated in the following seetion.
4. A NUMERICAL EXAMPLE
The examp1e treated in this seetion is t aken from [4] . Thus, verification of the hypotheses
in Theorem 2 . 1 is omitted here. The calculations have been carried out with the doub1e
precision of Fortran 5.0 on a p e . Let us define F : IRn ---+ IRn by
!t
:=
3
h 2 + YI
2YI - Y2
f. ' - 2Y i - Yi - I - Y i + 1
, .h2
f
?'y 3
n '.
�
n
Y
-'
h2
n-I
+
. with
3
Yi
•
h : = _1_1
+
( 4. 1 )
1/
If Y n is eliminated by means of f.. . we get the reduced system
3
h 2 + YI
J . = 2Yi - Yi-l - Yi+ l
+ Yi
,
h2
2Y,, _ 1 Yn- 2
+ Y,,-I
.f n - l
h2
where g : = ( Yn-I ) �
- = 2Yl - Y2
fl
-
Consider n = 1 0 .
have been
-
:1
- 9
-
-
•
( 4. 2 )
2
:= (O
•
.
.
.
•
O . 0. 14, 0.41 ) all d yU : = ( 1 . . . . . 1 ) . The
I I Fyk•i 112
<.f
:=
10- 1 3
I I F.r k.i I 1 2
<f
and
st o
pp i n�
('ritt'ria
( 4. 3 )
( 4 ..1 )
iteri a have cnsured mRchine cOllvergence, i . t' . t-h(' ite�at t's rel ll ai ll const ant t hert"­
artero Tab1e 4 . 1 des c ri b es the necessary work to attaill cOllvergellCt' for t he i t erat iolls (2. ;).
These
cr
;rO
:1
61
Only relevant steplengths have been taken into account in the tableo The leftmost column
indicates the steplength p; the foHowing two columns indicate, respectively, the values kp+ i
for which (4.3) and (4.4) are satisfied for the N-F iterations applied t o (4. 1 ) , whereas the
last two columns describe the corresponding values when the iterations are applied to (4.2).
The monotone behaviour in each column illustrates Theorem 2.5. For the actual iterates
with p = 1 see Tables 4.1 and 4.2 in [5] .
TABLE 4 . 1
p
1
2
3
4
5
6
7
8
9
10
11
12
1,3
14
15
20
23
24
29
30
44
45
82
83
II Fy k ,i 11 2 < €
6
8
10
10
12
13
15
16
16
17
17
17
18
19
20
23
26
27
31
32
45
46
83
83
II Fx k, i 1 1 2
8
9
10
11
12
13
15
17
17
18
18
18
19
19
20
24
26
27
32
32
46
46
83
83
<€
pk + i
II F yk ,i 11 2 < €
5
6
7
8
8
9
10
10
11
12
12
13
14
15
16
21
24
24
II F xk,i 1 1 2
< €
6
6
8
8
8
9
10
10
11
12
13
13
14
15
16
21
24
24
According to this table, the reduced system behaves numerically better than the
original one for every steplength. This example also shows that , if simplified iterations are
to be computed throughout with the initial Jacobian matrix, i.e. with no updating at aH,
then the amount of computational work for the reduced system may be much smaller than
the corresponding one for the original system.
62
REF E R E N C E S
[1]
[2]
[ 3]
[4]
[5]
[6]
[7]
ALEFELD;G . , Zur Konvergenz des Verfahre.ns der Tangiere.nden Hyperbeln und des
Tschebyscheff- Verfahrens bei konvexen Abbildungen, Computing 1 1 , 379-390( 1973).
BALUEV,A . , On the method of Chaplygin (Russian), Dokl. Akad. Nauk.SSSR 83,
781-784( 1 952).
MILASZEWIC Z , J . P. Improving Jacobi and Gauss- Seidel iterations , Linear Algebra
and its Applications 93, 161-170(1987).
MILASZEWICZ , J . P. and ABDEL MASIH ,S. On elimination and fixed point itera­
tions, Computers and Mathematics with Applications 25, 43-53( 1993 ).
MILASZEWICZ,J.P. and ABDEL MASIH , S . Errata to " On elimination and fixed
point iterations " , Computers and Mathematics with Applications 27, 1 13- 1 15( 1994 ) .
ORTEG A , J . M . and RHEINBOLDT,W. , Iterative .wlution of nonlinear equations
in s everal variables, Academic Press 1970.
WOLFE,M. A . On the convergen ce of some me.thods for determining zero.� of order
convex operators, Computing 26, 45-56( 1981 ) .
Departamento d e Matemática
Facultad de Ciencias Exactas y Naturales
Ciudad Universitaria
1428 Buenos Aires, Argent.ina
Recibido en junio d e 1 999.
Versión modificada en junio de 1 994 .