SIAM J. NUMER. ANAL.
Vol. 16, No. 1, February 1979
1979 Society for Industrial and Applied Mathematics
0036-1429/79/1601-0001 $01.00/0
AFFINE INVARIANT CONVERGENCE THEOREMS FOR NEWTON’S
METHOD AND EXTENSIONS TO RELATED METHODS*
P. DEUFLHARD
AND
G. HEINDL"
Abstract. For a given starting point, the sequence of Newton iterates is well known to be invariant
under affine transformation of the operator equation to be solved. This property, however, is not sufficiently
reflected in most convergence theorems that are presently in common use. For this reason, certain affine
invariant convergence theorems are given in this paper. The new theorems can be understood as refined
versions of the Newton-Mysovskii theorem, the Newton-Kantorovitch theorem (including optimal error
bounds) and a convergence theorem for approximate Newton processes. In addition, the concept of affine
invariance associated with Newton’s method is extended to S-invariance associated with related iterative
methods where S is some set of linear transformations not changing the iterates. As an application of these
considerations, a new convergence theorem for a class of generalized Gauss-Newton methods is given. This
theorem is S-invariant with an S containing all unitary transformations. Unlike previous competing
theorems, the new one reduces to the Newton-Mysovskii theorem when the Gauss-Newton method
reduces to Newton’s method.
1. Introduction. In 1939, Kantorovitch [11] presented a preliminary convergence theorem for Newton’s method which he substantially improved in 1948/49
(see [12], [13]). Since then, a large number of convergence results concerning this
method and related techniques have been published: one may, for examples, refer to
the work of Mysovskii [16], [17], Bartle [1], Ortega [18], Rheinboldt [23], Dennis [4],
[5], [6], [7], or, as more recent publications, to Rall [22] and Gragg/Tapia [10].
In order to explain the purpose of the present paper, let
F(x) 0
(1.1)
denote a nonlinear operator equation to be solved where F is a Fr6chet-differentiable
mapping from an open convex subset D of a real Banach space X to a real Banach
space Y. For the time being, assume that F’(x) is invertible for every x e D and that,
for a given starting point x D, the Newton iteration
(1.2)
x
k/l
:=
xk-F’(xk)-lF(x),
k =0, 1,.
,
is defined. One of the well-known properties of Newton’s method is that the iterates
x k 0, 1,-.., are invariant under any affine transformation
k’
FG := AF
(1.3)
where A denotes any operator contained in the space B(Y, Z) of all bounded linear
bijective mappings from Y to any Banach space Z. This property is easily verified,
since
G’(x)-IG(x)=F’(x)-IA-AF(x)=F’(x)-IF(x) for all x e D.
(1.4)
Hence, any transformation of the type (1.3) will not affect the convergence or
divergence of the Newton sequence {x}. As a consequence, the question of convergence of Newton’s method should be discussed in theoretical terms that are invariant
under affine transformation. Unfortunately, most of the convergence theorems that are
presently in common use do not satisfy this requirement. Although there exist one or
two affine invariant theoremsmsuch as Theorem 6. (1.XVIII) in the book of
Kantorovitch/Akilov [ 14], or certain results due to Dennis [6], [7]--the importance of
* Received by the editors July 12, 1977.
"
Institut fiir Mathematik, Technische Universitiit M/inchen, M/inchen, Federal Republic of
Germany.
2
P. DEUFLHARD AND G. HEINDL
these theorems seems to have been overlooked even by these authors, since in later
work they no longer pay attention to the invariance requirement. In particular, to the
authors’ knowledge, an affine-invariant version of the so-called Newton-Mysovskii
theorem (cf. [16], [17], or Ortega/Rheinboldt [19, p. 412]) seems never to have been
considered.
In order to illustrate the invariance requirement, the latter theorem will now be
applied to a simple example. Let X Y I 2,
(, z) rl- 1 < < 3,
D := {x
]
for allxeD, and
\
X
Let
:-2
F(x):=(
(1.5)
0
1, 2}
0.125)
:---
0.125
have the sup-norm. Then, with the notation
cv :=
IIF’(x)-lF(x)ll,
/3v := sup IlF’(x)-lll,
xD
yv := sup
IlF’(y)- f’(x)ll/lly xll,
x, yeD
xy
hF :’- OIFFTF,
hF2i--1 <
pF:’- aF
j=0
Ol F
if
1-hv’
hF < 1
one may straightforwardly verify that
(1.6)
av 0.127,
hv 0.381,
/3v 3,
"/v
2,
pv <0.205.
Thus the Newton-Mysovskii theorem guarantees convergence of the Newton iterates,
since
hF< 1
PF< 1.125.
and
However, if one applies the theorem to the mapping
(1 5’)
G:
0
then one obtains (in a similar notation):
(1.6’)
a av, /3 8.5,
1/2
,
F,
2,
h
1.0795.
Thus, since
ha> 1,
convergence of Newton’s method is not assured by the Newton-Mysovskii theorem
when applied to G. The same situation occurs when the so-called Newton-Kantorovitch theorem--as presented in more or less generality in Ortega [18], Rheinboldt
[23], Ortega/Rheinboldt [19], Krasnosel’skii et al. [15], or Gragg/Tapia [10]--is
applied to our simple example" this theorem, too, assures convergence when applied
to F but not when applied to Gmeven though the Newton sequence generated by G
AFFINE INVARIANT CONVERGENCE THEOREMS
3
is identical to the one generated by F! Obviously, this occurrence is theoretically
unsatisfactory.
That is why, in 2 of this paper, certain affine-invariant versions of both the
Newton-Mysovskii theorem and the Newton-Kantorovitch theorem are given including, in the latter theorem, affine invariant error bounds which are optimal in the sense
introduced by Gragg/Tapia [10]. Moreover, convergence results for approximate
Newton processes (also called Newton-like processes by Dennis [5]) are presented. In
this class of methods, the Fr6chet derivative F is replaced by an approximation MF. In
view of the property
(AF)’ AF’,
the approximation mapping MF is also required to satisfy
MAF AMF
(1.7)
for our convergence results to be affine invariant.
In some iterative methods, the generated sequence is known to be invariant only
under transformations contained in some subset S of all affine transformations with
domain Y. In these cases, it is reasonable to require only S-invariance for the
associated convergence theorems. This means that both the assumptions and the
statements of the theorems should remain unaltered, when F is replaced by AF for
any A S. As an example, consider the class of generalized Gauss-Newton methods
which is treated in 3. This class is defined by the iteration
(1.8a)
X
k+l
’-- Xk--I’F(xk)F(x k)
where for the mapping FF (from D to the space L(Y, X) of all bounded linear
mappings from Y to X) the following projection property is assumed to hold:
(1.8b)
FF(X)F’(x)FF(x)F(x) FF(X)F(x) for all x D.
Obviously, S can be taken as the set of all bounded linear bijective mappings A which
satisfy the relation
FAF(X) FF(X)A -1 for all x D.
(1.9)
In particular, if X, Y are Hilbert spaces and FF(X)=F’(x)*, the Penrose pseudoinverse (cf. [20], or [3, Chap. 8]) of the Fr6chet derivative, then this S contains at least
the class of unitary operators from Y onto Y. With these considerations as a basis, a
new convergence theorem for the type of Gauss-Newton method as defined by (1.8) is
presented. Unlike competing theorems (see e.g. Ben-Israel [2], Rheinboldt [23],
Pereyra [21]), the new theorem reduces essentially to the convergence statements of
the Newton-Mysovskii theorem (as given in 2), if the method (1.8) reduces to
Newton’s method.
The authors hope that the requirement of S-invariance as introduced here will
be used for a wider class of iterative methods and convergence results. In addition,
apart from any purely theoretical interest, the results presented here form an
immediate basis for recently developed numerical techniques (see [8], [9]) and
for studies of algorithms in terms of complexity theory--see e.g. the results due
to Traub/Wo.niakowsky [24] which satisfy the affine invariance requirement as
proposed here.
2. Convergence results for Newton’s method and approximate Newton processes.
Throughout this paper, the following notation is used: X, Y, Z denote rea! Banach
spaces, and D is an open convex subset of X. (x p) denotes the closed ball with
,
4
,
P. DEUFLHARD AND (3. HEINDL
center x 0 and radius p -> 0, while S(x p) is its open equivalent. B( Y, Z) is the set of all
bounded linear bijections from Y to Z, L(Y, X) is the set of all bounded linear
mappings from Y to X, I is the identity operator.
First, an affine invariant version of the so-called Newton-Mysovskii theorem is
given.
THEOREM 1. Let F: D-* Y be Frchet-differentiable with F’(x) invertible for all
x D. Assume that one can find a starting point x D and constants a, to >- 0 such that
a)
[[F’(x)-’F(x)][a,
b) IlF’(y)-a(F’(x + t(y-x))-F’(x))l[<-totl[y-xllforallx, y 6D
and for all [0, 1],
(2.1) c) h := 1/2ato < 1,
,
d) g(x p)c D with p :=
a
2 h2-l-h"
i=o
Then
(I) the sequence {x ’} given, by
x k+l := X k -F’(x k)- 1F(xk),
remains in g(x p) and converges to some
,
k=O, 1,-..
,
x* e g(x O) with F(x*) O,
(lI) the convergence rate can be estimated by
(2.2)
a)
IIx/l-xllllx-x-l[=,
k=l,2,...,
b)
IIx-x*llllx-x-ll=,
k=l,2,..-
with
e, :=
to
Z (h2k) 2’-1< 2(1
j=o
h 2k).
Proof. Set G(x) := x-F’(x)-lF(x) for all x eD. Then,whenever x, G(x)D, one
may conclude from assumption (2.1b)that
-
lla(a(x))- a(x)ll llF’(a(x))-IF(a(x))ll
IIF’(G(x)) (F(G(x)) F(x) F’(x)(G(x) x))ll
[Ia(x)-xl[2.
A detailed proof of the last inequality is given (for a more general case) in the proof of
Theorem 4 in 3. With this result and [23, Theorem 2.3 and Lemma 2.4], one readily
concludes that the sequence {x } remains in (x p) and converges to some x*
(x p)compare also the proof of 12.4.6 in the book of Ortega/Rheinboldt [19].
The fact that F(x*)= 0 can be proved by virtue of the inequalities
,
,
Ilf’(x*)-lV’(x )(x k+l_ xk)ll
Ilv’(x*)-’ (’(x*)- V’(x))(x +’ x )11 + [Ix
=(llx*-;ll+a)ll+’-xll for g=o,
+’
,...,
x
ll
AFFINE INVARIANT CONVERGENCE THEOREMS
5
which imply that
IIf’(x*)-’ f(x*)ll= klim
lim
koo
IIF’(x*)-lF’(xk)(xk+l--xk)l[=O.
The remaining statements can be shown along the lines of standard proofs of the
Newton-Mysovskii theorem (cf. [19, p. 412]). 171
Obviously, Theorem 1 is affine invariant as required in 1. Moreover, an
important property of this theorem is that it applies to F whenever the usual NewtonMysovskii theorem applies to AF for any A B(Y, Z). In order to see this property,
recall the assumptions of the usual Newton-Mysovskii theorem when applied to AF
(notation as above). Assume that there exist constants a(A), 13(A), /(A) >-_ 0 such that,
for a given starting point x the following conditions hold
,
a)
b)
I[(AF)’(x)-AF(x)II<=(A),
II(AF)’(x)-’II<--(A) for all x
II(AF)’(y)-(AF)’(x)II <- r(A)lly-xll for all x, y D,
(2.’)
c) h(A) := 1/2a(A)(A)7(A)< 1,
,
d) g(x p(A)) c D with p(A) := a(A)
Y h(A) z’-l.
]=0
With these assumptions, one obtains
IlF’(x)- f(x)ll
<- - (A),
IIF’(y)-l(F’(x + t(y x))-]I(AF)’(y)-I((AF)’(x + t(y x)) (AF)’(x))II
--</I(AF)’(y)-I// I[(AF)’(x + t(y x))-(aF)’(x)ll
<=(A)/(A). tlly-xll for all x, yeD and for all te[0, 1].
Then, with
te
:= a(A),
o
:= fl(A)T(A),
the assumptions of Theorem 1 are verified. If the estimates (2.1’b) hold for a selected
A, then certain/3(A), 7(A) exist for all A e B(Y, Z). Thus there exists an o for which
(2.1b) holds and which satisfies
w _--< fl(A)T(A) for all A e B(Y, Z)
(2.3)
no matter how sophisticated/3(A) and /(A) may have been chosen. As an illustration of
the above extremal property, Theorem 1 will be applied to the simple example introduced in (1.5), or (1.5’), respectively. Note that the constants aF,/3F, ,v, so,/30, To were
best possible, estimates. As best possible constants c, o) and h for Theorem 1 one
obtains
(2.4)
c
0.127,
o)
1, h
0.0635,
which is a considerable improvement of both (1.6’) and (1.6).
As a second theorem, an affine invariant version of the well-known NewtonKantorovitch theorem is given.
6
P. DEUFLHARD AND G. HEINDL
THEOREM 2. Let F: D -+ Y be Frdchet-differentiable. Assume that one can find a
starting point x o D with F’(x o) invertible and constants to > 0 such that
,
a) IlF’(x) -’F(x)ll c,
b) IIF’(x)-l(F’(y)-F’(x))ll<-olly-xll for all x, y D,
(2.5) c) h := otto -<1/2,
d) (x p_) D with p_ :=
1-41-2h
,
Then
(I) F’(x) is invertible for every x S(x p_),
(II) the sequence {xk} defined by
x k+l := x k F’(x k -1 F(x),
k=0,1,-..,
remains in S(x p_) and converges to a solution point x* of F(x) 0,
(III) the solution is unique in (x p_) U (D fq S(x p+)), where
,
,
,
1 +/1-2h
(IV) with 6) := p_/p+, one obtains the error estimates (k
2x/l- 2h 0 2
xll,
IIx x*ll <1 _(R)2 IIx’h
IIx -x*ll<-2-+’llxa-xll, Ch =1/2,
1, 2,. .)
if h < 1/2,
and
1 + 41 + 46)k/(1 + 6) )
Proo[. The theorem is an immediate consequence of combining the affine invariant convergence Theorem 6.(1.XVIII) in the book of Kantorovitch/Akilov [14] and
the optimal error bounds due to Gragg/Tapia [10] which were, however, given in
terms that are not affine invariant. Nevertheless, if one applies the results due to [10]
to the mapping
(2.6)
F’(x)-IF,
then the affine invariant statements (IV) may be obtained.
An extremal property similar to (2.3) can also be shown to hold for Theorem 2
when compared with certain versions of the Newton-Kantorovitch theorem that are in
common use, but are not affine invariant. In addition, one observes that Theorem 2 is
obtained by the "optimal scaling" (2.6).
Remark. This observation justifies the application of natural scaling (as introduced in [8]) which has appeared to be extremely useful in the numerical solution of
real life nonlinear problems (see also [9]).
In many applications, the Fr6chet derivative F’, which is needed for Newton’s
method, is tedious (if not impossible) to obtain. For these cases, approximate Newton
processes, also called Newton-like methods (cf. Dennis [5]) are constructed with
My: D L(X, Y) denoting an approximation of F’. As examples of approximate
Newton processes take the so-called simplified Newton method, where, for a given
7
AFFINE INVARIANT CONVERGENCE THEOREMS
D,
starting point x
Mr(x) F’(x ) for all x D,
or the class of Newton methods where F’ is approximated by a finite difference
operator. In both of these examples, the following transformation property is valid
MAF AMF
(2.7)
for all A B(Y, Z) and any Z. Detailed theoretical investigations of approximate
Newton processes have been made e.g. by Rheinboldt [23] or Dennis [5]. The
subsequent convergence theorem is a refined version of Theorem 4.3 in [23]. The new
theorem is S-invariant, if (2.7) holds for all A6S.
THEOREM 3. Let F: D -. Y be Frdchet-differentiable and MF an approximation of
F’. Assume that one can find a starting point
D with Mv(x ) invertible and
constants a, to > O, 60, 61, tx >--0 such that
x
a) IlMv(x)-F(x)ll<=a,
b) IIMv(x)-’(F’(y)-F’(x))ll<=oolly-xll for all x, y D,
c) [IMv(x)-a(F’(x)-Mv(x))[l<-_6o+6111x-xll and
(2.8)
IIM(x)-M(x)-Zll<-_llx-xll for allx D,
d) 60 < 1 and h :=
with r := max (to, Ix + 61)
(1 (o) 2 2
e) g(x ,p) cD withp:=
1-/1-2h
a
h
1-6o"
Then
,
(I) MF(X) is invertible for every x S(x p),
(II) the sequence {x k} defined by
,
x k+l := x k --MF(X k )-IF(x k ),
remains in S(x p) and converges to a solution x*
(III) with the notation
h :=
hto
p+ :=
o-’
one obtains
x*
and
x*
is unique in
,
k--0, 1,...
of F(x)= O,
1 + /1 2/
/
1-6o’
,
g(x 0-)
,
g(x O) U (D fq S(x 19+)).
Pro@ Replace Y in Theorem 4.3 of [23] by X, Do by D, F by Mv(x)-IF, A by
Mv(x)-Mv, and/3 by 1; then the statements (I) and (II) can be readily obtained. At
the same time, one concludes that there exists at most one solution of F(x)= 0 in
D 71S(x, 19+). Next, Corollary 3.3 and Theorem 2.6 in [23] together with the relation
19_-<19 imply that there exists exactly one solution x* of F(x) 0 in the ball g(x 19_).
Finally, the rest of statement (III) is obtained by observing that 19 -< 19+ and 19_ 19, if
,
p =p+.
[-1
8
P. DEUFLHARD AND G. HEINDL
Considerations similar to those following Theorem 1 yield that an extremal
property also holds for Theorem 3: one observes that Theorem 3 applies whenever
there is an AB(Y, Z) such that the assumptions of the associated (not affine
invariant) theorem--see e.g. [23]mhold for AF.
,
3. A convergence theorem for a class of generalized Gauss-Newton methods. In
this section, the following class of iterative methods is considered" for given x the
sequence {x k} is defined by
x k+l :=
(3.1)
xk--FF(xk)F(xk),
k=0, 1,’’"
where F" D --> Y denotes a Fr6chet-differentiable mapping and
satisfies the projection property
FI=" D --> L(Y, X)
FF(X)F’(x)FF(x)F(x) FF(X)F(x) for all x e D.
(3.2)
Equation (3.2) holds, for instance, when X, Y are Hilbert spaces, F’(x) has a closed
range F’(x)(X) for every x e D, and FF(X) is the Penrose pseudo-inverse F’(x)*--for
reference compare the book of Ben-lsrael/Greville [3, Chap. 8]. The Penrose pseudoinverse is known to be uniquely defined by a set of 4 axioms, one of which implies
FF(X)= F’(x)* for all x D, then (3.1) is well known as the
(3.2). If X [m, y=
method. Note, however, that (3.2) also covers generalized
Gauss-Newton
(ordinary)
inverses different from the Penrose pseudo-inverse. As an example, take the generalized inverse F’(x)- as defined in [8] and employed in multiple shooting techniques.
The subsequent convergence theorem is closely related to Theorem 1 of this paper.
THEOREM 4. Let F and F F be defined as above. Assume that one can find a
starting point x o D, a mapping K: D ff+, and constants a, oo, >- 0 such that
,,,
-
(3.3)
a) IIr(x)F(x)ll--<
b) Ilr(y)(f’(x + t(y x))- f’(x))ll _-< wtlly- xl[ ]’or all x, y e D with y x e
FF(X)(Y) and[or all te[0, 1],
all x, y e D (yielding (3.2)
c) IIFv(y)(I f’(x)r(x))f(x)ll <- (x)lly xll
[or y x),
d) <(x)-<ff<l [orallxeD, h :=1/2ao<l-ff,
or
,
e) g(x p) C D with p := 1-y-h"
,
Then the iterates (3.1) are well defined, remain in
g(x,o) and converge
to some
x* g(x O) with
FF(X*)F(x*) O.
(3.4)
Proof. Set G(x) := x FF(X)F(x) for all x e D. Then, for x, G(x) D, the following estimates hold:
IIG(G(x))- G(x)ll IIF(G(x))F(G(x))II
--< IIF(G(x))(F(G(x)) F(x) F’(x)( G(x) x))ll
+IIF (G(x))(I F’(x)F(x))F(x)[I
<--IIG(x)- xll 2 + ,(x)l[G(x)- xll.
-2
AFFINE INVARIANT CONVERGENCE THEOREMS
9
The last inequality may be shown by considering the function q C1[0, 1] that is
defined by
(t) := l[Fv(G(x))(F(x + t(G(x)-x))- tF’(x)(G(x)-x))]
where denotes a linear functional on X with the properties
I[F(G(x))(F(G(x))- F(x)- F’(x)(G(x)- x))-I
--I/r(G(x))(F(G(x)) F(x) F’(x)(G(x) x))l/.
(The existence of such a functional is a well-known consequence of the Hahn-Banach
theorem.) Then one obtains from (3.3) b)
[]I (G(x))(F(G(x))- F(x)- F’(x)( G(x) ))11
()-(0)
Jo
Jo ’(t)
dt
l[Fv(G(x))(F’(x + t(G(x)-x))-F’(x))(G(x)-x)] dt
llG(x)- xlJ
,
Jo
llG(x)- xll2.
dt
Following the lines of the proof of Theorem 4.1 in [23] one can show that the
sequence {x } remains in (x p) and converges to some x*. In order to obtain (3.4),
one proceeds using the estimates
+ IIF(x*)’(x )11
IIx +’- xll + llx
for all k =0, 1,.... Then, with the continuity of F(x*)F and the boundedness of
F(x*)F’, the result (3.4) isverified, which completes the proof.. U
Obviously, S-invariance of the above theorem is assured when S is the set of all A
satisfying
(3.5)
FA(X) F(x)A
-
for all x D.
Remark 1. In the special, but most important case, when X, Y are Hilbert spaces
and F(x) F’(x)* for all x D, the above set S contains at least the set of all unim
operators mapping Y onto Y. With A* denoting the adjoint operator, this statement
follows from
(A)’(x)* (AF’(x))*
((AF’(x))*(AF’(x)))t(AF’(x)) *
(F’(x)*A*AF’(x))*F’(x)*A*
(F’(x)*F’(x))*F’(x)*A-’=
for all x D and any unitary A.
10
P. DEUFLHARD AND G. HEINDL
Remark 2. If F’(x) is invertible for all x D and FF(X)--F’(x) -1, then (3.1)
reduces to Newton’s method. In this case, the function r and the constant ff in
Theorem 4 can be chosen to be zero: thus Theorem 4 essentially reduces to the
convergence statements of Theorem 1, the affine invariant version of the NewtonMysovskii theorem. This property seems to stand out in comparison with corresponding theorems due to Ben-Israel [2], Rheinboldt (cf. [23, Thm. 4.6]), or Pereyra [21].
Acknowledgment. The authors wish to thank Dr. H. Evans for his careful reading of the manuscript.
REFERENCES
[1] R. G. BARTLE, Newton’s method in Banach spaces, Proc. Amer. Math. Soc., 6 (1955), pp. 827-831.
[2] A. BEN-ISRAEL, A Newton-Raphson method for the solution of systems of equations, J. Math. Anal.
Appl., 15 (19"66), pp. 243-252.
[3] A. BEN-ISRAEL AND T. N. E. GREVILLE, Generalized Inverses: Theory and Applications, John
Wiley, New York, 1974.
[4] J. E. DENNIS, JR., On Newton’s method and nonlinear simultaneous displacements, this Journal, 4
(1967), pp. 103-108.
[5] , On Newton-like methods, Numer. Math., 11 (1968), pp. 324-330.
[6] , A stationary Newton method for nonlinear functional equations, this Journal, 4 (1967), pp.
222-232.
[7] , On the Kantorovitch hypotheses for Newton’s method, this Journal, 6 (1969), pp. 493-507.
[8] P. DEUFLHARD, A modified Newton method for the solution of ill-conditioned systems of nonlinear
equations with application to multiple shooting, Numer. Math., 22 (1974), pp. 289-315.
[9]
, A stepsize control for continuation methods with special application to multiple shooting tech-
niques, Tech. Rep. TUM-MATH-7627, Technische Universitit Miinchen, Institut fiir Mathematik, Miinchen, Germany, Dec. 76.
10] W. B. GRAGG AND R. A. TAPIA, Optimal error bounds for the Newton-Kantorovitch theorem, this
Journal, 11 (1974), pp. 10-13.
11] L. KANTOROVITCH, The method of successive approximations for functional equations, Acta Math., 71
(1939), pp. 63-97.
[12] , On Newton’s method for functional equations, Dokl. Akad. Nauk SSSR, 59 (1948), pp.
1237-1240. (In Russian.)
[13] , On Newton’s method, Trudy Mat. Inst. Steklov, 28 (1949), pp. 104-144. (In Russian.)
[14] L. KANTOROVITCH AND G. AKILOV, Functional Analysis in Normed Spaces, Fizmatgiz, Moscow,
1959; German translation, Akademie-Verlag, Berlin, 1964.
ET AL., Approximate Solution of Operator Equations, Walters-Noordhott,
Groningen, the Netherlands.
I. MYSOVSKII, On the convergence of L. V. Kantorovitch’s method of solution of functional equations
and its applications, Dokl. Akad. Nauk SSSR, 70 (1950), pp. 565-568. (In Russian.)
, On convergence of Newton’s method, Trudy Mat. Inst. Steklov, 28 (1949), pp. 145-147. (In
Russian.)
J. M. ORTEGA, The Newton-Kantorovitch theorem, Amer. Math. Monthly, 75 (1968), pp. 658-660.
J. M. ORTEGA AND W. C. RHEINBOLDT, Iterative Solution o]’ Nonlinear Equations in Several
Variables, Academic Press, New York, 1970.
R. PENROSE, A generalized inverse for matrices, Proc. Cambridge Philos. Soc.,.51 (1955), pp.
[15] M. A. KRASNOSEL’SKII
16]
[17]
[18]
[19]
[20]
406-413.
[21 V. PEREYRA, Iterative methods for solving nonlinear least squares problems, this Journal, 4 (1967), pp.
27-36.
[22] L. B. RALL, A note on the convergence o]’ Newton’s method, this Journal, 11 (1974), pp. 34-36.
[23] W. C. RHEINBOLDT, A unified convergence theory ]’or a class o.f iterative processes, this Journal, 5
(1968), pp. 42-63.
[24] J. F. TRAUB AND H. WOP.NIAKOWSKI, Convergence and complexity of Newton iteration ]’or operator
equations, Tech. Rep., Dept. Computer Sci., Carnegie-Mellon Univ., Pittsburgh, PA, March 1977.