SIAM J. NUMER. ANAL.
Vol. 14, No. 6, December 1977
SAFE STARTING REGIONS FOR ITERATIVE METHODS*
R. E. MOORE
AND
S. T. JONES"
Abstract. A search procedure based on interval computation is given for finding safe starting
regions in n dimensions for iterative methods for solving systems of nonlinear equations. The
procedure can search an arbitrary n-dimensional rectangle for a safe starting region for a quadratically
convergent iterative method. The procedure is more powerful than continuation methods.
1. Introduction. There are many good methods for iterative solution of
systems of nonlinear equations (see, e.g., [11], [12]). For any such method,
however, the problem remains of finding a safe starting point" an initial approximation from which the iterates will converge to a solution. In this paper, we
present a procedure for searching an arbitrary n-dimensional rectangle for a
region of safe starting points. A suitable iterative method possesses computationally verifiable sufficient conditions for existence and convergence.
We will present the procedure with a particular iterative method in mind, an
interval version of Newton’s method due to R. Krawczyk [5] (see also, [10]). It
will become clear how the procedure could be used, mutatis mutandis, with other
iterative methods.
By recasting the problem of solving a system in the form of a fixed point
problem, we can make use of sufficient conditions for the existence of a solution
based on fixed point theorems such as those of Brouwer or Schauder. The interval
binary search procedure to be described io this paper can start with an arbitrary
n-dimensional rectangle and an arbitrary algorithm for finding zeros of a system
(as long as there are computationally verifiable sufficient conditions for existence
and convergence in an n-dimensional rectangle). As a stopping criterion for the
search procedure we can use" satisfaction of the criteria for existence and
convergence appropriate to the algorithm to which the search procedure is to be
applied. When supplied with a suitable algorithm and a specific system of
equations, the search procedure will be ready to search an arbitrary rectangle B. It
will terminate in a finite number of steps with one of the following three results: a
sub-region of B which is a safe starting region for the algorithm; discovery that
there are no solutions in B; or (when further necessary bisections cannot be
carried out because of the limited resolution of the machine arithmetic being used)
a list of small sub-regions of B which would require higher precision machine
arithmetic to yield a result of one of the first two types.
In addition to finding safe starting regions for iterative methods, the search
procedure could be used to find safe starting regions for algorithms based on
simplicial subdivisions such as those of Scarf [13], Stenger [14], and Todd [15]
providing they can begin with a safe starting region in the form of an ndimensional rectangle. The conditions to be tested on a region for such algorithms
may be based on Miranda’s theorem (Miranda [7]) or on the Brouwer fixed point
theorem directly. Using interval computation, we can test whether a vector
* Received by the editors August 3, 1976, and in revised form February 3, 1977.
? Computer Sciences Department, University of Wisconsin--Madison, Madison, Wisconsin
53706. This work was supported by the Graduate School of the University of Wisconsin.
1051
1052
R.E. MOORE AND S. T. JONES
mapping takes an n-dimensional rectangle into itself. Such a test for existence is,
in fact, used in connection with the iterative algorithm of 3.
In 2 we present the search procedure in general outline as an interval
bisection procedure in n dimensions.
In 3 a specific iterative algorithm is studied to derive certain properties
which can be exploited by the search procedure--in particular, computationally
verifiable sufficient conditions for existence and convergence. In applying the
search procedure to a particular algorithm bisection rules can be devised which
make use of information arising during the computations of the algorithm. This is
also discussed in 3.
In 4 we show some computational results and comparisons with continuation methods (for getting into safe starting regions). It will be seen that the interval
search procedure works where some continuation methods fail.
2. Bisection in n dimensions. We seek a solution to a nonlinear system
(x,x,..., Xn)= 0,
(2.1)
fn(Xl, XZ,
,Xn)=O
for real valued fl, f2,""", f, which are continuous in an open set D in
vector notation the system (2.1) can be written as f(x)= O.
Suppose we could compute the exact range of values,
R". In
(X), [i (X) max/
/ (X) [ (X), [i (X)], / (X) min/
(X)
xX
xX
of each/ when x lies in an n-dimensional rectangle
X:{x" x_i <:xi <.i,
1,2,... n}D.
Then we could use the following simple cyclic n-dimensional bisection procedure.
Suppose X() is any n-dimensional rectangle in D for which/(X()) includes
Then X may contain a solution to the
the number zero for all
1, 2,.
system (2.1). However, if there is some for which/(X) does not contain zero,
then there is no solution in X Either/(X)> 0 or /(X())< 0. This is a
nonexistence theorem, so to speak. We can make it into an exclusion principle.
We can perform a cyclic sequence of bisections of X as follows.
Bisect X in coordinate direction x 1. Exclude half or halves of X for which
0 /(X) for some i. Construct list (1) from the half or halves of X() for which 0 is
n. Bisect, in coordinate direction x2,
1, 2,
in the range of values of fi for all
all resulting regions X for which
from
and
list
each region in list (1);
construct
(2)
n.
Continue
this
in
way with list (kn +j) resulting from
1, 2,...,
0/(X),
of
the
direction
coordinate
bisection in
regions in list (kn +/"- 1). Any solutions
xj
which the system (2.1) may have will lie in regions contained in list (m) for any m.
If the list (m) becomes empty for some m, then there are no solutions of (2.1) in
.
.
,
x(O).
1053
SAFE STARTING REGIONS
In one dimension (n 1) this procedure, applied to a function which crosses
the axis once in [a, b (unique solution in [a, b ]), will produce the same sequence
of subintervals of [a, b] as the usual bisection method. On the other hand, unlike
the usual bisection procedure, this bisection procedure can also find multiple
zeros.
-I
+/
FIG. 1. Bisection in two dimensions.
A two-dimensional illustration may help to make the general procedure clear.
See Fig. 1. Twenty-three bisections carried out just as described produces a
smaller rectangle th the diameter of the initial rectangle containing the solution.
The solid curves represent loci of zeros of [1 or f2 separately. The regions in
between are marked + +, +
+, or
according to the signs of ]’1 and [2 (resp.)
in those regions. At the stage shown, only the small rectangle containing the
intersection of [1 0 with f2 0 need be further bisected. Thus, a rectangle need
be further bisected only when it contains a zero of [1 and a zero of [2. The lists
described for the procedure contain the following numbers of regions. List (1):2,
list
(2):4, list (3):4, list (4):3, list (5):4, list (6):2, list (7):3, and list (8): 1.
Even when we cannot compute the exact ranges of values of the functions
]’,, it is still possible to use bisection procedures in n dimensions. If interval
F,, ([8], [9]) of the functions/1,"
extensions, F1,"
L are available (as they
almost always will be), then we can compute intervals F1 (X),.
F, (X) containing the ranges of values of fx,
f, (resp.) for x in an n-dimensional rectangle X.
If any one of the intervals F(X),
1, 2,.. n does not contain the number 0,
then there is no solution of (2.1) in X. Since F(X) may be wider than the actual
range of values of for x in X, this may require more bisections than if we were
able to compute the exact range of values of fl,""", f,. Nevertheless, such an
]’1,"
,
,
,
,
1054
R.E. MOORE AND S. T. JONES
approach can be used effectively, as will be shown in this paper, to find safe
starting regions for iterative methods.
As a stopping.criterion for the bisection procedure we can make use of
computationally verifiable sufficient conditions for convergence of a chosen
iterative method to a solution of (2.1). Thus, the bisection procedure will
terminate when a safe starting region is found for a more efficient iterative
method.
In Fig. 1 we can see that the small rectangle containing the solution can be
reached in only 8 bisections if we could always choose first the bisected half which
contains the solution. While it might not be possible to devise a procedure which
will always make the correct choice on the first try, we can devise procedures
which will often make the correct choice on the first try. This will reduce the time
required to search and, if necessary, bisect the remaining region until a region is
found which satifies sufficient conditions for convergence for some more efficient
iterative method. Preliminary studies seem to indicate that we can also improve
the efficiency of a bisection procedure in n dimensions by using more sophisticated rules for choosing coordinate directions than the simple cyclic rule. Both
these possibilities will be explored in the next section.
Dussel 1] has developed a method using interval computation of gradients in
a recursive, cyclic bisection procedure for bounding the minimal point of a
strongly convex function in a sequence of n-dimensional rectangles whose
maximal width goes to zero.
_
3. An interval search procedure for Krawczyk’s method. We assume that
D Rn--> R is continuously differentiable in the open set D. We assume,
:further,
the availability of machine computable interval extensions F, F. of the
functions [i,
1,. , n, in (2.1) and of the coefficients [i of the Jacobian matrix
of the system. Denote by In[D] the set of n-dimensional rectangles contained in
D. An element of In[D] can be represented by an interval vector X=
(X1,"
Xn) where X/is a closed bounded interval of real numbers.
More precisely, the interval valued functions F/, FI. are assumed to have the
,
following properties.
_ _
1) F,., F., (i,/" 1,..., n) are interval valued functions defined on In[D].
(This will automatically be the case if F/(D) and FIj(D) are defined-which we can verify by machine computation!)
2) f (x) Fi (x), ]j(x) F(x) for all x in D.
3) (inclusion monotonicity [8]) For all X, B in In[D], X B implies F/(X)
F/(B) and F(X) F.(B) for all i,/" 1,.
n (In matrix notation, X B
implies F(X) F(B) and F’(X) F’(B)).
Properties 2) and 3) hold, for instance, for natural interval extensions of rational
functions, [8], and interval extensions of exp, log, sin, cos, etc.
We will use the following notation in this section (see [8]). For an interval
[a, b], a =< b, define
,
I[a, b]l
max (lal,
Ibl),
w([a,b]):b-a,
m([a,b])=(a+b)/2.
1055
SAFE STARTING REGIONS
For an interval vector X (X1,’’’, X,), define
Ilxll- m.ax IX l,
w (X)
max w (X/),
m (X)
(m (X1),
m (X,,)).
For an interval matrix A, define
Ilmll-- m.ax ]=E IAiil,
w(Aij),
w(A)= max
i,]
m(A)= the real matrix with components m(Aij).
Consider the algorithm ([5], [10]), for k 0, 1, 2,.
X(k+l := X’) K(X’)) (intersection componentwise),
(3.1)
,
where
K(X)) :=
m(X))- y)F(m(Xk)))+ R()(X)- m(X<)))
with
R): I- IAk)F’(x<k)),
(I is the n n identity matrix)
and
_
Y, an approximation to [m(F’(X(k)))] -1 for k =0 and for k >0 if
Y-)
otherwise.
The algorithm (3.1) does not require the inversion of interval matrices. It has
been shown to have the following properties ([5], [10]).
1) XCk +l) X) for all k 0, 1, 2,.
2) If X<) contains a solution of (2.1), then so does K(X)). Thus, if
K(X)) 71X) is empty, then there is no solution in X<). This is another
computationally verifiable sufficient condition for nonexistence of a solution in an n-dimensional rectangle X<) (in addition to 0 e F(X<))).
3) The sequence {tlR)II} is nonincreasing and
.
w (x
+x)) __< IIR )llw (X)).
4) If K(X))_X), then there is a solution of (2.1) in X). This is a
computationally verifiable sufficient condition for existence of a solution
of (2.1) in an n-dimensional rectangle X).
s) If K(X))=_ X) and IIR)II < 1., then there is a unique solution to (2.1)in
X). Furthermore, the solution is in X) for all k 0, 1, 2,. and
{ w (X))} -> O.
This is a computationally verifiable sufficient condition for convergence of
the iterative algorithm (3.1) to a solution of (2.1).
.
1056
E. MOORE AND S. T. JONES
The convergence is at least linear; in fact
<__
{
iIg
/=0
,11
}
<=
Under certain conditions, the convergence is quadratic. Assuming the conditions
of property 5) above, we have the following:
THEOREM. If the Jacobian o]’ the system (2.1) is nonsingular in a
neighborhood of the solution, (for instance, if ][[F’(X())]-I[I _<- b); if the elements ofF’
satisfy a Lipschitz condition" for some L >0, w(F’(X)ij) <-_Lw(X) for all X c_ X()
and all i,] 1, 2,..., n; and if IA k) is a sufficiently good approximation to
[m(F’(X{)))] so that IA)= [m(F’(X)))] -1 +E ) with ]IE)I]<-_ Cw(X{)), then2
the convergence of the algorithm (3.1) is quadratic; that is, w(X(k+l)) <-q[w(X(k))]
]’or some q independent of k.
Proof. We have
-
IIR )11 lIE )11 IIF’ (X 311 + &Ill m (F’(X(k)))]- 11[ nL w (X())
(
<- { CIIF’ (X{))[I + 1/2bnL } w (X{k )).
Therefore,
w(x(k + 1)) w(K(X(k))) ilR()llw(X())
<= { ClIF’ (X())ll + 1/2bnL }[ w (X())]2.
We can take q CIIF’(X())I + 1/2bnL and the theorem is proved.
We can now present a search procedure for finding safe starting regions for the
algorithm (3.1).
Let B be an arbitrary n-dimensional rectangle (interval vector) contained in
the domain D of the functions in the system (2.1). We do not require the
nonsingularity of the Jacobian matrix f’ in B.
We begin by noting that for any region X c_g_ B exactly one of the following
conditions will hold:
1) X satisfies the existence and convergence criteria of algorithm (3.1).
2) X satisfies one of the nonexistence criteria of algorithm (3.1).
3) X satisfies neither 1) nor 2).
Determination of which of the above conditions holds is called analysis of the
region X.
The search procedure consists of a recursive application of this analysis,
beginning with the region X B. At each level, we analyze the region X. If X
satisfies condition 1), we designate it a safe starting region for the solution of the
system (2.1). If X satisfies condition 2), then no solution of (2.1) is contained in X
and this region is excluded from further consideration. If condition 3) holds, we
bisect X, if possible, in some appropriately chosen coordinate direction and select
one of the two resulting half-regions for analysis at the next level. We save the
other half-region, at the current level, for analysis in the event that the selected
half-region is subsequently excluded. This ensures that no potential safe starting
regions are lost during the search. Further, exclusion of both half-regions resulting
from the bisection of X automatically excludes X. This ensures that each region is
analyzed at most once. The reader may wish to verify that the search procedure
SAFE STARTING REGIONS
1057
described is equivalent to the generation and preorder traversal of a binary tree.
(See [4].)
If condition 3) holds and it is not possible to bisect X because of the limited
resolution of computations using fixed precision computer arithmetic, there may
still be a solution of (2.1) in X. In this case we add X to a list of regions too small for
further analysis (without going to higher machine arithmetic precision) and
continue the search as if this region had been excluded. Since B is of finite
dimension and its components are of finite width, there can be only a finite number
of such regions obtained by bisection.
Thus, the search procedure, described in detail below, will, in a finite number
of steps, do one of the following three things"
1) find a safe starting region X within B for convergence of the algorithm
(3.1) to a solution of (2.1);
2) discover that there are no solutions of (2.1) in B;
3) terminate with a list of small regions within B which might contain
solutions of (2.1), (the remainder of B contains no solutions).
Search procedure for algorithm (3.1):
List T is the list of subregions of B yet to be tested.
List P is the list of subregions of B which may contain a solution to (2.1) but
which are too small for further analysis.
Unless otherwise indicated, Step rn + 1 follows Step m.
Step 1 (Initialization). Set list T to empty; set list P to empty; set X to/3.
Step 2. Compute F(X).
Step 3 (Exclusion). If 0 F(X), go to Step 11.
Step 4. Compute Y [m(F’(X))]-; if not possible, go to Step 9.
and K(X).
Step 5. Compute
Step 6 (Exclusion). If X f-I K (X) is empty, go to Step 11.
Step 7 (Existence). If K(X)_X, then X (and also K(X)) contains a
solution, continue; if not, go to Step 9.
8
< then X is a safe starting region X () for the
(Test II). tt
Step
algorithm (3.1)terminate search; otherwise set B to K (X) and
go to Step 1.
Step 9 (Bisection). Bisect X according to rules described below; if no
further bisection is possible, add X to list P and go to Step 11.
Step 10. Set X to half-region selected according to bisection rules; add
remaining half-region to head of list T; go to Step 2.
Step 11 (Test list T). If list T is empty go to Step 12; otherwise set X to
region at head of list T, delete this region from list T and go to Step
2.
Step 12 (Test list P). If list P is empty, terminate with no solution in B;
otherwise print list P and terminate.
Some comments are in order. We will elaborate on Step 9 presently. In order
to use this search procedure with an iterative method other than the algorithm
(3.1), Steps 4-8 would have to be replaced by tests appropriate to the alternative
iterative method for existence and convergence. Note that in Step 6 we have taken
advantage of another exclusion test (in addition to that of Step 3) which is
available for the algorithm (3.1). By putting the untested half of the most recently
bisected region at the head of the list of regions yet to be tested, the procedure can
IIR
IIR
IIR
1058
R.E. MOORE AND S. T. JONES
be seen to be a "depth first" search. A sequence of smaller and smaller regions is
searched for a safe starting region (hopefully, to find one quickly). The procedure
"backtracks" (searching neighboring regions of the same or larger size) only after
exhausting a particular "line of search".
We now discuss bisection rules for Step 9 of the search procedure. When
bisection of a region X is to be carried out, there are two decisions to be made:
1) in which coordinate direction should X be bisected?
2) which bisected half of X should be searched first?
Note that the procedure always saves the untested half of a bisected region
for possible examination later.
If X cannot be further bisected within the limited machine number precision
being used, then we can indicate "possible solution in X" and go to Step 11.
We only bisect a region X after it has been determined that: 0 F(X); and
either
1) we cannot find an approximation Y to [m(F’(X))]-I;
or 2) K(X)O X is not empty, but K(X) is not contained in X.
Thus, we only bisect a region X when it still might contain a solution.
The number of bisections required to find a safe starting region will depend
on the rules used to choose coordinate directions and bisected halves. Further
work on this question is certainly needed. We have, on the basis of preliminary
computer experiments and some theoretical considerations to be given in 4,
arrived at some provisional rules which give encouraging results.
In order to guarantee that X() is a safe starting region for algorithm (3.1), we
need
(and K(X)
_
IIR )ll III-
<1
X)).
We will assume that y(O) has been found as an approximation to
[m(F’(X()))] We can write
-.
F’(X)) m(F’(X)))+ W
where
1/2[- 1, 1] w (F’ (X)i ).
Then IIR()II is approximately IIY<)wll. This motivates our choice of a rule for
Wj
selecting a coordinate direction in which to bisect X().
We attempt to decrease IIRI[ by decreasing
w IF’ (X)] := max w (F’ (X)ii).
ii
In many problems, the system (2.1) will have a sparse Jacobian matrix.
Furthermore, we will know (from the given functions fl,""", f,) which of the
variables x l, x2,’’’, x, occur in any given element of the Jacobian matrix (and
hence in F’ (X)ii ).
To choose a coordinate direction for the bisection of X:
1) find a pair i, j for which w(F’(X)ii)= w(F’(X))
1059
SAFE STARTING REGIONS
2) bisect X in each of the coordinate directions in turn, which occur in F’(X)i
and choose the first one for which
,
w{F’(XI)i F’(X2)}
is minimum,
where X X1 LI X2 and X X2 are the halves of X resulting from the above
trial bisections.
Having chosen, in this way, a bisection direction, we select one of the
resulting halves Xh, h 1, 2, for which
Im(F,.(xh))l
is minimum.
This choice is motivated by an attempt to select a bisected half of X which is
most likely to contain a solution of (2.1). It is a symmetry test--selecting a half of
X on which the values of the functions f, f2,
f, seem to be closest to zero "on
the average".
This test is, to be sure, only weakly justified and the question of how best to
select a bisected half during the search procedure certainly deserves further study.
Nevertheless, this is the decision rule which was used to obtain the results given in
the next section.
It can be shown by example that the test for existence, K(X) X (Step 7 of
the search procedure), might not be satisfied even for a very small region X
containing a solution if the solution lies exactly on the boundary of X. If we obtain
a termination of the search procedure resulting only in a list of some small regions
which may contain solutions, we could enclose an intersecting group of these in a
slightly larger region and repeat the search procedure there. This would likely
yield, finally, a safe starting region satisfying the existence test.
,
4. Computational results and comparisons. A simple two-dimensional
example will illustrate how the search procedure works. Consider the system
f(Xx, x:z):= x2 + x2- 1 0,
f(Xl, X2): x-x2=O;
and consider the two-dimensional rectangle B ([-1, 1], [0, 1]) shown in Fig. 2.
We can apply the search procedure described in 3 to look for a solution to (4.1)
(4.1)
in B.
4
--I--
I i 11
3
2
FIG. 2. A two-dimensional example.
In order to follow what happens, note that the function f vanishes on the
semi-circle shown and ’a vanishes on the parabola shown. There are two solutions
.
1060
E. MOORE AND S. T. JONES
of the system in B, where the curves intersect. For an interval vector X
we can take
v(x)= (X
\
+X
(X1, X2)
1)
and
F,(X)= (2X1
\2X1
2_2)
as our interval extensions of the vector function
f(X )
(fl(Xl, X2)
\f2(X 1, X2)]
and the Jacobian matrix of the system
2X2’
/’(X)= 2Xl -11
2xl
respectively.
Note that the Jacobian matrix is singular at the points x 0, x2 [0, 1 in B.
This causes no di]ficulty for the search procedure. The search proceeds as follows
(see Fig. 2).
Initially, X B ([- 1, 1 ], [0, 1]).
1) m(F’(X))=
has no inverse, so bisect; coordinate direction 1
0 -1
chosen; symmetry rule chooses either half (say right), X ([0, 1], [0, 1]).
2) K(X)C-X, so bisect; coordinate direction 1 chosen; symmetry rule
chooses right half; X= ([0.5, 1.01, [0, 1]).
3) K(X)C-X, so bisect; coordinate direction 2 chosen; symmetry rule
chooses upper half; X ([0.5, 1.0], [0.5, 1.0]).
4) K(X) X, so bisect; coordinate direction 1 chosen; symmetry chooses left
-
half; X= ([0.5, 0.75], [0.5, 1]).
5) K(X)ff;X, so bisect; coordinate direction 2 chosen; exclusion rule
(0 F(X)) chooses lower half; X ([0.5, 0.75], [0.5, 0.75]).
so exclude ([0.5, 0.75], [0.5, 0.75]); ([0.5, 0.75], [0.75,
6) K(X)f-IX=
1.0]) excluded in 5), so exclude ([0.5, 0.75], [0.5, 1.0]) and choose X=
([0.75, 1.0], [0.5, 1.0]) from list T.
7) K(X)ff:X, so bisect; coordinate direction 2 chosen; exclusion rule
(0 e! F(X)) chooses lower half; X= ([0.75, 1.0], [0.5, 0.75]).
8) K(X)_X and IIRI1<0.34< 1, so solution exists in X and a safe starting
,
region has been found.
X{)= X= ([0.75, 1.0], [0.5, 0.751).
Six bisections were needed by the search procedure.
The algorithm (3.1) will now converge rapidly to a solution from X{). Four
iterations produce a solution to an accuracy of 8 decimal places. Taking {m(X{k))}
1061
SAFE STARTING REGIONS
as our approximations to the solution x* e X(), we have
lira (X() x’l]--< 1/2w(X(’)
and we find the results in Table 1.
TABLE
k
2 W(X(k))
0
0.1250000000
0.0416666716
0.0027429238
0.0000126809
0.0000000075
2
3
4
The search procedure has been implemented in NUALGOL on the UNIVAC
1110 computer at the University of Wisconsin-Madison as a recursive procedure.
We have tested the program on some examples chosen from a report [3] which
discusses a continuation method for finding fixed points. If g has the fixed point x,
then f(x):= x g(x)= 0 has the solution x and vice versa. The examples from [3]
n,
1, 2,.
on which we chose to test our search procedure have the form,
,
/l(x)
(4.2)
xi- ai- biXilXi2Xi3,
1 =< i, il, i2, i3 =< n
where, for each
1, 2,..., n, the integers il, i2, i3 and coefficients a and b
were "randomly" generated under the restrictions:
al>0,
1/4>b>0,
a+bi<l;
i, il, i2 i3 distinct integers in {1, 2,.
, n}.
For the interval extensions of f and f’, we took
Fi(X) Xi
ai
1
0
F’(X),
biXi lXi2Xi3,
for/’= 1
for j
-biXipXiq for j
i, il, i2, i3
il, i2, i3
with {ip, iq, j}= {il, i2, i3}.
The numerical values of the indices and coefficients actually chosen by the
computer are given in Table 2 for three cases corresponding to n 5, 10, 20.
We considered three choices for the initial region B: 1) Bi=[0, 1], 2)
B--[-1, 1], and 3)B=[-2, 2] (for i= 1, 2,..., n).
For all three values n 5, 10, 20 neither the first nor the second choice of B
< 1 required for convergence
required any bisections to reach the condition
of the algorithm (3.1). For the third choice, B [-2, 2], the bisections shown in
Table 3 were carried out for n 5 (9 bisections) and n 10 (17 bisections) to
reach the convergence criterion [IRll < 1.
IIRII
1062
R.E. MOORE AND S. T. JONES
TABLE 2
Function coefficients and variable indices.
2
3
4
5
A
B
I1
I2
I3
0.25400000
0.37799999
0.27099999
0.44200000
0.07100000
0.12500000
0.21300000
0.19399999
0.06300000
0.18500000
2
3
1
2
4
5
4
5
4
3
2
5
3
10
2
3
4
5
6
7
8
9
10
A
B
11
12
13
0.25428722
0.37842197
0.27162577
0.19807914
0.44166728
0.14654113
0.42937161
0.07056438
0.34504906
0.42651102
0.18324757
0.16275449
0.16955071
0.15585316
0.19950920
0.18922793
0.21180486
0.17081208
0.19612740
0.21466544
4
1
3
10
2
9
6
10
6
3
10
8
6
8
7
7
8
2
10
4
6
5
5
7
6
8
20
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
A
B
I1
I2
I3
0.24863995
0.87528587
0.23939835
0.47620128
0.24711044
0.33565227
0.13128974
0.45937304
0.46896600
0.57596835
0.56896263
0.70561396
0.59642512
0.46588640
0.10607114
0.26516898
0.20436664
0.56003141
0.92894617
0.57001682
0.19594124
0.05612619
0.20177810
0.16497518
0.20198178
0.15724045
0.12384342
0.18180253
0.21241045
0.16522613
0.17221383
0.23556251
0.24475135
0.21790395
0.20920602
0.21037773
0.19838792
0.18114505
0.04417537
7
18
10
12
8
16
12
19
13
12
16
14
7
13
10
8
7
15
9
18
13
15
2
9
17
11
16
3
9
19
10
13
13
3
16
11
11
0.17949149
4
20
6
7
16
11
15
18
17
13
8
4
20
10
10
9
13
8
16
11
In [3] only the case Bi--[0, 1] is considered. We found this B to be a safe
starting region already for the algorithm (3.1), which would then converge
quadratically for this problem in a few iterations. For the case n 20, it was
reported in [3] that, for five different sets of coefficients, the "number of scalar
function calls per problem" was 1,160-1,500. (We did, however, use the restric-
1063
SAFE STARTING REGIONS
TABLE 3
=5, Bi=-[-2, 21
Ordinal number
of bisection
Direction
chosen
Half
chosen
1
[0,21
[0, 1]
[0,2]
[0,1]
[0, 21
[0, 11
[0,2]
[o, 11
[0,0.5]
2.556
2.556
1.704
1.797
1.552
1.500
1.051
1.049
1.oo2
0.720
[0,2]
[0,11
[0,2]
[0,1]
[0,2]
[0,1]
[0, 2]
[0, 11
[0,2]
[0,1]
[0,2]
[0,1]
[0,2]
[0,1]
[0,2]
[0,1]
[0, 0.5]
2.576
2.576
2.542
2.542
2.542
2.542
2.395
2.395
2.395
2.395
2.354
2.354
2.354
2.354
1.570
1.480
1.020
0.988
0
2
3
4
5
6
7
s
9
3
3
2
2
4
4
1
n=lO, Bi=-[-2,2l
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
4
4
5
5
2
2
3
3
7
7
6
6
10
10
tion bi < in addition to those in [3] in order to cut down on computer time to find
approximate inverses for Y in these preliminary studies. We also restricted the
four indices i, il, i2, i3 to be distinct in order to simplify programming of F’.)
Continuation methods have been discussed by a number of authors (see, e.g.,
[2], [3], [6], [11], and [12]). For instance, the basic appealing idea in [6] is to
consider the problem
g(x, t):= f(x)- f(xo)+ tf(xo) 0
for an "arbitrary" Xo. When
0, the problem has the solution x
solve the problem
g (x, 1) f(x)
x0.
In order to
O,
one subdivides the interval [0, 1] into small enough steps tp+l--tp SO that an
approximate solution to g(x, tp)= 0 is a safe starting point for Newton’s method
for solving the system g(x, tp/l)= 0.
1064
R.E. MOORE AND S. T. JONES
While there is no indication given in [6] of how to computationally verify the
sufficient conditions derived for safe starting points for the intermediate problems, this could probably done using interval methods as indicated in this
paper.
A more serious drawback of the continuation method [6], however, is that, in
order to prove that it will work from a given Xo, it was assumed that the Jacobian
matrix is invertible in a neighborhood of x0 which has to be large enough to also
contain a solution. As was seen for the two-dimensional example (4.1), our search
methods works even with initial regions containing points where the Jacobian
matrix is singular.
In fact, for the norm we used ([Ixl[- max (Ixxl, Ix21)), the conditions given in [6]
for the continuation method to work cannot be satisfied for the point Xo=
(0.75, 0.75) in the safe starting region we found for the two-dimensional example
(4.1). The conditions [6] are that there must be an r > 0 such that
II/(x0)ll <-for all
llx xoll <-_ r,
3ro< r/2.
To satisfy these conditions for the continuation method the "arbitrary"
starting point Xo must lie within a distance of no more than about 0.1 of the
solution! In particular, if Xo (xol, Xo2), then x01, Xo2 must satisfy
(3/4)Xo21 h- X02 < 1 < (5/4)Xo1 -b Xo22,
(3/4)x1 <x0 < (5/4)x1.
The reader can verify for himself that this is a rather small region.
REFERENCES
[1] R. DUSSEL, Einschliessung des Minimalpunktes einer streng konvexen Funktion auf einem
n-dimensionalen Quader, Interner Bericht Nr. 72/2, Institut fur Praktische Mathematik,
Universitat Karlsruhe, Karlsruhe, West Germany, 1972.
AND H. SCARF, The solution of systems of piecewise linear equations, Math. of
Operations Res., (1976), pp. 1-27.
R. B. KELLOGG, T. Y. LI AND J. YORK, A constructive proofofthe Brouwerfixed point theorem
and computational results, Tech. Note BN-810, Institute for Fluid Dynamics and Applied
Math., Univ. of Maryland, College Park, 1975.
D. KNUTH, Fundamental Algorithms, Addison-Wesley, Reading, MA, 1968, pp. 315-319.
R. KRAWCZYK, Newton-Algorithmen zur Bestimmung yon Nullstellen mit Fehlerschranken,
Computing, 4 (1969), pp. 187-201.
H.T. KUNG, The complexity of obtaining starting points for solving operator equations by Newton’s
method, Analytical Complexity, J. F. Traub, ed., Academic Press, New York, 1975.
C. MIRANDA, Un’ osservazione su un teorema di Brouwer, Boll. Un. Mat. Ital., 3 (1941), pp. 5-7.
R. E. MOORE, Interval Analysis, Prentice-Hall, Englewood Cliffs, NJ, 1966.
On computing the range of a rational function of n variables over a bounded region,
Computing, 16 (1976), pp. 1-15.
A test for existence, this Journal, 14 (1977), pp. 611-615.
J. M. ORTEGA AND W. C. RHEINBOLDT, Iterative Solution of Nonlinear Equations in Several
Variables, Academic Press, New York, 1970.
L. B. RALL, Computational Solution of Nonlinear Operator Equations, John Wiley, New York,
[2] B. C. EAVES
[3]
[4]
[5]
[6]
[7]
[8]
[9]
10]
11]
[12]
,
,
1969.
SAFE STARTING REGIONS
1065
13] H. SCARF, The approximation offixed points of a continuous mapping, SIAM J. Appl. Math., 15
(1967), pp. 1328-1343.
[14] F. STENGER, Computing the topological degree of a mapping in R", Numer. Math., 25 (1975), pp.
23-38.
15] M.J. TODD, The Computation ofFixed Points and Applications, Lecture Notes in Economics and
Mathematical Systems, No. 124, Springer-Verlag, New York, 1976.