STATISTICAL ANALYSIS OF BOUNDARIES:
A NONPARAMETRIC APPROACH
by
Charu Krishnamoorthy
A dissertation submitted to the faculty of the University of North Carolina
at Chapel Hill in partial fulfillment of the requirements for the degree of
Doctor of Philosophy in the Department of Statistics.
Chapel Hill
1991
Approved by:
Advisor
Reader
CHARU KRISHNAMOORTHY.
Statistical Analysis Of Boundaries: A Nonparametric
"
Approach. (Under the direction of Edward Carlstein.)
ABSTRACT
A data set consists of independent observations taken at the nodes of a grid.
unknown boundary
e partitions the grid into two regions,
An
...( and ...('. Those observations with
index in ...( (...(') come from distribution F (G). The two distributions F and G are entirely
unknown, and they need not differ in their means, medians, or other measures of "level." The
grid is of arbitrary dimension, and the mesh rectangular.
Our objective is to estimate the
boundary, with no distributional assumptions.
A class of nonparametric estimators is proposed.
We obtain strong consistency for
these estimators (including rates of convergence and a bound on the error probability). The
boundary estimate is selected from an appropriate collection
cr of candidate
boundaries, which
must be specified by the user. The candidate boundaries, as well as the true boundary, must
satisfy certain intuitively natural regularity assumptions.
This includes a "smoothness"
condition on the boundary and a "richness" condition for the candidate pool,
In practice, one may be faced with a
cr which is not sufficiently
cr.
"rich": How robust is
the estimator in this situation? This question is addressed by studying the asymptotic error of
,
the estimator, and comparing it to the smallest possible error in
cr.
The boundary-estimation problem has applications in diverse fields, including: quality
control, epidemiology, forestry, marine science, meteorology, and geology.
Our method
provides (as special cases) nonparametric estimators for the following situations: the changepoint problem; the epidemic-change model; templates; linear bisection of the plane; Lipschitz
boundaries. Each of these applications is explicitly analysed.
-11-
.
A simulation study provides numerical evidence that the boundary estimators work
well.
In these simulations, the two distributions actually share the same mean, median,
variance, and skewness. As an illustration, a boundary estimate is
calcu~ated
on a data-grid of
U.S. cancer mortality rates. A non-standard bootstrap procedure is proposed for studying the
variability of the boundary estimator.
Simulations of this bootstrap procedure in the linear
bisection case are used to construct "indifference zones". In our examples these zones seem to
accurately reflect the true variability of the boundary estimator.
-iii-
•
Dedicated to my mother and the memory
of my father: I hear their voices more
clearly each passing day.
•
-iv-
..
TABLE OF CONTENTS
A.
INTRODUCTION
1
B.
REVIEW OF THE LITERATURE
5
B.O
Introduction
5
B.1
The Change-Point Problem
5
B.2
B.3
B.4
B.1.(a)
Parametrics
B.1.(b)
Nonparametrics
B.1.(c)
Testing of Hypothesis
B.1.(d)
Dependence in the X/s
B.1.(e)
Bayesian Analysis
Variations of the Change-Point Problem
B.2.(a)
Epidemic-Change Problem
B.2.(b)
Multiple Changes: With More Than Two Distributions
B.2.(c)
Distributions Dependent on the Indices
B.2.(d)
Non-Abrupt Change
B. 2. (e)
Distinguishing {F,J from {G,J
General Boundary Problem (d 2': 2)
B. 3. (a)
Parametrics
B. 3. (b)
Nonparametrics
B. 3. (c)
Image Processing
Surveys and Bibliographies
C.2
14
The Statistical Problem
c.1.(a)
Introduction
C.1.(b)
Boundaries
C.1.(c)
Outline of this Chapter
11
13
C. THE BOUNDARY ESTIMATOR
C.1
8
14
Examples
16
1-Dimensional Case
l.a
The Change-Point Problem
l.b
The Epidemic-Change Model
2-Dimensional Case
2.a
Linear Bisection
2.b
Templates
-y-
2.c
•
Lipschitz Boundaries
Higher Dimensional (d ~ 3) case
3.
C.3
CA
C.5
Oriented Hypercube
The Boundary Estimator
20
C. 3. (a)
The Basic Idea
C.3.(b)
Definition of the Boundary Estimator
Towards Consistency Properties of the Boundary Estimator
C. 4. (a)
Measuring the "Distance" Between Boundaries
C. 4. (b)
The Norm 5(.)
C.4.(c)
Regularity Conditions on Boundaries
C.4.(d)
Asymptotic Results
Examples Revisited
22
27
I-Dimensional Case
l.a
The Change-Point Problem
l.b
The Epidemic-Change Model
2-Dimensional Case
2.a
Linear Bisection
2.b
Templates
2.c
Lipschitz Boundaries
Higher Dimensional (d ~ 3) case
3.
D.
E.
F.
Oriented Hypercube
ROBUSTNESS OF THE BOUNDARY ESTIMATOR
33
0.0
Introduction
33
0.1
Definition of "Asymptotic Error Target"
34
0.2
Robustness of
e
35
I
SIMULATIONS & NUMERICAL APPLICATIONS
38
E.O
Introduction
38
E.1
Efficient Computation
38
E.2
Simulation Studies
42
E.3
U.S Cancer Mortality Data
53
EA
Application of the Bootstrap to
e
I
PROOFS
55
61
Theorem 1
------------------------------------------------.----------------- 61
Theorem 2
------------------------------------------------------------- 62
Theorem 3
-------------------------------------------------.----------------- 62
-vi-
•
Theorem 4
---------------------------------------------------------------- 63
Theorem 5
------------------------------------------------------------- 62
Theorem 6
------------------------------------------------------- 63
Proposition 1--------------------------------------------------------- 61
Proposition 4 -------------------------------------------------------- 70
Proposition 5 ------------------------------------------------------------ 70
Proposition 6 -------------------------------------------------------- 71
Proposition 7 ------------------------------------------------------- 71
REFERENCES
73
APPENDIX
80
Source Code For The Linear Bisection Problem (Simulation)
Source Code For Lipschitz Program (Data)
Source Code For The Bootstrap Calculations (Simulation)
Preliminary Estimates Of The Lipschitz Boundary:
Different Lipschitz Constants.
-vii-
LIST OF FIGURES k TABLES
FIGURE A.1:
A Boundary in CU 2
1
FIGURE C.1:
A Boundary in CU 2 with nl=10, n 2 =20, 111=200
TABLE E.1:
Number of Simple Computations per Estimator calculation, for ...... 41
various examples.
FIGURE E.2:
Two Different Weight Functions
44
TABLE E.3:
Simulation Study when d=l
45
TABLE E.4:
Simulation Study when d=2
46
TABLE E.5:
Maximum Values of ['iii' 0(8,
FIGURE E.6:
Grid Size 05 x 05: L 1 , L 2 , fj L oo Norms
:;.
Histograms & Quantiles of 1000 simulations of ['iii' 0(8,8 1)],
FIGURE E.7:
Grid Size 15 x 15. L1 , L2 , fj Loo Norms
:;.
Histograms & Quantiles of 1000 simulations of ['iii' 0(8,8 1)]
°
1 )]
15
from simulations, NREP=1000 47
48
49
.
FIGURE E.8:
Grid Size 25 x 25. L 1 , L 2 , fj Loo Norms
.;:
Histograms & Quantiles of 1000 simulations of ['iii' 0(8,8 1)] .
50
FIGURE E.9:
Grid Size 35 x 35. L 11 L 2 , fj Loo Norms
:;.
Histograms & Quantiles of 1000 simulations of ['iii' 0(8,8 I)] •
51
FIGURE E.10: Grid Size 45 x 45. L 1 , L 2 , fj Loo Norms
;,.
Histograms & Quantiles of 1000 simulations of ['iii' 0(8,8 1)]
52
•
FIGURE E.ll: Cancer Mortality Of White Males (1970-1979). Fitted Boundary ..... 54
FIGURES E.12.a, h, c, d: Estimated Variability of the Boundary Estimator
-viii-
59
A. INTRODUCTION
We observe a collection of random variables (r.v.) {Xi: i E I}, indexed by nodes i of a
finite d-dimensional grid. The grid is taken to be in the d-dimensional unit cube 'Ud:=[O,I]d.
The unknown boundary
regions,
e and!2.
e
is simply a (d-I)-dimensional surface that partitions 'Ud into two
All observations Xi made at nodes i E
e are from distribution F, while all
observations Xi made at nodes i E !2 are from distribution G. The objective is to estimate the
unknown boundary
e, using only the observed data
{Xi}' Figure A.I illustrates the set-up in
the case d=2.
FIGURE A.I
The 2-Dimensional Case: A Boundary In 'U 2
iE I
!
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
• •
• •
• •
• e.
• •
• •
• •
•
•
•
•
•
•
•
•
• •
•
•
•
•
•
•
•
•
•
•
•
•
•
Xi - F, iE
e
•
-6
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
~.
•
•
•
•
•
•
Xi-G, iE ~
The change-point problem is a special case of the boundary problem when d = 1 and it
also serves as a motivation for the general boundary problem.
~=(O,
Then, 'Ud:=[O, 1), 6=[0, 0),
1] where 0 < 0<1. Another special case of interest when d = 1 is the "epidemic change"
problem described by ~=[01' 02)'
6=[0, 01) U [02' 1], 0 < 01 < 02 < 1.
These problems have
been studied extensively under many different set-ups. Some commonly made assumptions are
listed below. Each author (whose work is discussed in Section B) makes one or more of these
assumptions:
(a) The distributions of F and G are known, in parametric form (e.g., normal). The "change"
is assumed to be in a specific parameter (often the mean).
parameters (if any) are also known.
In some cases the nuisance
Knowledge of the actual pre- and/or post-change
parameters is sometimes assumed. Thus in the simplest case, the only unknown is the time of
change, 0.
(b) Parametric assumptions similar to (a) are made only for F, the pre-change distribution.
Nothing is assumed for G, the distribution of the "out-of-control" process after the change at
time 0.
(c) The forms of F and G may not be known, but the change is assumed to be in the level,
Le., G(x)=F(x-7]) for some 7], where 7] mayor may not be known. This includes the case of
changes in the mean and/or median, and where F is assumed to be stochastically
larger/smaller than G. This is the context in which much of the nonparametric work has been
done.
(d) The pre and post change distributions F and G, are assumed to be continuous.
(e) The two distributions F and G, are assumed to be discrete with finite support.
(f) The X/s are assumed to be independently distributed.
These cases (a) - (f) outline specific situations where the change-point problem has
been studied.
More generally, the d- dimensional boundary problem can been studied in the
-2-
context of:
(i)
Independence of the underlying X;'s or dependence of the underlying X;'s (time-series,
random fields).
(ii) Estimation of the change-point(s)/ boundaries or testing for the existence of change-points/
boundaries.
(iii) Derivation of sampling distributions of test statistics under Ho, HI'
We estimate the boundary in the general d-dimensional case, for independent X;'s,
making no distributional assumptions on F and G. The general boundary estimator developed
here is based on empirical cumulative distribution functions, and hence does not require any
parametric assumptions about F and G.
unknown boundary
e,
While we require some prior knowledge about the
it is not necessary that it be parametrisable.
Chapter B gives a broad survey of the literature in this general area.
Much of the
work reviewed concentrates on the one-dimensional, at most one change (AMOC) problem.
Chapter C presents our fully nonparametric boundary estimator (for arbitrary d),
obtained with none uf the assumptions (a) - (e) mentioned above.
Asymptotic results on
bounds of the error probability and the strong consistency property of the estimator are stated.
These results hold under set-theoretic regularity conditions, which are developed in Chapter C.
These regularity conditions specify that we consider as candidates only those boundaries that
divide the unit cube into non-trivial partitions, that we consider as candidates reasonable
approximations to the true boundary, that the pool of candidates is not too "big" in size
(compared to a function of the sample size), and finally that the unknown boundary is
reasonably smooth (given the resolution of the grid).
Several examples illustrating the
application of the boundary estimator are also discussed in detail. Included as examples are
change-point and epidemic change situations in the one dimensional case, templates and
Lipschitz curves when d=2.
-3-
Chapter D develops robustness properties of the boundary estimator when we have
imperfect knowledge about the form of the unknown boundary.
In such a situation the
condition of "richness" of the candidate pool may be violated. The main result in this chapter
gives an upper limit on the asymptotic error in this case. The physical conditions under which
this upper limit is minimised are also discussed.
Chapter E deals with numerical applications of the boundary estimator.
computation methods are discussed.
Efficient
Results of simulation studies in the change point case
(d=l), and the linear bisection situation (d=2) are given. It is seen that the average error is
reasonably low even for moderate sample sizes (35 x 35). As a practical example, the boundary
estimator is calculated for a grid of data on cancer mortality rates in the U.S., yielding a
Lipschitz curve which demarcates areas of relatively higher and lower cancer rates.
A non-
standard bootstrap procedure to study the variability of the boundary estimator is also
suggested. An "indifference zone" is defined as a measure of this variability when d=2.
Chapter F contains proofs of all theoretical results presented in Chapters C & D.
The Appendix contains the FORTRAN codes used for computations.
works cited, follow Chapter F.
-4-
References to
B. REVIEW OF THE LITERATURE
B.O. INTRODUCTION
The chief advantage of nonparametric methods is their generality. Thus our approach
to the boundary problem with no distributional assumptions, allows for broader applicability of
the method.
comparable
A weaker set of assumptions is required for its validity than in the case of
parametric
estimators.
A nonparametric
method
corroboration in cases where parametric assumptions are made.
also provides
robust
Since the advantage of our
method lies in its lack of assumptions, the relevant literature will be reviewed with special
emphasis on the assumptions made by other authors in deriving their estimators.
The literature on the general boundary problem is not vast. However, much work has
been done in the change-point problem, which is a special case of it. The change-point problem
is a good starting point both historically and as a motivator for the boundary problem.
Another special case of the boundary problem in dimension, d=l, is the epidemic change
problem.
B.l. THE CHANGE-POINT PROBLEM (d:!)
B.1.(a) Parametrics.
The change-point problem has been extensively studied under
para-metric distributional assumptions.
Hinkley (1970) and Hawkins (1977) considered the
problem of change in mean when F & G are normal; they derived the asymptotic distribution
of the likelihood ratio statistic under the null hypothesis of no change. Hinkley and Hinkley
(1970) obtained similar results when F & G are binomial. The change-point problem has also
been studied by Haccou et al. (1988) when F & G are exponential; and Hsu (1979), Gastwirth
and Mahmoud (1986) when F & G are gamma. Srivastava and Worsely (1986) and Henderson
(1986) consider the case of F & G multivariate normal differing only in their means.
Bhattacharya and Frierson (1981) and Lombard (1983) studied this problem using a
control chart approach. These authors assume that there is a small parameter change, where
the general parametric family of F & G is known. Scariano and Watkins (1988) consider a
class of processes that includes linear combinations of the Poisson process and processes based
on binomial observations.
B.l.(b) Nonparametrics.
The basic change-point problem was first considered by
Page (1954, 1955, 1957) using "cusum" procedures, for on-line detection of change in the
r
distribution of a process. He used the cumulative sum statistic Sr=.E (X i -1]) to estimate/test
1=1
change-points when 1] is the known mean level before change.
Little nonparametric work was done for a decade. Then, Bhattacharyya and Johnson
(1968), Sen and Srivastava (1975), and Petitt (1979) all studied the problem nonparametrically
using some form of a linear rank statistic. All these approaches assume that the change is a
shift in the level: G(x)=F(x-1]). Assumptions on the continuity of F & G, and existence of
density functions are also made.
•
Hinkley (1972) studies the case where the forms of F and G are not known. However,
he assumes knowledge of a "sensible" discriminant function (discriminating F from G).
Darkhovskij (1976) proposed a nonparametric estimator based on the Mann-Whitney
statistic.
He assumes
f F(x)dG(x)
::f=
!'
which excludes the case where F and G are both
symmetric with a common median, even though the two distributions may differ in other
parameters, e.g., their spread.
Darkhovskij (1984, 1986), and Darkhovskij and Brodskij (1980) consider a nonparametric approach assuming that both F and G are discrete with finite support. Asatryan and
Safaryan (1986) study the change-point problem assuming that F and G are continuous. They
use a sequence of two-sample statistics that test for equality of distribution functions. In all
-6-
these papers it is further assumed that ()
IS 10
a known interval bounded away from the
endpoints of (0,1).
Completely nonparametric approaches, similar to ours, have been taken by Cs5rgo and
Horvath (1987) and Carlstein (1988). Csorgo and Horvath assume that F & G are continuous.
They use quantile functions to derive a strongly consistent estimator of
e.
Carlstein assumes
nothing about F & G except that they are distinct. Using empirical cumulative distribution
functions and a general norm for the discriminant functions he establishes strong consistency
results and bounds for the error probabilities.
Diimbgen (1988), and Perez-Abreu (1989)
develop completely nonparametric estimators of the change-point, with no assumptions on F &
G.
Diimbgen's method is based on empirical cumulative distribution functions, as is ours,
while Perez-Abreu's method is based on empirical probability generating functions.
B.l.(c) Testing of Hypothesis.
Hypothesis testing for a change-point may take two
forms:
(a)
Ho (F and G are the same)
vs
HI (A change-point,
e exists),
or
Deshayes and Picard (1981) propose a non parametric procedure for testing (a). Picard
(1985) also tests (a) assuming both F & G are Gaussian. Csorgo and Horvath (1987) test (a)
assuming F & G are continuous.
continuous, G
=F(x -77).
by: Hinkley (1970),
Schectman and Wolfe (1988) test (a) when F and G are
Under distributional assumptions, tests for (a) have been proposed
Hawkins (1977), Hawkins (1986) when F & G are normal; Hinkley and
Hinkley (1970), when F & G are binomial; Srivastava and Worseley (1986), when F & G are
multivariate normal differing only in the means; Gastwirth and Mahmoud (1986), and Hsu
(1979) when F & G are gamma; and by Haccou, Medis, and van de Geer (1988), when F & G
are exponential.
Hypothesis (b) has little history
10
the literature.
-7-
The sampling distribution of the
estimator of 8
under a specified Ho(8 = 00) may not be easily obtainable in a nonparametric
situation. A nonstandard bootstrap approach to approximate the sampling distribution under
Ho(6=8o) was studied by Elliott (1988) , and by Diimbgen (1988).
B.l.(d) Dependence in the X/so Picard (1985) considered change-points in the context
of Gaussian processes. The change was assumed to be in the covariance function while the
mean remains constant. Under the null hypothesis of no change, she derives the limiting distri-
bution of a test statistic based on empirical spectral distribution functions. Henderson (1986)
has considered the problem of change in the mean-vector of a multivariate normal distribution,
assuming full knowledge of the covariance matrix.
Darkhovskij (1984, 1985, 1986) has
proposed a nonparametric estimator. He assumes a strong mixing condition on the dependence
of the X/so Carlstein and Lele (1990) establish consistency for their nonparametric estimator,
when the X/s are ergodic.
B.1.(e) Bayesian Analysis. Several authors have studied the change-point problem in
a Bayesian framework.
Chernoff and Zacks (1964) assume location shifts, and normality for
Xi' for the amount of change and for the post-change mean.
Kander and Zacks (1966)
extended these results to the case where the distributions F and G belong to an exponential
family.
Smith (1975, 1980) develops a change-point estimator, (assuming prior probabilities
for the change-point, and for the number of changes), by minimising posterior expected loss for
some appropriate choice of loss function. He gives detailed analyses under parametric distributional assumptions of normal or binomial.
Cobb (1978) has a "conditional frequentist" approach. He assumes complete knowledge
of F and G, and that they have identical supports. Petitt (1981) assumes a locational change
and continuity of F & G.
B.2. VARIATIONS OF THE CHANGE-POINT PROBLEM (d=l).
Variations of the classical change-point problem are of interest in the context of the
-8-
,;
boundary problem, as their generalisations to d
~
2 will also generalise the boundary problem.
B.2.(a) Epidemic-Change Problem. A variation of the single change-point problem is
that of the epidemic-change. The epidemic-change model allows for two change-points Oland
O2 ,
During the epidemic interval [0 1 ,0 2] the X/s follow the epidemic distribution G, and
outside this interval the they follow the distribution F.
Siegmund (1986) considers this
problem when the underlying distribution is normal and the change-points signify shifts in the
mean level.
Bhattacharya and Brockwell (1976) also study the epidemic-change model with
shift in a specified parameter. In most cases, these results can be extended to k > 2 change
points, where k is known.
B.2.(b) Multiple Changes: With More Than Two Distributions. The multiple change-point problem generalises the epidemic-change problem in two ways. It allows for multiple
changes and for a different distribution function after each change-point. Thus the the X/s
may follow distribution F j on the interval [OJ' OJ + I)' This set-up has been been studied by
Schectman and Wolfe (1985) and Darkhovskij (1985).
"level".
The former assumes shifts in the
Haccou and Meelis (1989) have given a ma.ximum-likelihood based solution for this
problem when the underlying X;'s are known to be exponentially distributed.
Schectman (1982), and Schectman and Wolfe (1985) consider the problem of
estimating and testing for multiple change-points. Their nonparametric approach assumes that
G(x) = F(x-f1).
B.2.(c) Distributions Dependent on the Indices.
problem is when Xi ,.., F(i, . ) : i E
e, Xi
A generalisation of the change-point
,.., G(i, . ) : i E
fl,
i.e.,
e demarcates a change in
the form of functional dependence of X;'s distribution on i. The classical change-point problem
is a special case of this problem, by putting F(i, . ) == F( . ), G(i, . ) == G( . ).
A natural example of this is the case of shift in the form of the mean function Jl(i).
Consider Jl(i)=Jlo + (i - 0)[,81 I{i ~ O} + ,82 D{i > O}].
-9-
This is the classical two-phase regression
problem of change in the slope of a regression function. See Sen (1980) for a discussion of the
problem and a test based on aligned rank order statistics. Sen (1983) offers solutions based on
recursive residual rank tests.
More recently, Miao (1988) has suggested a nonparametric
estimator using differences of grouped means.
Scariano and Watkins (1988) have offered
strongly consistent estimators of the change-point when the X/s come from a Poisson process.
B.2.(d) Non-Abrupt Change. So far we have discussed "instantaneous" change in distributions. A generalisation of this is the "smooth" change problem.
We observe X j ...., F(71j) and assuming the change is in the parameter 71j of F, Lombard
(1987) has considered the problem of estimating {Ol' 02} when:
~ 82} + (A I +A2)O{i > 02}'
I
and Al and A2 are unknown nuisance parameters. The classical change-point problem is the
71j
= AII{i ~ 0d + (AI +;2-!J A2) D{OI < i
B.2.(e) Distinguishing {F n} from {G n}. The formulations discussed so far are based
on underlying F and G that are independent of the sample size, n = II].
Consider the case where one wishes to differentiate between sequences of distributions
{Fn} and {G n}, i.e.,
X?...., F n: i E
e,
X? ...., G n: i E ~, when the data consist of n r.v.'s
X?
Bhattacharya (1987) has considered a special case of this under parametric assumptions. He assumes Fn=F(., 71), Gn=F(., 7J+8,~1) where 8>0 and 71 are unknown and In-CO
at a specified rate.
Under independence of the X/s, a collection of regularity conditions and
using a consistent maximum likelihood estimate of 71 (under the hypothesis of no change), he
derives a limiting distribution for the test statistic.
Diimbgen (1988) has considered this set-up, assuming:
liminf
J [dFn(x) + dGn(x)]
>
0 where An = {x E IR: Fn(x)
"# Gn(x)}.
An
His fully nonparametric estimator is strongly consistent for
-10-
e
and has bounds on its
error probabilities similar to Carlstein(1988).
B.3. GENERAL BOUNDARY PROBLEM (d
> 2) •
The general boundary problem is a higher-dimensional analogue of the change-point
problem. Going from d=l to d
~
2 necessitates careful formulation of the concepts of boundary
and error of estimation. Our approach is set-theoretic: we define the boundary as a partition
of CUd' and measure the error by a pseudo-metric based on symmetric difference of sets.
A challenge in higher dimensions (that does not arise in the case d=l), is the
requirement of some degree of smoothness of the boundary.
handles that.
The regularity condition R.4
A certain amount of prior knowledge of the form of the boundary is usually
required for results in higher dimensions - in the one-dimensional case the set of points {2/n t ,
3/n t , ..., nt/nIl is asymptotically dense in [0,1] and is the logical candidate pool.
The
regularity condition R.2 formulates the prior knowledge required in our case. Finally, in order
to derive strong consistency of our estimator, the cardinality of the candidate pool has to
restricted. In the one dimensional analogue, the logical candidate pool described above, poses
no such problem.
The following is a review of relevant work in the area of the general boundary
problem. The literature when d
~
2 is not vast.
B.3.(a) Parametrics. Rudemo, Skovgaard, and Stryhn (1990) estimate the boundary
in the two dimensional case, when Xi's are independent, assuming that F & G are known,
using maximum likelihood methods. For boundaries with bounded total length (similar to our
Regularity condition R.4.), their curve estimator converges in Lt to the true boundary.
B.3.(b) Nonparametrics.
The general boundary problem has been considered by
Brodskij and Darkhovskij (1986). For arbitrary d they estimate boundaries definable in terms
of a known function of K unknown parameters (K known).
-11-
Under mixing conditions on the
dependence structure of of X;'s, their estimated K-vector of parameters is found to be weakly
consistent for 6. They assume that both F and G are discrete with finite support.
B.3.(c) Image Processing. The boundary problem has also been explored in the context of edge detection and image processing, which are inherently two dimensional problems.
The data on hand is a set of "noisy" observations on (finitely many) pixels in a grid.
An
underlying assumption is that pixels close together tend to have similar distributions. "Edges"
separate regions with different "shades."
It is assumed that there is a "true" value for each
pixel and that noise in the data may obscure it. Since the shades of gray levels can come from
a whole spectrum, the problem is intrinsically more complex than partitioning a set with
respect to two distributions F and G.
Image processing techniques often assume parametric distributions for the noise, e.g.,
normality. Yakimovsky (1976) and other authors use t and F tests to discriminate between
regions of dissimilar means or scatter (of "intensities").
Assume for simplicity that we are considering the general boundary problem of
estimating a partition defined by two regions
e and!';2.
Define colour codes, Zi = O{i E
e} for
all points {i E I} for which data {Xi: i E I} is available. Thus Zi = 1 iff Xi "" F and Zi = 0 iff
Xi"" G. In our formulation, the values of {Zj: i E I}, are deterministic.
A Bayesian formulation adopted by Geman and Geman (1984) and Besag (1985) is
that the Z;'s are random variables and constitute a realisation of a locally dependent Markov
random field, Le., P{Zi=11 Zj: j ::f:. i} = P{Zi=11 Zj: j E (\} for some "neighbourhood" 0i
of i (which does not include i).
Thus, the regions
e and ~
are themselves random. These
authors assume complete knowledge of the probability structure underlying this Markov
random field. It is also assumed that the data {Xi: i E I} are conditionally independent, given
the corresponding Z;'s and further that this conditional density is known.
Both of these Bayesian papers in principle seek to estimate the true scene {Zj: i E I}
-12-
through maximisation of posterior probabilities. They suggest computationally more feasible
approximations to the Bayesian maximiser.
Geman and Geman use "simulated annealing"
procedures and Besag suggests an initial estimate based on maximum likelihood; subsequent
estimates follow from the method of Iterated Conditional Modes (ICM).
B.4. SURVEYS AND BIBLIOGRAPHIES.
Shaban (1980) provides an annotated bibliography of the change-point and the twophase regression problems.
Wolfe & Schectman (1984), and Csorgo & Horvath (1988) have
surveyed some nonparametric approaches in the field.
Siegmund (1986) discusses detection
and estimation of time(s) of an abrupt change; this is also a good source of references for the
epidemic-change case.
Petitt (1981) cites and comments on many of the works from a
Bayesian standpoint.
Basseville & Benveniste (1986) and Telksnys (1986) provide further
references for the change-point and related problems as well the general boundary problem. A
survey of many image processing techniques is given in Haralick and Shapiro (1985).
-13-
c.
THE BOUNDARY ESTIMATOR
C.l. THE STATISTICAL PROBLEM
C.1.(a) Introduction.
The boundary problem is as introduced in Chapter A.
estimate the unknown boundary
e
induces a partition of CUd: (0,~).
distributed as F if i e
0,
We
given data {Xi' i E I}, where I ~ CUd' is a grid. The
e
The observations {Xi' i E I} are independent, and Xi is
and follows G if i e ~ .
The grid I is generated by divisions along each coordinate axis in CUd [see Figure C.l].
Along the jth axis (1:5 j :5 d), there are nj divisions which are equally spaced at
linj' 2/n j' ..., n}ln j' Observations are made at the resulting grid nodes i:=( i1/n 1, i2/n2' ...,
id/nd) E CUd' where i j E {I, 2, ..., nj}'
Thus the grid mesh need not be squares.
A
rectangular mesh allows for different sampling designs along the different dimensions; this in
turn may reflect differing sampling costs in the different dimensions [see Examples in Section
C.2].
The collection of all nodes i is denoted by I, and the total number of observations is
d
I II :=.11
n ·•
J=1 J
In any set A ~ CUd' the number of observations (i.e., grid nodes) is
I A I :=#{i E A}.
C.1.(b) Boundaries. The notion of a boundary in CUd is formulated in a set-theoretic
way:
the unknown boundary
e
is identified with the corresponding partition (0, ~) of CUd'
This general formulation is free of the dimension d, and allows enough flexibility to treat a
wide variety of specific situations [see Section C.2].
The sample-based estimate of
e will
boundaries, with generic element T.
corresponding partition (T, T) of CUd'
I c:r I :=#{T E c:r}.
be selected from a finite collection
c:r of candidate
Again, each candidate T is identified with its
The total number of candidates considered is
The collection
regularity conditions:
~
~
must be explicitly specified by the user in accordance with certain
must be rich enough to contain candidate boundaries that are close to
the true 6; the cardinality
I I I;
I~ I
must nevertheless be controlled in terms of the sample size
and, the candidate boundaries, as well as the true boundary, must be sufficiently
"smooth." These regularity conditions are intuitively natural and technically manageable [see
Sections CAl, but they do require the user to have some prior knowledge about the form of
e
[see Section C.2].
In order to generate a rich collection
~
with controlled cardinality, we consider
candidate boundaries which are "anchored" to the grid nodes in I [see Section C.2].
It is
natural to so anchor the T's, because in practice one cannot hope to get better resolution from
an estimated boundary than whatever degree of resolution is available from the data-nodes I.
Because of our regularity restrictions, we can handle individual examples in which the
set
e is non-convex [Example 2.c, Section C.2] or disconnected [Example l.b, Section C.2]; cf.
Ripley & Rasson (1977), who assume convexity and compactness but make no other regularity
restrictions.
Even though the X/s are independent, the "spatial" indexing of the observations is
nevertheless crucial in our formulation of boundary estimation. The physical lay-out of the
d-dimensional grid is precisely what leads us to consider particular types of boundaries (e.g.,
change-point boundaries [Example l.a, Section C.2] in the case d=1; linear-bisection
boundaries [Example 2.a, Section C.2] in the case d=2). If we throwaway the spatial indexing
of the observations, then we are faced with a clustering problem.
-15-
FIGURE Q.l
The 2-Dimensional Case: CU 2 with n 1 =10, n 2 =20, 111=200
iE 1
!
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
• •
• •
• e.
• •
• •
• •
• •
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Xi - F, iE
6
•
-e
•
•
d• •
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
'•.
•
•
~.
•
•
•
C.l.(c) Outline of this Chapter.
•
•
•
•
•
Xi-G, i E ~
The notions discussed in this section will be
illustrated by specific examples in Section C.2. The proposed boundary estimator is described
in Section C.3; theoretical properties of our estimator are presented in Section CA. In Section
C.5, we explicitly apply our method -- and the theoretical results -- to the examples from
Section C.2.
C.2
EXAMPLES
The I-Dimensional Case.
In the case d=I, it is natural to think of CUd as a "time"
axis; then I indexes observations at equally spaced intervals of time.
Example 1.a: The Change-Point Problem. The boundary
number () E (0, 1), inducing the partition:
e is simply an arbitrary real
6:=[0, (}), ~:=[(), 1].
Estimation of the
to
-16-
"change-point"
e
has been extensively studied in the literature [see Section B.l]; most of this
other work assumes either parametric knowledge of F and G, or assumes that F and G differ in
a known way (e.g., by a shift in "level").
In the quality-control setting,
e
demarcates a
change from an "in-control" production process to an "out-of-control" production process.
Since we make no assumptions about G, our method will identify the onset of any type of
disorder in the distribution of the output.
The candidate boundaries T E GJ are essentially all the times at which observations
were made, Le., all t E {2/n1' 3/n1' ..., ndnl}' with 1':=[0, t), T:=[t, 1]. These candidates are
anchored in I, yielding a collection GJ that is appropriately rich but whose cardinality is the
same order as the sample size.
Example l.b:
The Epidemic-Change Model.
The boundary
e
consists of two points
{Ol' 02}' with 0< 01 < 02 < 1, inducing the partition: 0:=[0, 01) U [02' 1], !2:=[01' 82), In this
model, 01 represents a change to the epidemic distribution G, and 82 represents a return to the
pre-epidemic distribution F.
Again, previous work on the epidemic-change model requires
parametric knowledge of F and G, and/or assumes a shift in "level" [see Section B.2.(a)]. Note
that
e is not a connected set in this example.
The candidate boundaries T E GJ are all pairs {t l , t 2 } anchored in I, with:
t 1 E {l/n1' 2/n 1 , ...,
t 1 < t 2,
...,
T:=[t 1 , t 2 )·
The 2-Dimensional Case. In the case d=2, it is natural to think of 'lid as a geographic
area; then I indexes observations which are regularly spaced in the East-West direction and
which are also regularly spaced in the North-South direction.
Consider the' following application from forestry:
The observations {Xi: i E I}
represent heights of trees, where F is the distribution for a healthy stand and G is the
distribution for a diseased stand. If the disease kills very young and very old trees, then F and
-17-
G may share the same "level," but may differ in terms of "dispersion." The point here is that
our estimator will identify the boundary between "healthy" and "diseased" without any prior
knowledge regarding the effect of the disease on the distribution of heights.
If the disease
spreads radially through the population, then the boundaries considered should be circular or
elliptical templates [see Example 2.b, Section C.2].
Marine scientists often rely on voyages of commercial vessels in order to obtain data.
When a trans-Atlantic voyage is being made, it is relatively inexpensive to record observations
at a large number (nl) of closely-spaced intervals in the East-West direction. But it is very
expensive to extend the grid in the North-South direction (i.e., to increase n2)' since this entails
a whole new trans-Atlantic voyage.
So it is important in practice to allow for different
sampling designs in the different dimensions.
Our method does allow the n/s to differ, and
our theoretical analysis [Section C.5] shows how the n/s affect the rate of convergence for our
estimator.
In image-analysis, the data-set may arise from a pixellated binary classification of the
true but unknown shape
0
against the background~. In this formulation, F and G are two
Bernoulli distributions with distinct parameter values (e.g., with PF:::::: 1 and Pa:::::: 0).
Example 2.a: Linear Bisection. The boundary
e is an arbitrary straight line-segment
connecting endpoints on two distinct edges of CU 2; this boundary induces a partition (0, ~) in
an obvious way.
The candidate boundaries T E If are all straight line-segments connecting endpoints on
two distinct edges of CU 2 -- but the endpoints must be anchored to the grid: An endpoint on
an East-West edge must have coordinate i 1/n 1 , i1 E {2, 3, ..., n1 -1}, while an endpoint on a
North-South edge must have coordinate i 2 /n 2 , i2 E {2, 3, ..., n2 -1}.
Example 2.b: Templates. A template is a boundary in CU 2 which can be perturbed via
a finite number of "parameters;" these "parameters" may allow for translation, rotation,
-18-
.
elongation, etc. Circles, ellipses, and polygons can be handled as templates.
Consider, for example, an arbitrary rectangular template
e whose edges are parallel to
the edges of 'U2' and whose vertices are all in (0, 1)2. The interior of the rectangle is ~, and
the remainder of 'U2 is
e.
Note that
e is not a convex set.
The candidate boundaries T E <:r are all rectangles whose edges are parallel to the edges
of 'U2' and whose vertices are anchored to grid nodes: The East-West coordinate of a vertex
must be in {I/n}, 2/n}, ..., (ncI)/nd, and the North-South coordinate must be in {I/n2'
2/n2' ..., (n2-I)/n2}. The region
I is the interior of the rectangle together with its edges.
Example 2.c: Lipschitz Boundaries. To define a Lipschitz boundary, we identify the
lower edge of 'U2 as the z-axis, and the left-hand edge of 'U2 as the y-axis.
boundary
e
is the curve in 'U2 corresponding to a function Ye('): [0, 1]...... (0, 1) satisfying:
I Ye(z)-Ye(z') I $
Co I z-z' I 't/ z,z' E [0, 1]. The constant Co simply controls the slope of the
boundary. The points (Y, z) E 'U2 with y > Ye(z) comprise the region
is
A Lipschitz
e; the remainder of 'U2
~.
The class of Lipschitz boundaries is extremely rich. Note that these boundaries are not
readily expressible as templates, and that
e and ~ need not be convex sets.
The candidate boundaries T E <:r correspond to piecewise linear functions y -z{ . ):
[0, 1]......(0, 1) which are anchored to the grid in the following way: For each z E {Oint, I/n},
..., nt/nIl, the associated value of y-z{z) is in {I/n2' 2/n 2, ..., (n 2-I)/n 2}; at intermediate
values of z, the function y -z{z) is defined by linear interpolation.
boundaries T for which
I y-z{z)-y-z{z-I;}) I
$ c/n l
We restrict <:r to those
't/z E {I/n l , 2/n l ,
...,
nt/nd, where
c:=3co+ 1. This Lipschitz-type restriction controls the slope of the candidate boundaries. For
this Example, it is convenient to have n 2 2: n l , The region if is defined analogously to
Higher Dimensional (d
e.
> 3) Case. Boundary estimation in the case d=3 has natural
applications to meteorology and geology. Planar bisection of 'U3 can be handled analogously
-19-
to Example 2.a, in this Section.
In geology, the cost of extending the grid in the "depth"
direction may again necessitate a sampling design with differing n/s. We will discuss here the
case of an oriented hypercube, which is a template in CUd'
Example!l.:. Oriented Hypercuboid. We define a relationship"
>" in CUd: For a pair
> b" if
'"
The boundary
e
is defined by a pair of vectors f!:. and! E CUd' such that!
> f!:..
These vectors induce as one of the regions, the set e~,!]:={YECUd: aj$. Yj$. bj , 'r/
1 $. j $. d}, where Y :=(Yl' Y2'
'"
and the vector
L
:= (/1,12"
•••
'Yd)' The vector f!:. defines the "origin" of the hypercuboid
•. ,Id ) defined by Ij=bj-aj' the lengths of the edges of the
hypercuboid. It can be easily seen that a vector of lengths defined in this fashion belongs to
CUd' The region
!2 is the hypercuboid
e~ '!] and
e comprises the complement of e~ ,!].
The candidate boundaries T E GJ are the hypercuboids e[~, '!.] where
JL > ~
E CUd'
~''!. EI,andO<zj'Yj<l,'r/l$.j$.d.
I
C.3. THE BOUNDARY ESTIMATOR
C.3.(a) The Basic Idea. Our main statistical tool for selecting an estimate
e from GJ
is the empirical cumulative distribution function {e.c.d./.}.
For a candidate boundary T E GJ,
compute the e.c.d.f. h~x):= L id O{X i $.
treats all observations from region
x}1 I if I, which
if as if they were identically distributed; similarly compute
which treats all observations from region
e.c.d.f.
is
actually
a
sample
[ I if n e I F(x)+ I if n!21 G(x)]1 I if I,
[ IT n e I F(x)+ IT n!21 G(x)]1 IT I.
T as
of
the
the
unknown
mixture
latter
e.c.d.f.
analogously
distribution
~
estimates
Therefore, the difference between the two e.c.d.f. 's can
be approximated as follows:
I hr'x)-hr'x) I
x}1 I T I,
if they were identically distributed. The former
estimate
while
h~x):= L ilI O{X i $.
I[I if n e I I 1if I]-[ IT n e I I IT I] I . I F(x)-G(x) I·
-20-
Notice that this last expression can never exceed I F(x)-G(x) I (because (1' n 6) ~ l'
and (I n 6) ~ I)j moreover, its maximising value
T=€>.
I F(x)-G(x) I
This suggests a natural approach for estimating 0:
is attained precisely when
Choose as your estimator the
candidate boundary which maximises Iii:z{x)-b:z{x) lover all T E~.
This basic idea will
now be refined and generalised.
C.3.(b) Definition of the Boundary Estimator.
Rather than restricting attention to
the difference Iii:z{ . )-b:z{ . ) I at a single specified x-value, we instead consider the differences
d[:= Iii:z{x;)-b:z{X;) I for each i E I. This allows the data to lead us toward the informative
x-values, without prior knowledge of F and G.
Now combine these differences dT using a general norming function S(d[, dr, ..., dfr')'
The norm S(·) must satisfy certain simple conditions [see Section CAl, but special cases
include:
the Kolmogorov-Smirnov norm Sl\s(d l ,d 2 , ... ,d N ):= sup {d;}j
1 SiS N
N
the Cramer-von Mises norm Scv(d l ,d 2 , ... ,d N ):=(L: dUN)I/2 j
;=1
N
the arithmetic-mean norm S am( d}, d 2 , •••, dN):= L: d;/N.
; =1
Finally, we must standardise to account for the inherent instability in the e.c.d.f.
Suppose ii T ( .) (say) is based on a very small amount of data (i.e.,
°
relative to I II)j then
d:
o
is unstable. Thus
d:
o
I To I
is very small
may be "large" (as compared to other dT's)
merely due to random variability. We should down weight this particular candidate To in our
search through
~
factor ITo I/ I I
for a maximiser. This downweighting is accomplished by the multiplicative
I.
e is defined as the candidate boundary in ~ which maximises
the criterion function D(T):=( I l' I / I I I)( I I I / I I I). S(dr, dr, ..., d fr') over all T E ~.
Formally, e:=argmax D(T). Observe that e is calculated solely from the data at hand.
Theoretical properties of e are presented in Section CA.
The boundary estimator
Tt:~
-21-
C.4. TOWARDS CONSISTENCY PROPERTIES OF THE BOUNDARY ESTIMATOR
C.4.(a) Measuring the "Distance" Between Boundaries.
performance of our boundary estimator
e,
In order to assess the
we must first quantify the notion of "distance"
between two boundaries (say, T and a). Our "distance" measure is a(T, a):=minp(f 0 e),
'\(1 0
en,
where
o
denotes
set- theoretic
symmetric
(AoB):=(AnBC)U(ACnB), and ,\(.) is Lebesgue measure over CUd'
difference,
i.e.,
Intuitively, '\(foe)
represents the "area" that is misclassified by f as an estimator of e. For a given boundary T,
the a priori labeling of the two induced regions as (f, I) [rather than (I, f)] can be arbitrary
[e.g., Example 2.a]. Therefore, a candidate boundary T is considered "close" to
a
if either f
or I is nearly the same region of CUd as e.
The function 0(· , .) has the following desirable properties of a "distance."
Proposition 1: The function 0(· , .) is a pseudometrie. That is, it satisfies:
(La)
[Non-negativity] a(T, 8)
(1.6)
[Identity] a(8, 8)=0.
~
O.
•
(l.e) [Symmetry] a(T, 8)=8(8, T).
(l.tI) [Triangle Inequality] a(T, 8) ~ a(T, T')+a(T', 8).
Properties l.a, l.b, and l.c are obvious from the definition; a proof of property l.d is in
Chapter F.
When we discuss "consistency" and probability of "error" for
e as an estimator of a,
it will always be in the sense of a-distance.
C.4.(b) The Norm S( . ).
There are some constraints on the choice of S( .).
The
following conditions are intuitively reasonable, and they enable us to simultaneously handle a
whole class of boundary estimators
e.
Definition: A (unction S( .): R~.....R~ is a mean-dominant norm if it satisfies:
(!.a) [Symmetry] S(·) is symmetric in its N arguments.
-22-
(I.d) (Identity] S(I, 1, •••,1)=1.
(I.e) (Monotonicity] S(d1 , d 2 ,
..., d N ) ~ S(d~, d~,
(1./) [Mean Dominance] S(d1 , d 2 ,
...,
dN)
~
..., dN) whenever d i ~ d~ Vi.
N
E
diJN.
i=l
It is straightforward to check that:
Proposition B.i The functions SKi·), So,,(·), and S.m( .) are mean-dominant norms.
Although
C.4.(c) Regularity Conditions on Boundaries.
our
results
require
no
distributional assumptions, they do assume certain set-theoretic regularity conditions on the
boundaries.
It will be seen that these regularity conditions are intuitively natural, and that
they are simple to check in specific applications [Section C.5].
In order to study the asymptotic properties of our method, we will let the number of
grid-nodes increase:
I I I...... 00.
Since the candidate boundaries and the estimator depend on
the particular grid, we henceforth equip er and
Similarly, the number of observations in A
~
e with
'lid is now
explicit subscripts:
er I and
e
l.
I A II'
Beaularity Condition fJ1Jl;. [Non-trivial Partitions]
For each T E ~ I' 0 < A(T)
< 1 and 0 < I T I II I I I < 1. Also, 0 < A(e) < 1.
Condition R.l prohibits consideration of trivial partitions. It is permissible to consider
candidate boundaries with A(1') arbitrarily small as
I 11 ...... 00
1.a, Section C.2); such candidates generally do not influence
(e.g., 1'=[0, 2/n 1 ) in Example
e
I'
due to the downweighting
factor in D(T).
Beaulant, Condition
l.JJul1l
[Richness of c:r I]
3 c52~O and a sequence {T I } withT/EerIVI, such that:
6
III2·a(e,TI )-+O as
Condition R.2 requires er I to contain some candidate boundary T I that "'gets close" to
-23-
the true
e (with rate I I
1
62
).
If no such "ideal" candidate was available, we could not possibly
hope to statistically select an estimator e! from GJ! in such a way that consistency holds. It is
easy to satisfy R.2 when the candidate boundaries are anchored to the grid in a natural way
[see Sections C.2 & C.5].
Regularit, Condition
~
[Cardinality of c:r
3 63 ~ 0, such that for each 1
n
> 0, I c:r11 • exp{ -1' I I I
Condition R.3 quantifies the balance between
II I
1-26
3} -
and
I GJ 11.
°
I 11-+00.
as
Basically, the number
of candidate boundaries must be substantially smaller than an exponential of the sample size.
This constraint still allows for extremely rich collections GJ! [see Section C.5], including but not
limited to GJ/s with cardinality
I I I k,
for some finite and fixed k. Note that:
Propo3ition J.;.
Condition R.3 is satisfied for all 63 E [0,
t) whenever I c:r 11
I I I ~, v > O.
is of the order
In order to discuss the final regularity condition, we need the notion of cells in 'U d •
Recall that the grid (described in Section C.l) induces a "rectangular" partition of 'U d . A cell
is simply one of these d-din.ensional "rectangular" regions, including its edges and vertices [see
Figure C.I]. Thus there are
III
cells in 'U d , and they are not strictly disjoint. A generic cell
is denoted C, and the collection of all cells in 'U d is denoted e!. For an arbitrary set A
the collection of perimeter cells of A is defined as:
C n AC
:/;
¢J}. So,
~ J<A)
~J<A):={CEe!:
CnA:/;¢J
~
'Ud ,
and
consists of those cells which intersect with both A and A c, i.e., cells
which are on the "perimeter" of A.
Regularity Condition LJ1dl;. [Smoothness of Perimeter] Denote tl'I:={e,
3 64 ~ 0, such that
I II
6
4. supAEtl'1
A(c:PJ<A» -
°
as
T: T E c:rI}'
111-+00·
Condition RA guarantees "smoothness" of the boundaries, relative to the grid: As the
grid mesh becomes finer, the cell-wise approximation to the boundary (i.e.,
~I<'))
must shrink
in "area." This prohibits boundaries which wander through too many cells in 'Ud [see Example
-24-
I.c in Section C.5].
It is trivial to directly check R.4 in the I-dimensional case [see Section C.5]. In the
2-dimensional case, we can actually reduce R.4 to a simple calculation of the lengths of
boundaries. Consider a boundary in 'U 2 that is expressible as a rectifiable curve r(t), i.e., r( . )
is a continuous function from [a, b] ~ R} into 'U 2 ' with coordinates r(t)=(r}(t), r 2 (t»,
satisfying:
k
L(r):=
sup
a=t
o < t} < ... < t k=b
i
L=}II r(tj)-r(t i _}) II
<
00.
The quantity L(r) is just the length of the boundary. The following relationship holds between
the perimeter cells and the length:
Theorem 1: Let d=2. IT the set A E a-I corresponds to a boundary expressible as a rectifiable
C1lMle
r( . ), then ~(c:PJ<A»
5 18· (L(r)+I)/min{n 1, n 2 }.
Proof of this result is in Chapter F. Now it is easy to check R.4 by approximating the lengths
of boundaries [see Section C.5].
C.4.(d) Asymptotic Results.
We assume that:
F::f G;
e/ is
based on a mean-
dominant norm.
Define the following combined Regularity Conditions:
R(6) means: R.I, R.2, R.3, and R.4 hold, and 0 $ 6 $ min{6 2 , 63 , 64 }.
~(6)
means: R.I, R.3, and R.4 hold, and 0 $ 6 $ min{63 , 64 }.
~(6)
means: R.I, R.2, and R.4 hold, and 0 $ 6 $ min{6 2 , 64 }.
~(6)
means: R.I and R.4 hold, and 0 $ 6 $ 64 ,
a':=min{'\(e), '\(0)} and
00
p:=J£F'\(e)+J£G'\(~)'where J£p= f
I F(x)-G(x) I dF(x),
-00
00
J£G:= f
-00
The following relationships can be easily established:
R(6)
=>
~(6)
and ~(6), '</ 6 ~ 0;
R(6) => R(6'), & RI;(6) => RI;(6'), whenever 0 $ 6' $ 8, k E {2,3}.
-25-
I F(x)-G(x) I dG(x)
The main theoretical results of this chapter are:
TAeorem I.;, [Strong Consistency]
If R(5) holds, then I 11 6 • ace,
e
I ) -+
0 as
111-+00, with probability 1.
Theorem J!.;, [Bound on Error Probability]
If ~(O) holds, then for any e
> 0,
.....
2
2
2
p{a(e,eI»e}::510·Ic:rII·exp{-K'CT·p·e •
111
~ No(e), for some function
N.
I Ill,
for
III
sufficiently large so that
Further, K is a universal constant, and CT & p are
constants for the problem.
The "rate" of convergence obtained in Theorem 2 depends upon 6 ~ O. Constraints on
6 will follow from the regularity conditions discussed above.
In Section C.5 we will see the
actual rates that can be obtained in particular applications.
The general theoretical result which paves the way for Theorems 2 & 3 is Theorem 4
a generalisation of Theorem 3.
Theorem .Jl
If R3 ( 5) holds, then for any e
> 0,
for
sufficiently large so that
II I
~
NI(e), for some function N I .
III
Further, K is a universal
constant, and CT & p are constants for the problem.
The proofs of Theorems 2, 3, and 4 are in Chapter F.
Theorem 3 says that the probability of error decreases exponentially as a function of
sample size, but that this effect is counterbalanced by the number of candidate boundaries
considered.
(R.3).
The precise nature of this trade-off between
II I
and
I GJ II
is discussed above
From Theorem 4 we can understand the effects of the underlying constants of the
-26-
problem,
°°°
2 , 3 , 4 , IT,
Jl. The rates 62 and
°
4
error probability bound by requiring
II I
°°
°°
2, 4 ,
However R.3 has to hold with
When the constants
IT
arising from conditions R.2 and R.4 influence the
to be larger than some integer which is dependent on
~ 3
for strong consistency (Theorem 2).
and Jl are small the speed of convergence is slower. A low Jl
suggests that the underlying distribution functions F and G are more alike and hence the
estimator
e is less efficient in telling the regions e and ~ apart.
When
IT
is small we have a
partition for which one region is small in terms of Lebesgue measure and data points indexed
by nodes in that region are few in number. This makes error due to sample variability more
likely.
The value of the constant
J(
is influenced by the mechanics of the proof.
interesting to note that the bound depends only on
I I I, I GJ II , e,
6,
IT
It is
and Jl and that the
effect of each one of these factors can be isolated.
C.S. EXAMPLES REVISITED
In this Section we explicitly apply the theoretical results of Section 4 to the Examples
from SectionC.2. In each case a class of strongly consistent boundary estimators is obtained.
Note that R.l is satisfied [by construction] in each of Examples La, 1.b, 2.a, 2.b, 2.c, and 3;
this regularity condition will not be further discussed.
The I-Dimensional Case.
Example l.a:
where
LzJ
The Change-Point Problem. In R.2, take T I to be the point LOnd/nl'
z.
denotes the largest integer less than
holds for any
°<
2
1. For R.3, we observe that
R.3. holds for any 03<~'
Since 8(8, T I )
I GJ II < I I I; hence
~
1/n 1 and
R.2
Proposition 3 applies, and
For each A E if I' there is only one "perimeter cell."
A(c:PJ<A))=1/n 1 , and R.4 is satisfied for any 64
Therefore
°
< 1. This establishes R(o) for any E [O,~), so
that Theorems 2, 3, and 4 apply for the class of change-point estimators
Example1.b:
I II =n 1 ,
The Epidemic-Change Modd.
-27-
el'
Take T I to be the pair
{(L 01nd+ 1)/n 1 ,
L92nd/nl}' so that 1'r is an "inner approximation" to §. Again
I II =nl'
so R.2 holds for any
°<
2
o(e, T r)
is of order 1/n1 and
1. Next we can apply Proposition 3, because
I ~rl < 1/1 2,
so that R.3. holds for any 03<~' For A E fi'r there are now two "perimeter cells," yielding
A(c:Pr(A))=2/n 1 and satisfying RA for any
°<
4
1.
Thus R(o) holds for any
°E
[O,~), and
Theorems 2, 3, and 4 apply: We have convergence of our estimators for the epidemic-change
model.
Example1.c:
Rationals
~
Irrationals. We have heavily emphasised the set-theoretic
nature of our approach -- in particular, we have exploited the partition (6, §) of CUd' The sets
6:={ rational numbers in [0, In and §:={ irrational numbers in [0, In constitute a perfectly
legitimate partition of CUI' Yet we would be surprised if the proposed method applied in this
Indeed, an obvious problem arises with the "smoothness" of the perimeter (RA):
situation.
every cell is in c:P r(6), so A(c:P r(6))
== 1 and RA is violated.
The 2-Dimensional Case.
Example 2.:.f!;. Linear Bisection. To check R.2, consider the following special case: e
connects an endpoint on the lower edge of 9.J. 2 to an endpoint on the left edge of CU 2; the
coordinate on the lower edge is 91 E (0, 1), and the coordinate on the left edge is
(J2
E (0, 1].
Take T I to be an analogously oriented segment, with lower edge coordinate L91n 1J/n 1, and left
edge coordinate L9 2n 2J/n 2. Then o(e,Tr)=t(9182-L81ndL92n2J/n2n1) $ l/min{n 1, n 2}, and
hence R.2 is satisfied whenever
(*)
(max{n 1 , n2})02(min{n 1, n2})0 2-1
----
O.
Other configurations of e similarly yield (*) as a sufficient condition for R.2. Since
I ~rl < 1/1 2,
we can handle R.3 via Proposition 3.
Now observe that L(r)
$..J2
for any
A E fi'r' so by using Theorem 1 we find that RA is satisfied whenever (*) holds, with
place of
° in
4
°
2,
Condition (*) forces the grid design to asymptotically become finer in both
-28-
dimensions.
0< ex ~
t.
Consider in particular min{n l , n2}= III a and max{n l , n2}=
III I-a,
where
Then (*) holds for any 62 E [0, ex), and R(6) holds for any 6 E [O,ex), so that
Theorems 2, 3, and 4 apply to our estimators of a linear bisection boundary.
Example 2.J!.;.
Consider the rectangular template 0 discussed in Section
Templates.
C.2, Example 2(b). Take T I to be the candidate boundary corresponding to the largest :r
~ ~.
Then 8(0, T I) ~ '\(~)-'\(:rl) ~ 4/min{n l , n2}, so that R.2 is satisfied whenever (*) holds.
Since
I ~II < 111 2 ,
we can again use Proposition 3 to deal with R.3. Note that L(r) ~ 4 for
all A E ~I' so that Theorem 1 reduces R.4 to condition (*), with 64 in place of 62,
analysis of the case min{n l , n2}= III a and max{n l , n 2}= I Ill-a, 0 < ex ~
The
t, is exactly as in
Example 2.a.
Example f.c:
piecewise
linear
YTiz):=ma~i2/n2:
For R.2, consider T I corresponding to the
Lipschitz Boundaries.
function
YT ( .)
I
defined
at
each
i2 E {I, 2, ..., n 2 -I} and i 2/n2
Z
E {O/n l ,
~ Ye(z')
I/n I , ...,
ndnI}
by
Vz' E [z-i , z+ill}. Note that
I
TIE ~ [l because for each z E {I/n l , 2/n l , ..., ndnl} we have:
Since y T ( . ) is dominated by Ye( . ), we can write (for
I
8(0, T I)= "'"
LJ
I JI
sufficiently large)
k
nl
(k-l)
k=l Ii}"
Jn l (Ye(z ,)-YT/ Z "»
dz,
with each integrand bounded above by the r.h.s. of the preceding inequality. Therefore R.2 is
satisfied whenever
(t)
For R.3, note that
I ~ II
~ (n 2)n l , so Proposition 3 does not directly apply. In this
situation, R.3 reduces to
(tt)
Lastly, we use Theorem 1 to handle R.4. Each A E ~I has L(r) ~ ~c2+I, because for
-29-
e we find:
k
L
k
II r(tj)-r(t j_ l ) II =
i=l
L ~ I Ye(tj)-Ye(t i _
l) 1
2+ 1t j-t _ 1 2 :5 ~c2+1
i 1
i=l
whenever O=t o < t l
< ... < t k =l,
and for if we find:
Thus RA reduces to (t), with 64 in place of 62,
Consider the case n l =
I
II
Q
and n 2=
1
Il l -
Q
,
where 0 < Q :5!.
Condition (t) is
satisfied for all values of 62 E [0, Q) and condition (tt) holds for any 63E [0, l'2 Q ). Thus R( 6)
holds for any 6 E [0, min{ Q, l'2 Q } ) , so that Theorems 2, 3, and 4 apply to our estimators of
Lipschitz boundaries. Note that the best rate of convergence
I 11 6
is obtained from the non-
symmetric grid design with Q=t.
Higher Dimensional (d
~
3) case.
Example!t. Oriented hypercuboid. To establish the regularity conditions, we will need
the following results which are proved in Chapter F:
d
Proposition
d
d
d
J1 0 1=1
< II f·I -< [i=1
II z· - II (Z'-f')] < E f· :5 d· m!J%£I', where 0:5 f l':5 Z,':5 1.
' i=1 I
I
-i=1 I
•
Proposition 5: Consider vectors .!!.'
!' £' !-,
.!! +,
! + in CUd'
where the i th component of
.!!. is denoted by ai' etc. Assume that Vi:
a7"
< a·1<- a.+
<
b7"
< b·1<- b:+
and
1I
,I
min{(a.a7")
(a.+ -a.)
(b.-b7")
(b:+
-b.)}
> 1.
I
"I
.'
I
I'
I
, - Ri
For R.2, if 0=ef.e
hypercuboid e~I,
I
xi:
,.e], consider the candidate boundary T I corresponding to the
l] where i
[n.a.]
• ~.
&
+1
and
!f!.
are defined component-wise by:
I
[n.b.]
,and Yi :=~ , where [p]=largest integer not exceeding p.
&
By definition aj < bj Vi. Notice that for ni sufficiently large, aj:S x{
-30-
:s y{ :s bi , Vi,
and hence
1'I=e[i, yI] ~ e~ '! ]=~. Therefore,
'"
d
d
8(6, T 1) $ ..\(~ 0 l' 1)=..\(~)-..\(TI)i~ 1 (bj-aj) i ~ 1[( bj-a;) - (bj-Y; + x; -aj)]
< d· max(b
.-y!, + x!, -a,-) < 2d/[min
n.J'
j
,
ii
-
the second-to-last inequality following from Proposition 4. Therefore R.2 holds if,
Ott)
6 -1 d
6
ncr); ~ 2 n(}) -+ 0, where n(1) $ n(2) $ ... $ ned)' are ordered n/s.
For R.3, notice
I GJ I I $ I 11 2
and by Proposition 3, R.3 is satisfied for 63 E [O,~),
independent of the dimension d.
Given a hypercuboid era,
b], define the vectors Na", Nb-, Na+, Nb+in CUd' component-wise
~
N
by:
ai:=
[n·a·]-1
'n'.
I
,
+
ai
[n·a·] + 2
'~.
,
:=
,
bi:=
[n·b·]-1
I
n'j
._[n j bj ]+2
, b/.-
nj
'V 1 $ i $ d.
By construction it can be seen that:
'Vi , a7"& -< a·I -< a·+
"
b7"
< b·, <
b·+
and min{(a.-a7")
(a.+
-a)
(b.-b7")
(b.+
-b.)}
I ,
'
•
I'
I
I'
I
I'
I
I
Further, since aj < bj 'Vi, ultimately
a/
n.
>.1..
-
$ bj- . It will follow from Proposition 5 that
c:PI(e~,.eD ~ e~-, .e+] - e~+,
,e-].
Therefore, '\(~I(e~, .e])}; ..\(e~-, .e+] ) - ,\(e~+,,e-D
=ft
j=1
(b.+.a7")
I
I
-ft
d
(b7"-a.+)=ft (b.+.a7") -II (b'+
,
i=1
I
,
j=1'
j
=1
'
6)
• a·I- . nj
-
$ 6d/[m.in nj], by Proposition 4.
t
Therefore, RA is satisfied whenever condition (ttt) holds, with 64 in place of 62 ,
Note that for condition Ott) to hold, it is necessary that the grid mesh get finer in all
dimensions.
If the design is such that, 'Vi, nj= I I I C\ 0 < aj < 1,
d
L aj=1,
i =1
condition (ttt)
holds for 62 E [0, mjnaj) and R(6) holds for 6 E [0, mjnaj)' It follows that the best 6 is reached
1
1
when we can afford the symmetric grid design, i.e., aj = ~ for all i. Also, if ni=log I I
I
for
some i, then 62 =0.
Examples (l.b) and (2.b) are special cases of the oriented hypercuboid set-up, when the
dimension d is 1 and 2 respectively. The following is a comparision of the best rates obtained
-31-
in these cases:
d=l
(Example l.b) 6<12 (Controlled by condition R.3) (R.2 and R.4 yield 62 < 1)
d=2
(Example 2.b) 6<12 (Controlled by R.3 and R.2 & R.4)
d=3
(Example 3)
6<13 (Controlled by R.2 & R.4) (R.3 yields 63 < ~)
d>3
(Example 3)
6<1
(Controlled by R.2 & R.4) (R.3 yields 63 < ~)
d
In discussing the best rates of convergence,
III
lower dimensions cardinality
considerations pre-dominate while in higher dimensions volume restrictions are overriding.
-32-
D. ROBUSTNESS OF THE BOUNDARY ESTIMATOR
D.O. INTRODUCTION
Knowledge of the form of the boundary
assumed in the formulation of
e/.
e (rectangle,
Lipschitz function, etc.) has been
This information is used to construct "rich enough" classes
lff/ (see Condition R.2), so that boundaries that are "close enough" to the true boundary
e are
e defines a circle, then lff[
is available on the location of e
available as possible estimates. Thus, if the user has knowledge that
is constructed to include only circles. If further information
(say the centre of the circle is known to be in the left side of CU 2 ), then a smaller set lff[ of
circles can be constructed which will still satisfy R.2. Smaller Ilff ~ means easier computability
and sharper bounds on the error probability (see Theorem 3).
motivation for small lff[.
Therefore there is practical
It is however possible for lff[ to be
"too small" if it violates
condition R.2. This may happen due to overzealousness in reducing the computational burden,
or due to incorrect prior information. Thus, a question of practical interest is: What is the
price paid if the classes lff[ do not satisfy R.2.?
We shall explore the case of imperfect knowledge of the form of
e.
For example,
when d = 1, if we do not know how many change-points (from F to G and vice-versa) there are,
we can incorporate our uncertainty into the model by including in lff[: cases of one changepoint, two change-points (epidemic-change), ... , up to K change-points. By Proposition 3 we
will still satisfy R.3, the condition limiting the cardinality of lff[.
However, we do need to
know K, an upper bound on the number of change-points, in order to satisfy R.2.
As another illustration consider the case where the true
only squares.
Formally, let
e
but lff[ contains
be the actual boundary. The candidate pool is lff[.
e~= arg min 8(0,T), be the element in lff[ closest to 0.
TElff[
e is a circle,
r
Let
Then 8(0 ,0) is the minimal error
that we can expect to make in estimating 0. Thus if the error of our estimator, 0(61'0), is of
the order of 0(0j,0) we shall be reasonably satisfied. Since our intent is to study
61 as
an
estimate of 0, we will look at the asymptotic behaviour of 0(81'0) and' compare it to the
asymptotic behaviour of 0(0[,0).
0[,
From the definition of
necessarily unique: There might well be a class of candidates
0[
it is clear that
0[
is not
that are optimal in terms of
the pseudometric 0, but each of them might yield a different value of 0(6 1'0[). Because of
this possibe ambiguity the study of 0(6 I,0[) is not appropriate here.
. We say that the estimator
61
is still as close to 0 as 0
61 is
r is
"robust" even when <;J I is not sufficiently rich, if our
close to 0.
We shall investigate whether 0(6 1,0) is
1
comparable to 0(0 ,0), even though the latter may be asymptotically positive.
D.1. DEFINITION OF "ASYMPTOTIC ERROR TARGET"
Given 0 and the candidate family {<;J I: 'v'I}, define the "asymptotic error target,
1] n.
lim
min 0(0, T) = lim 0(0, 0[)
I I I - 0 0 T E GJ I
I I I -00
1]:=
We want to study the "robustness" of our estimator
from our assumptions - in particular, R.2.
The quantity
61
1]
with respect to departures
is the natural parameter for
measuring the severity of the failure of R.2, as can be seen by the next proposition.
Proposition.2;, Condition R.2. holds
¢}
11=0.
This result also justifies the use of lim rather than lim, since the latter could yield zero
even if R.2 fails and hence does not adequately quantify the departure from our assumptions.
Given that R.2 fails (by the amount 17), we can assess the robustness of
0(6 1 ,0) to the target
61
by comparing
1].
We shall now define the function p, which plays an important role in the analysis:
p(T):= I A(T n 6)A(I)-A(I n 6)A(T)
-34-
I.
When the problem was analysed with TJ > 0, it was found that the function [p(e) - p(T)] is
central to the proofs. The function [p(e) - p(T)] behaves similarly to 8(e, T), and Lemma 5
formalises their relationship.
Furthermore when the partition is "balanced", i.e.,
u=!,
8(e,T)=2· [ p(e)-p(T)].
Lemma 5: For 1
> 0,
o(e, T) < 1 => p(e)-p(T) < 1, and 8(e, T) > 1 => p(e)-p(T) > u· 1> 0,
where u=min[l(S), l(e)], is a constant for the problem. It follows that:
[p(e)-p(T)] :5 8(e, T) :5 u -1. [p(e)-p(T)].
For example, when condition R.2. holds, by Lemma 5, it will follow that
min [p(e) - p(T)] = 0
lim
I I 1.....00 T E GJ I
However, unlike 8(e,T), [p(e) - p(T)] is not a pseudometric. In fact it is not even symmetric
in its arguments and has little intuitive appeal as a measure of error. Nevertheless, to further
analyse the robustness of
eI , we will define the analogue of TJ, for the function [p(e) -
TJ*:= lim
I I I -00
p(T)].
min [p(e) - p(T)] = lim [p(e) - p(e")]
GJ I
I I I -00
I
T E
Since TJ is the lim of the non-negative sequence 8(e, ei), it is the largest accumulation
point of the sequence. Therefore TJ is a conservative quantification of the limiting "distance" of
the best candidate (in the pool) from the unknown boundary e. Since
e E GJ
I
[!
by definition
e
of eI, 8(e, I) ~ 8(e, ei) and hence,
II
lim
' .....00
8(e,e I ) ~
TJ.
This tells us that TJ is a lower bound on the asymptotic error of
no better with
e than TJ.
I
e
I'
In other words we can do
The more interesting question, which will be answered, is : Can we
do as well as or close to TJ?
D.2. ROBUSTNESS OF
eI
The asymptotic properties of
e were developed in Chapter C, assuming that:
I
-35-
F::j:. G;
e is based on a mean-dominant norm; and that the boundaries satisfy regularity conditions
1
R.I-R.4 (described in Chapter C).
In this chapter we will still assume F::j:. G and mean-
dominance, but the regularity assumptions are relaxed to just:
By Propositon 6, the definition of
1]
R.l holds; R.3 holds with
generalises condition R.2.
Thus the robustness
analysis applies to a broad selection of cases. The robustness results involving
1]
in this chapter
will be generalisations of results in chapter C all of which assume R.2.
Proposition 1;. 0 ~ fl· ~ " ~
(T -
1 "..
Furthermore, all the bounds in the inequalities are
attainable.
This result quantifies the relationship between
tell us that
R.2 holds
¢>
7] = 0
¢>
7]* = O.
1]
and 7]*. Propositions 6 and 7 together
All three inequalities in Proposition 7 become
equalities when 7]=0. Also, the fact that we always have u :5 ! implies that the upper bound
on 7] is at least 27]*. When the partition defined by 6 is "balanced", i.e., .-\(6) = .-\(0) =!' the
third inequality becomes an equality, since by definition 8(6, T)=2· [p(6)-p(T)], in this case.
Theorem i;, [Robustness] Assume: ~(O). Then,
(T5)
Further, when ~(e)
= ~{~)=!, (i.e., for a balanced partition),
(T5')
Theorem 5 follows from Theorem 6, which generalises Theorem 3 in Chapter C.
Notice that the left inequalities hold always, while the right inequalities hold a.s.
From the assumption F ::j:. G and R.l., we have that u· J1. > 0 (See Lemma 7, page 67). From
Proposition 7 it follows that 7] :5 [u· J1.]-1 . 7]*. From the numerical bound in (7.3) of Lemma
7, the r.h.s. of (T5) is at least i7]*.
-36-
It is a/ways true that 0 < J.LF + fJe ~ ~ (Lemma 7).
Therefore when the partition is
balanced, J.L ~~, and the r.h.s. bound in Theorem 5, is at least
case:
~
F(x)=B{x
< lim
8( a, aI)
4·17
~
Tll-oo
J.Lp
+ J.Le ~
Consider the following
OJ, and G(x)=Distribution function of CUniform[O,I].
calculations show that J.LF + J.Le=~.
11
j. TJ.
~ -3-.
a.s.
Then direct
In this case, for a balanced partition, we have
Also, from Lemma 7, whenever F and G are both continuous,
1, and the r.h.s. reduces to 2· TJ, for a balanced partition.
Theorem i.;. [Bound on error probability, when 11>0]
H ~(O) holds, then for any 0 < Z < 1, and 0 < e ~ e*(z),
e
> ('1*+e)/CTJ.Lz} 510· 1c:r I I·ezp{-K.e 2 • III} for III sufficiently large,
(T6)
p{8(e,
so that
I I I ~ N 2(e),
l)
for some function H 2• Further, K is a universal constant, and CT It J.L are
constants for the problem.
This gives a bound on the probability that the error exceeds the r.h.s. in Theorem 5, i.e., the
probability of a non-robust
e/.
This theorem is proved in Chapter F.
Notice that the r.h.s. in the theorem is independent of the actual values of TJ and TJ*, so
that the non-robustness probability bound is seen to be unaffected by the severity of the R.2
violation.
-37-
E. SIMULATIONS & NUMERICAL APPLICATIONS
E.n INTRODUCTION
There are four broad categories of numerical applications:
(1)
Development of efficient algorithms for the computation of the boundary estimator, as
a prerequisite for computational studies.
(2)
Simulation studies of finite sample behaviours, to be compared with the asymptotic
theoretical results.
(3)
Numerical computations of the boundary estimator for EPA Cancer data, to illustrate
its applicability in practical situations.
(4)
Development of a bootstrap algorithm for measuring the variability of the boundary
estimator and simulation studies of the proposed bootstrap algorithm.
All computations were done on the Academic Computing Services CONVEX C240
series super computer, "Gibbs", at the University of North Carolina at Chapel Hill.
Fortran
77 was the language of choice. The program codes can be found in the Appendix.
E.! EFFICIENT COMPUTATION
A required basis for applications (2), (3) and (4) is a computationally feasible
algorithm for calculating the boundary estimator. To compute the boundary estimator based
on data {Xj,i E I}, for each candidate boundary T E GJ, we need to compute the criterion
function D(T). The boundary estimator is that candidate which maximises D(T). Thus, the
number of simple computations ell required to calculate the boundary estimator once is,
el
~
I GJ I x #[Computations for calculating one
D(T)]
Thus ells a function of the sample size I J I, and the size of t.he candidate pool
I GJ I.
D(T):=( IT
..., d~/) for all T E~,
I/ I 11)( I I 1/1 II) .S(d[, dr,
where d[:= I ii:r<Xi)-h:r<X i ) I for each i E I,
and
hT , and hT are e.c.d.f.'s based on T and I respectively.
The number of computations for calculating one D(T) seems on first thought to be of the order
of
I I I 2,
since we have to calculate
II I
dr's, and each dr requires us to go through the
I I I observations once, in order to calculate hT ( Xi)
and aT( X i)'
i.e., e/~ I~I x
111 2
Notice that the calculation of the various dr's involves sorting the X;'s within each individual
Thus, for each fixed T, the {I $ i $
candidate region, T and T.
I II}
ii:r<xi)'s
calculated from an ordered subsequence of the ordered sequence X( i)'
I I I}
{I $ i $
estimator
61 is
can be
Similarly, the
h :r<X i )'s can be calculated from the rest of the ordered X( i) 's.
Thus the
a function only of [R(i), D{i E T}: i E l, T E ~l, where R(i) is the rank of Xi in
the sorted sequence, and I{ i E T} is the indicator function that is 1 if the index i is in region
T.
The suggested computational algorithm exploits this fact. We shall denote the number
of computations
using
our
algorithm
X (1) $ X(2) $ ..... $ X ( I I I
-1)
ei.
by
We
first
sort
the
X/s
so
that
$ X (II I ). Since the norm S is assumed to be symmetric,
we can use the ordered sequence instead of the original sequence in the calculations of S. The
rank of X(i) is i, and its original index in the grid is stored in IND(i). Sorting of the X;'s and
computation of the vector IND(i) together take at most 2 x I I 1 2 calculations. Now
function only of [D{IND(i)
ET
}:i E l,T E ~l.
I T I ii:r<x i)
61 is
a
is the number of observations in
Tthat are $ Xi' The various ii:r<xj)'s (and similarly h:r<Xi)'s) are calculated recursively, as
follows.
Let Y i stand for the i th ordered observation.
observation Y 1 comes from
I T I ii:r<Y 1)
is 1 if the the lowest
T, and 0 if it does not. And given I T I ii:r<Y i), I if I ii:r<Y i + 1) is
one more than I T I ii:r<Y i) if Y i + 1 comes from T and the same as
Formally,
-39-
I if I ii:r<Y i)
if it does not.
ITlii:Z{Yd= D{IND(I)eT}
1T IhT(Y j + 1)= I T I ii:z{Y j )
and for ;=1,2" . ·111-1,
Also,
1 T I h:z{Yi)
that while Vi'
[TI,
= ;-
and
IT
+D{IND(i + 1) e T}
lii:z{Yi)' since; is the rank of Yi in the full data set. Notice
III are used
in the explanation above, the calculations depend only on
the function D{IND(i) e T}. We also get the actual values of I T I and I T I for "free", since
Iif I = I T I h T(Y 111)' and I T I = I I I - 1T I·
Thus, for any fixed T e If, to calculate D(T),
we need to go through [D{IND( i) e T }: 1 ~ ; ~
1I I]
exactly once. Therefore,
ei :::: 21 I 12 + Ilf I x III
which is an order of magnitude less than what was initially expected. Thus,
111- 1
eifel::::
The storage used for computing
1~ I is
In most applications,
number of computations,
ei,
+
I~I-I
e is less than 4 I I I.
I
a function of
I I I.
Table E.!. gives the approximate
for various situations, and specific values of
ei
when the sample
size is 100.
As can be seen from Table E.l, in the case of Lipschitz boundaries the number of
computations required for even moderate sized data sets can be quite astronomical.
In analysing the EPA Cancer data set (section E.3) of dimensions (40 x 64), a
restrictive search for the optimal Lipschitz boundary was conducted.
preliminary estimate of the boundary: (b I ,b 2 ,
• • • . , b64 ),
where 1 ~ b i
phase the grid is partitioned into eight subgrids of dimensions (40 x 8).
... " b64 ) fixed, the values of (b I ,b 2 ,
(b I ,b2,
... " bg).
Keeping (b I ,b 2,
the values of (bg ,b lO ,
estimate, (b I ,b 2,
•••• ,
b I6 ).
.... , b64 ).
••• "
We start with a
~
40.
In the first
Keeping
(b g ,b IO '
bs) are optimised. This procedure yields (say),
... " bg) and
(b l i ,b IS " .. " b64 ) fixed we optimise for
At the end of the eighth such pass, we have as our new
The same pl'Ocedure is conducted for the various possible
divisions of the grid into subgrids.
This entire process is iterated until there is no further
change in any of the 64 co-ordinates of the boundary.
-40-
TABLE E.!: Number of Simple Computations per Estimator calculation, for various
examples.
Boundary
Dimension
No. of Candidates
Change Point
!
I II
Epidemic Change
1
111 /2
Linear Bisection
2
Template (Oriented
Rectangle)
2
Lipschitz Curve on
Grid (Mx N)
with constant=c
2
#Calculations,
eI ~ 2IIP+~/x III
I~I
31/1
2
3.0x 1tr
2
3
21 1 1 + 111 /2
8111 2
61 II
2
111 2 /2
M(2c+1)N
2
#Calculations
when I I 1=100
5.2x
uP
8.0x 1tr
3
2111 + 111 /2
5.2x
uP
21 I 12 +M2 N(2c+1)N-l
-1
M= 1I 12 / 3,N= 1 I 1 /
when c=2
when c=3
1 3
Oriented
Hypercuboid
7.8x 1rP
2.6x uP
3
2 I 1 I 2 + I I I 3/2
/ I /2/2
5.2x 1rP
More generally, consider a (Mx N) grid, where N=k· n for some integers k and n. In
the first phase we consider k subgrids of dimensions (M x n).
For each subgrid we consider
(2c+1;n candidate boundaries. There are k subgrids and n possible divisions into k subgrids.
Thus the total number of candidate boundaries
I~ I
considered in one iteration is
n·k·(2c+1;n=N·(2c+1;n. The number of candidate boundaries in the unresirictedsearch is
M· (2c+1)N -1.
Thus the ratio of the number of computations required for the unrestricted
method to the number of computations required for one iteration of the restricted search
method is approximately (2c+ 1)N - n -
1.
In the case of the EPA cancer data, c=2, N=64 and
n=8 and this multiple is approximately 2.8 x
1oJ8 !
As illustrated in section E.3, it is
computationally feasible to conduct the restricted search method repeatedly.
We choose one boundary over another only when there is an increase in the criterion
-41-
function, D(T): 0 ~ D(T) ~ D(e/) ~
i, where e/ is the unrestricted boundary estimator.
This
guarantees that our restricted estimator will eventually stabilise.
The various simulation studies discussed below substantiated the above approximate
formula for the number of computations.
For selected simulations, the CPU time per
boundary calculation was noted. It was seen that the ratio:
[CPU Time per Boundary Estimate]f[Estimated No. of Calculations,e/l
was approximately a constant for the various runs.
E.2 SIMULATION STUDIES
The finite sample behaviour of the boundary estimator e I' was studied when d=l and
d=2. In all cases F and G are entirely unknown to the statistician, do not come from the same
parametric families, are both continuous with common mean, median, variance and skewness.
Here, F is the N(O,l) distribution, and G has density
~
When d
g(x)=0.697128· x 2 • D{ I x
I < 1.291}.
2, Brodskii and Darkhovskij (1986) require that F and G are discrete with finite
supports, that the measurements come from a square grid and that the boundary is necessarily
defined by a finite set of parameters. Thus in our 2-dimensional situation no other boundary
estimator is appropriate.
In the one dimensional case, the boundaries are as in Example 1.a., the change point
problem.
See Table E.3.
Note that, in this case the boundary estimator is similar to the
change point estimator proposed by Carlstein (1988), but is not identical.
The criterion
function in the case of our boundary estimator is:
D(T):=( I T 1/1 11)( I T 1/1 II)· S(d[, dJ, ..., dJ;/) for all T E c:r,
whereas the criterion function for the Carlstein (1988) change point estimator is:
DdT):=~( I T
1/ I II )( IT I/ I II ). S(d[, di,
..., dJ;/) for all T E c:r.
Sample sizes 25, 50, 100, 200, 225, 625, 1225, and 2025 were studied, when e=OAO.
All of these results are presented in Table E.3. Sample sizes 50, 100, and 200 were studied so
-42-
the results can be compared to that of Carlstein( 1988). Sample sizes 25, 225, 625, 1225 and
2025 are of interest, as compared to the corresponding results when d=2.
The results of
Carlstein(1988) are given in parentheses when appropriate.
It may be noticed that in the one dimensional case ( I I
I =50,
100, 200), the boundary
estimator out-performs the change point estimator for all three choices of norm. We helieve
that this is due to the difference in the criterion functions. (Simulations done on the same data
as the boundary estimator, but using the discriminant function Dc resulted in numbers very
close to Carlstein(1988)). Also, in the case of the change point estimator the reported error is
the average of 18 average of 8(8,
e).
eI, while in the case of the boundary estimator the reported error is the
However since the errors in terms of 18 -
eI were generally less than half,
this is only a marginal advantage for the boundary estimator.
The only difference between
D(T) and DdT) is in the weighting functions used to down-weight candidate boundaries that
are close to the extremes, i.e. one of the regions generated by the boundary has area
~
O.
Figure E.2. is a plot of w}(x)=4x(l-x), on one hand and w2(x)=2~x(l- x), on the other. The
boundary estimator uses w}(x), while the change point estimator uses w2(x).
w}(x) ~ w2(x), everywhere in the range.
regions w}(x)
~
In the middle ranges w}(x)
~
Notice that
w 2(x), but in the tail
w 2(x). Thus w} de-emphasises the tail values more than w 2, and since the true
8=0.4, the boundary estimator is favoured to win this one.
Both when d=1 and d=2, the number of simulated replications, NREP, was 1,000. In
all cases, the three different mean-dominant norms (Arithmetic-Mean, Cramer-von-Mises, and
Kolmogorov-Smirnov) were computed.
In the two dimensional case the boundaries are as in Example 2.a, i.e., linear bisection,
with
e
connecting the points (0.85,0.00) and (0.25,1.00).
(25 x 25), (35 x 35), and (45 x 45) were studied.
Grid sizes (5x5), (15xI5),
In all these cases the exact true 8 is never
available as a candidate boundary - a likely practical situation. Table E.4. presents the results
of the simulations when d=2.
-43-
,,:
.j"
I
.
---r",
I .
~---------------------:-----~-<"=--
II:·
I
!I.
I
Figure E.2.
Two Different Weight Functions
i
,
!....
,
0
.-
....
())
",
0
!'.'.
....
OJ
0
:
"
ol="
0
<.D
c
-
-t-'
U
.l§
tl..&...
o~
,"
! .
"
r--
U>
.
"';1(01:),,"
..
:.~ '
j ,';
:
~,,,:; ..
:
.l'i;"
L/)
0
,i.
-t-'
.c
.-w
O'l
3:
'¢
0
;;.
I"')
0
,
C'I
0
,....
0
0
o 0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
A: Lebesgue Measure of T
0.8
0.9
1.0
:::.
TABLE
U
Simulation Study when d=1.
ern
Sample
Size
Norm S(·)
E{8(6,
25
arithmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.1498
.1435
.1160
(.0036)
(.0036)
(.0031)
50
arithmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.1172 (.257)*
.1100 (.235)*
.0918 (.179)*
(.0032)
(.0030)
(.0027)
100
arithmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.0765 (.201)*
.0772 (.178)*
.0730 (.144)*
(.0029)
(.0026)
(.0023)
200
arithmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.0346 (.097)*
.0441 (.097)*
.0550 (.096)*
(.0017)
(.0019)
(.0019)
225
arithmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.0275
.0361
.0476
(.0015)
(.0016)
(.0017)
625
arithmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.0071
.0099
.0249
(.0005)
(.0005)
(.0010)
1225
arithmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.0036
.0018
.01'l7
(.0002)
(.0002)
(.0006)
2025
arithmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.0022
.0027
.0094
(.0001)
(.0001)
(.0004)
{standard error/
tEach "e:cputed diatance" ia an empirical eatimate baaed on 1000 realizationa of
error of thia eatimate ia given in parentheaea.
* From Carlatein (1988).
-45-
er; the auociated atandard
TABLE E.4: Simulation Study when d=2.
Grid Size Norm S(·)
E{0(6,0 I )}
(standard errorl
5 X 5
arithmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.3190
.3050
.2750
(.0042)
(.0043)
(.0043)
15 x 15
arithmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.0950
.0956
.1011
(.0038)
(.0033)
(.0025)
25 x 25
arithmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.0156
.0209
.0509
(.0007)
(.0007)
(.0014)
35 x 35
arithmetic-mean
Cramer~von Mises
Kolmogorov-Smirnov
.0101
.0114
.0304
(.0002)
(.0002)
(.0008)
45 x 45
ari thmetic-mean
Cramer-von Mises
Kolmogorov-Smirnov
.0059
.0064
.0202
(.0001)
(.0001)
(.0006)
tEach. "erpected di8tance" i8 an empirical e8timate ba8ed on 1000 realization8 of
error of thi8 e8timate i8 given in parenthe8e8.
eI" the auociated 8tandard
Grid sizes larger than 45 x 45 were not considered, since the average errors of the
Arithmetic-Mean and
misclassified), for
Cramer-von-Mises estimators
I I I =2025.
was smaller
than
0.01
(i.e.,
1%
The average errors of the estimators in terms of E{o( 6,0 I)}
and the corresponding standard errors are Figure E.2 reported in Tables E.3 & E.4:
NRE?
E{0(6,0 I )} :=
0(6,01) / NREP, where
is the estimator in the the k th replication.
I:
01
k=l
It can be seen that the boundary estimator performs well for moderate sample sizes, in
terms of IE{ 0(6,0 I)}' which measures the average "area misclassified". The difference between
any two consecutive values of 1E{0(6,0/)} for fixed norm but increasing sample size is also
significant in terms of the standard errors.
Notice that for sample sizes
~
100, the
Kolmogorov-Smirnov norm performs the best and the arithmetic-mean norm the worst.
For
higher sample sizes Sam emerges as the best choice of norms and S[( 5' as the worst. As a
-46-
matter of fact, the improvement of the arithmetic mean norm over the Kolmogorov-Smirnov
norm gets more dramatic with higher sample sizes.
The standard errors also decrease
markedly, with higher sample sizes.
Histograms of the values of [\fii. a(8,e I)] for each of the five grid sizes and each of the
three norms were graphed to study the asymptotic distribution of the error of the boundary
estimator, since the factor 'Vi seemed to approximately stabilise the o(8,e I)'s in the cases
studied.
Figures E.6 - E.I0 each presents triplets of histograms corresponding to the three
norms, for the different grid sizes, along with the quantiles. For a given grid size, the three
norms show more agreement in the lower quantiles than in the upper quantiles. Thus in the
most favourable cases, the three norms are comparable. The maximum values of ['f1i' o(8,e / )]
are given in Table E.5, which shows their performances in the worst cases.
TABLE E.5. Maximum Values of ['Vi.8(8,e / )] from simulations. NREP-I000.
Grid Size:
5x5
15x15
25x25
3.5x35
45x45
am Norm
Cv Norm
KSNorm
2.4
2.4
2.4
7.5
7.5
7.5
10.7
7.3
7.0
1.3
2.4
7.6
0.9
2.3
.5.7
It may be noted from the histograms that the error of our estimator is approximately
zero with increasing frequency; for grid size (45 x 45) the three norms exhibit 55%, 52% and
34% near zero observations. The most dramatic decrease in the range of the o(8,e I)'s is seen
in the case where the grid size is increased from (25 x 25) to (35 x 35).
It may also be noticed by comparing values of lE{o(8,e / )}'s (for 111=25, 225, 625,
1225, and 2025), from Table E.3. and EA. that for the same sample size and choice of norm
the boundary estimator performs better
dimensional case.
In
the one dimensional case than in the two
This may be explained by the fact that the sIze of the candidate pool,
-47-
05 X 05 Gridl Histograms of (05*Errors)1 All Three Norms
051 L 1
I
~.
I
I
I
(auantiles
I
maximum
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
-300
t-
•
•
I
B
I
I
III
1
200
quartile
median
quartile
1-100
2
)
minimum
2.4000
2.4000
2.4000
2.4000
2.0000
1.6000
1.2000
0.8000
0.2000
0.2000
0.2000
05/L2
I
I
I 01
I
(auantiles
I
maximum
200
1-100
B
I
II
I
I
I
I
1
I I II
2
quartile
median
quartile
minimum
,
a
I
I
I
I
(auantiles
I
I-
..
maximum
250
.... 200
-150
-100
I I
I
2.4000
2.4000
2.4000
2.4000
2.0000
1.6000
1.2000
0.6000
0.2000
0.2000
0.2000
.'
05/L3
I
I
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
1-300
I-
)
I
I
1
quartile
median
quartile
-50
2
minimum
- -
..
)
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
(
2.4000
2.4000
2.4000
2.2000
2.0000
1.4000
0.8500
0.4000
0.2000
0.2000
0.2000
15 X 15 Gridl Histograms of (15*Errors)1 All Three Norms
151 L 1
U
I
I
(auantiles
~
maximum
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
..... 400
-300
~200
0
~
1
r
1
2
3
r
4
r
5
I
I
6
7
quartile
median
quartile
1 00
~
)
minimum
7.4700
7.3950
6.9300
4.5300
1.5300
0.6000
0.4050
0.1950
0.1950
0.1950
0.1950
15/L 2
HllJ
H
(auantiles
maximum
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
400
300
200
quartile
median
quartile
100
0
1
2
3
4
5
6
7
)
minimum
15/L3
"
(auantiles
maximum
400
300
200
quartile
median
quartile
100
0
I
1
7.4700
7.2000
6.4016
3.5400
1.7400
0.7950
0.4050
0.1950
0.1950
0.1950
0.1950
2
3
4
5
6
7
minimum
-i'-
)
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
7.4700
6.7930
4.3935
3.0600
2.0700
1.2000
0.7350
0.4050
0.1950
0.1950
0.1950
25 X 25 Gridl Histograms of (25*Errors)1 All Three Norms
251 L 1
If-++-I
(ouantiles
I
maximum
t- 200
quartile
median
quartile
-100
III r
-I
I
I
'I
I
I
I
I
1
2
3
4
5
6
7
8
9
I
0
1
10 11
,~
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
>-400
t-300
)
minimum
10.650
1.850
1.250
0.725
0.325
0.275
0.200
0.200
0.200
0.200
0.200
25/L2
[]
maximum
I-
300
t-200
lL..
0
1
t-100
-,
r
-r
I
I
I
2
3
4
5
6
7
)
(Ouantiles
I
I
I
I
quartile
median
quartile
minimum
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
7.2750
3.8487
2.0488
1.1400
0.6750
0.2750
0.2000
0.2000
0.2000
0.2000
0.2000
,
~
.,
25/L3
(Ouantiles
maximum
300
200
100
0
1
2
3
4
5
6
7
quartile
median
quartile
minimum
-;0-
•
)
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
7.0500
6.0740
4.3250
2.6675
1.7500
{).8750
0.4750
0.2750
0.2000
0.2000
0.2000
~
35 X 35 Gridl Histograms of (35*Errors)1 All Three Norms
351 L 1
,
0]]
I
(auantiles
1---1
maximum
100.0"10
99.5"10
97.5"10
90.0"10
75.0"10
50.0"10
25.0"10
10.0"10
2.5"10
0.5"10
0.0"10
~400
~300
'- 200
I
I
quartile
median
quartile
'""100
I
I
I
I
I
I
-.
I
I
0.20.3 0.40.50.6 0.70.80.9 1.0 1.1 1.2 1.3
)
minimum
1.2950
1.2245
0.8750
. 0.5600
0.3850
0.2800
0.2450
0.2450
0.2450
0.2450
0.2450
35/L2
[]
(Ouantiles
I
I
I
I
maximum
100.0"10
99.5"10
97.5"10
90.0"10
75.0"10
50.0"10
25.0"10
10.0%
2.5%
0.5%
0.0%
~300
~200
I.
quartile
median
quartile
'""100
. -,
I
1
2
)
minimum
35/L3
.
maximum
400
300
200
quartile
median
quartile
100
1
;
. ..
.-
Ouantiles
[ll]
0
2.4150
1.6100
1.1883
0.7350
0.3850
0.2800
0.2450
0.2450
0.2450
0.2450
0.2450
2
3
4
5
6
7
8
-S1-
minimum
100.0%
99.5%
97.5%
90.0"10
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
7.6300
4.8289
3.6024
2.2015
1.3650
0.8050
0.3850
0.2450
0.2450
0.2450
0.2450
45 X 45 Gridl Histograms of (45*Errors)1 All Three Norms
451 L 1
I]]
(auantiles
H
I
I
500
--400
-300
f- 200
I
,
0.2
I
, ,
I
0.3 0.4
0.5
0.6
I
quartile
median
quartile
f-100
,
I
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
maximum
I-
r
0.7 0.8
0.9
)
minimum
0.90000
0.81000
0.54000
0.31500
0.27000
0.18000
0.18000
0.18000
0.18000
0.18000
0.18000
45/L2
[]
I
I
(auantiles
I
I
maximum
f-500
f-400
f-300
I-
200
quartile
median
quartile
f-100
I
I
I
1
2
minimum
J
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
2.2500
0.9000
0.8100
0.5400
0.3150
0.1800
0.1"00
0.1800
0.1800
0.1800
0.1800
45/L3
[]]
(auantiles
I
I
I
I
maximum
1-300
I-
II ~ - .
0
200
1-100
I
I
t
1
I
1
2
3
4
5
quartile
median
quartile
minimum
-5#.--
)
100.0%
99.5%
97.5%
90.0%
75.0%
50.0%
25.0%
10.0%
2.5%
0.5%
0.0%
5.6700
4.5432
3.2389
1.9755
1.2600
0.7650
0.2700
0.1800
0.1800
0.1800
0.1800
I~ I= II I
in the one dimensional case whereas
I ~ I =61 I I
in the two dimensional case.
When the candidate pool is bigger there is more likelihood of error.
E.3. U.S. CANCER MORTALITY DATA
Maps are a powerful tool for epidemiological investigation; perhaps the most celebrated
example is John Snow's (1855) study of the London cholera epidemic of 1849.
Cliff and
Haggett (1988) provide a colourful history of maps in an epidemiological context. U.S. cancer
mortality rate maps have been compiled by Riggan, Creason, Nelson, Manton, Woodbury,
Stallard, Pellom, and Beaubier (1987), for use in "developing and examining hypotheses about
the influence of various environmental factors," and for investigating possible associations of
cancer with "unusual demographic, environmental, industrial characteristics, or employment
patterns."
In particular, for cancer of the trachea, bronchus, and lung (including pleura and
other respiratory sites), white males exhibited a pronounced geographic mortality pattern
during 1970-1979 [see Figure E.ll]. The heavy concentration in the South-Eastern region of
the data-grid (a region roughly equivalent to what is historically and culturally known as "The
South") might be related to the following factors: prevalence of cigarette smoking, especially
as associated with the region's "cash crop" tobacco; access to preventative, diagnostic, and
therapeutic health-care, especially as a function of economic and educational conditions;
employment in the textile and furniture manufacturing industries, involving extensive exposure
to airborne fibers and dusts.
In order to explicitly delineate this region, with minimal constraints on its shape, we
estimated a Lipschitz boundary shown in Figure E.l!.
The algorithm used the arithmetic-
mean norm to estimate the boundary, as it had previously proven to be the most effective in
the simulation studies when analysing large grids
(I I 1 > 100).
The program employed a
restricted search (see section E.l) among candidates with Lipschitz constant equal to 2.
Preliminary estimates using
Lipschitz constant=1
-53-
and
Lipschitz constant
~
3 yielded
I
o
·0
f'.
(J)
...
.
:· 0:I:
·
...
0
""0
o
C
.. .. . ...• .....•
o
:J
..c
I
o
L.-
3:
a
.. a
C/)
W
l-
D
.... ... .. .. ...
. .. . ... ... ... ...
. ..
D
()
... ..
c
1-.
fJ)
.
.....
... 0" ...... ...... ...... ...
W
~
CD
I...L-o
00.>
..!:
>-()
+ + +
+ + +
... + +
.
+ + +
..
o
o
-.Jr«0.>
I-£.
.
.
•
•
D
... .. ... 0
..
00
...
...
......................................... ... CO
.
n:::()
W§
z
•
«
o
+ + + •
+ + • +
...
...
.
,
I...L-
...
...
...
.,.
.
...
+ •
~
+ + + • • + + + • • •
• • + • •
• • •
• • • • • •
• •
+ • •
•
•
•
•
•
•
•
•
•
•
•
•
•
•
0
•
•
•
•
•
0
D •
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
0
•
•
•
•
•
••
•
0
•
•
•
••
0
•
+ •
•
•
••
••
••••••••••••••••••
• • • • • • • • • • • • • • C·· • • • • • • • • •
•
0
••
•
0
a
•••••••••
0
•
•
•
•
•
+ + + + + •
•
•
D
0 0 • •
0 • + 0
•
•
+
•
a..
•
•
...
...
... .. . .. ...... . .. . .. .
0 • • • • • •
...
...
...
•
•
•
•
•
•
•
•
+ • + +
•
••
•
•
z
:::>
o
OJ
N
l-
I
...
• D
• • •• •+ •• •• •• 0• •• •• •• • • • • •
o-
+
· 0 0 · · • • • • • • • • • • ••
•
o
...
+ ••••••
•
OU
.
a a
...
«
....J
...
..
+
... ...
... .
... ... . ...
fJ)
fJ)
...
...
•
... •
0.>
..
.
........ 0'"
~L
•
...
...
et: ......
... ..
W
..
+
... CO
fJ)
...
•
... •
+
.
I-~
>o....J 0:::
o«
..
.
...
...
...
... ...
.................................... ... 00 ...
... ... ...
...
...
+ + +
+
a
•
. ... ...
.......... 0'"
~
.. ... ... ... ...
...
•
...
...
W
I- o
if)
CL
0::
«
-.-J
>ro
....J W
~ I«
0:: ~
o r-
~ if)
w
boundaries that were visually too smooth and too spiky, respectively.
These results using
Lipschitz constants 1 to 5 are included in the Appendix as are the program codes.
Besides encompassing The South, the estimated boundary interestingly also includes
the heavily industrialized and polluted Eastern seaboard.
e
E.4 APPLICATION OF THE BOOTSTRAP TO
l
We would like to increase the applicability of
e by extracting information about
I
possible error in estimation. The following bootstrap procedure is suggested.
There are three unknowns in the boundary problem: the two unknown distributions F
and G and the boundary 0. The original probability stl'Ucture can be defined by the triplet
(F,G; 0). Having estimated 0 with
e
the empirical c.d.fo's generated from the two regions
I'
induced by
el are the estimates of F and G.
defined by
(F, G; eI)'
We now have the estimated probability structure
Bootstrap samples can be drawn from this probability structure.
Formally, {Xi: i E 1} are data from the population described by (F,G; 0), i.e., Xi
F if i E
e and Xi
G if i
E~.
Define Lhe estimates of the unknown distributions F and
G by
F(x) = ~
O{X i $
icel
and
Draw {X:: i c ell as i.i.d. observations from
to give a bootstrap sample
x}/leII
X;
of size
F,
II]
and
{xt:
i c ~l} as i.i.d. observations from
from the "estimated" population
(F,G;
G,
e
I ).
ej=eAXn for each such bootstrap sample, and use these ej's to approximate the
distribution of el' We could completely enumerate the bootstrap distribution of ej by
considering all [IeII10}l*I~III~}l1 subsamples of the original sample. However, we shall restrict
Calculate
-55-
ourselves to approximating the bootstrap distribution of 6j
by taking NBOOT bootstrap
samples.
If NBOOT bootstrap samples are drawn from one set of initial data of size
get {6jl'6i.e,
,6i,NBOOT}, each defining a partition of the grid. If each of these 6j,,'s was
a number or a vector, we could look at the
numbers to understand the variability of
Lipschitz curve example (2.c;
parameters.
I I I, we
6/.
varianCf~,
percentiles and histograms of these
But as can be understood in the context of the
Section C.2),
e
may not be described by a finite set of
Also, in the two dimensional case a picture demonstrating the variability of
6/
may be valuable. Since 6j" defines the partition (0j". ~jk) one way of studying the variability
of 6/ is by looking at the (1-0')100% indifference zone, Z(l.a) which is defined as follows:
NBOOT
Define Ci:=
L
~
~
i{i E 6jk} = #{times i is inside 6i,,: k
=1,NBOOT},
iEI
k=l
and Z(l.ay=={i
E
I:a/2*NBOOT $
Ci $ (1-a/2)*NBOOT}
i.e.,Z(l.a) excludes all points in I, that either belong to the 0jk's at least (I-O'/2)*NBOOT of
the time or belong to the ~j" 's at least (I-O'/2)*NBOOT of the time. Notice that Z(l.a) is a
function of the underlying probability structure given by the triplet
(F, G;6 I)'
A large or
wide Z(l.a) is an indication of high variability in the OJ's, and conversely a non-existent or
narrow Z(l.a) will mean that the 6i's have low variation. Furthermore, should the boundary
6/ lie completely in Z(1.a)(6/), we may infer that the "bias" of 6j (as an estimator of 6/) is
low.
Next, we pass these conclusions about the bootstrap distribution of 6j (variance and
bias) to the sampling distribution of
6/.
In practice, this entire procedure is to be carried out
for a single set of data from the grid I.
Simulations of the entir'e bootstrap procedure were done in the case of d;:::;2, linear
bisection, grid size=15 x 15.
The parameters are as in section E.2: Linear bisection, with
connecting the points (0.85,0.00) and (0.25,1.00); also NBOOT=1000, and 0'=0.10.
-56-
e
A
thousand replications from the true population (F,G;6) were simulated and a picture of ZO.90
generated from the triplet (F, G; 6) so as to establish a basis of comparison for the bootstrap
generated Z~.90's that follow.
O'!e hundred replications (NREP) of the entire bootstrap
procedure were simulated. In each of these one hundred cases, we obtained a picture of Z~.90'
Also obtained were the NREP
6['s,
their errors 0(6,6 [)'s, and the proportion of points in the
grid that fell in Z~.90'
presents ZO.90 and Figures E.12.b-d are
Figure E.12.a.
"indifference zones"
illustrating three of the Z~.90' These were selected to represent the "good", "average" and
"bad" outcomes obtained.
In each of these three cases, the
replications is drawn in the grid.
6[ that gave rise to the bootstrap
Notice that they fall within ZC;.90
as does 6 in Figure
E.12.a. This suggests a sort of "unbiasedness" of the boundary estimator (from Figure E.12.a)
and provides a degree of validation to the bootstrap procedure (Figures E.12(b-d)).
Also reported is the proportion of points in the grid that fell in Z~.90' It can be seen
that the "indifference zone" Z(l.oj is comparable in size to the simulated Z {l-o}" We believe
that the bootstrap can be used as an estimator of variability of 6[! when we are confident that
the sample size is large enough to give a reasonable estimate of 6.
This approach could be generalised by plotting for each point in the grid, the
proportion
et /NBOOT,
constructing a "surface" over the grid I, where the height of the
surface at any point i E I will reflect its affinity to the regions {8jk'S}. We have confined our
current analysis to indifference zones, however.
Here's another generalisation of the bootstrap procedure.
Since what we want is
neither a "narrow" Z* nor a "wide" Z* but rather the "correct" Z* relative to the true
sampling distribution of
(say) distance of
6[.
6[,
we could put a "buffer" around
6[ of all
points that are within v [
The grid is now partitioned into three regions: the buffer and two
disparate regions A and B (say). If now
calculated from the points in A and
F is
G is
calculated from the points in B, the estimates of
r:-
-;j (-
F
and
G will
eland
be better, since the regions A and B can be expected to be more homogeneous than
el'
estimates
The trade-off in this case will be the reduced number of points on which the
F and Gwill. be based.
,
-58-
Figures E.12.a, b, c, d.
e
Estimated Variability of Bo'Undary Estimator,
I' 'Using bootstrap replications
NBOOT =1000, Grid Size=15x 15, 0:=0.10
The set ZO.90 of variability ofe I is denoted by'. 's in E.12.a
and the set
Z" 0.90 of bootstrap variability is denoted by'. 's in E.12.b,c,and d.
Figure E.12.a
The original 8=(5,6,10)
Proportion of the Z 0.90 area is 0.21.
15**
14**
13**
12**
11**
10**
9**
8**
7**
6**
5**
4**
3**
2**
h*
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
o0
0 0 0 0 0
o0 0
o0
0
0
0
0
0
0
0
0
0
0
0
0
0
o 0 0 0 000 0
1
1
1
1
1
1
1
1
1
1
1
1
1
o
o
1
o
o
1
1
1
1
1
1
1
1
0 0 0 0
0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 000 0
000 0 0
o0 0 0 0
o0 0 0 0
o0 0 0 0
o0 0 0
o0 0 0
. 000
Figure E.12.b
e =(5,6,10), a(8,~)=0.000
1
Proportion of the Z~.90(81) area
15**
14**
13**
12**
11**
10**
09**
08**
07**
06**
05**
04**
03**
02**
01**
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
-59-
IS
0.16.
000 0 0 0 0 0
o0 0 0 0 0 0 0
o0 0 0 0 0 0 0
000 0 0 0 0 0
o0 0 0 0 0 0 0
o 0 0 0 000
0000000
000 0 0 0 0
o0 0 0 0 0
o0 0 0 0 0
o0 0 0 0 0
o0 0 0 0 0
o0 0 0 0
o0 0 0 0
o0 0 0 0
Figure E.12.c.
0(8,6 2 )=Q. 031
Proportion of the Z~.90 (8 2 ) area is 0.28.
62 =(5,7,9),
15**
14**
13**
12**
11**
10**
09**
08**
07**
06**
05**
04**
03**
02**
01**
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
o
0
0
o0 0
o0 0
000
o0 0
o0 0
o0 0
o0 0
o0 0
o0 0
o0 0
o0
o0
o0
o0
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0 0 0
0 0 0
0 0 0
0 0 0
000
000
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
000
000
0 0 0
0 0 0
Figure E.12.d.
6 3 =(2,10,6) ,0(8,631-=0.378
Proportion of the Z~.90(83) area is 0.22
15**
14**
13**
12**
11**
10**
09**
08**
07**
06**
05**
04**
03**
02**
01**
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
o
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
-60-
1
1
1
1
1
1
1
1
1
1
1
1
0 000
000
o0
o
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
F. PROOFS
Proposition 1: The function 8(· , .) is a pseudometric. That is, it satisfies:
(l.d) [Triangle Inequality] 8(T, a) ~ 8(T, T')+8(T', a).
Proof 2f Proposition!. Propertyl.d: Expanding 8(6, T) in terms of 6, T, and T':
8(6, T)=min{ ..\(T n ~ n T')+,\(T n ~ n I')+..\(I n 0 n T')+,\(I n 0 n I'),
..\(I n ~ n T')+..\(I n ~ n I')+..\(T n 0 n T')+'\(T n 0 n I')}=:min{U, V}, (say)
Making the analogous expansions of 8(6, T') and 8(T', T), in terms of 6, T, and T', we see
that 8(6, T')+8(T', T) equals one of four possible expressions, e.g.,
W:=,\(T' n ~ n T)+'\(T' n ~ n I)+,\(I' n 0 n T)+'\(T' n 0 n T)+
+'\(T n I' n 0)+'\(T n I' n ~)+'\(I n T' n 0)+'\(I nT' n ~).
Since
the
summands
8(6, T)=min{U,V}
~
of 8(6, T')+8(T', T).
in
U
are
a
subset
of
the
summands
in
W,
we
have:
U ~ W. Similar inequalities hold for the other three possible expressions
0
The parameter of the boundary problem is the unknown but fixed triplet: (6; F,G).
We have the data set {Xi,i E I}, and the candidate pool is 'rr.
The asymptotics are as
111-00.
Theorem 1: Let d=2. If the set A E
a- I corresponds to a boundary expressible as a rectifiable
curve r(.), then ~(c:PI<A» $ 18· (L(r)+I)jmin{n l , n2 }.
Proof:
For any set U ~ 'U 2 ' let U denote the closure of U, let c:B(U):=U n iJc denote the
boundary of U, and let j(U) denote the interior of U.
For the set A we are assuming:
c:B(A)={r(t) E 'U 2 : t E [a, b]}, with j(A) # ¢> and j(A C ) # <p.
Define K:=r2. L(r)· max{n 1 ,n 2 H
Consider intervals [aj,b j ], 1 ~j ~ K, such that
a=a1
< b 1 =a2 < b2= ... =a[( < b[(=b. Let rj(t) be the curve r(t)
let Lj:=L(rj).
restricted to t E [aj' b j ], and
In particular, choose the intervals so that Lj=L(r)/K V'j.
Denote
Rj:={rj(t) E CU 2: t E [aj,b j ]}. Let C j E e I be a cell containing the point rj(aj); and let lt j be
the collection of neighbouring cells C E e I that share a common edge or vertex with C j
(including C j itself). Since Lj ~ t(1/max{n 1 , n2}), the points in R j form a subset of the points
in 3(lt j ). Thus we have:
K
#{CEeI:CE~J<A)}~#{CEel Cn~(A):;6¢>}=#{CEeI: Cn(U
R j ):;6¢>}
J=l
K K K
=
L: c(e}} ~ 1 (R j n C) :;6 ¢>} ~ j~l L: c(e I I{R j n C :;6 ¢>} ~ j~l L: c(e}{C E ltj } ~
Finally, recall that A(~J<A)) = #{C E eI : C E ~J<A)}/nln2'
K· 9.
0
Theorem:':' [Strong Consistency]
If R(6) holds, then
I 11 6 ·8(6,
e
0 as
I) -
111-+00, with probability
1.
Proof: From Theorem 4, Theorem 2 follows from the Borel-Cantelli Lemma because,
by R.3,
Theorem J;. [Bound on Error Probability]
If R:J(O) holds, then for any e > 0,
....
2
2
P{8(6,6I»e}~10·Ic:rII·e%p{-K'CT'JJ.e
I II > No(e).
Proof:
2
·III},
for
III
sufficiently large so that
Further, K is a universal constant, and CT &; JJ are constants for the problem.
From Theorem 4, Theorem 3 follows as the case 0=0. 0
Theorem i;. [Robustness] Assume: R 2 (0). Then,
(T5)
".~,,~
lim
8(6,e l ) ~ [CT'JJ]-l.,,*.
a.s.
Ill-oo
Further, when A(9) = A(e)=t, (i.e., for a balanced partition),
(T5')
-62-
f!22f;, For any 0 < x < 1, and 0 < c $ c*(x), by Theorem 6,
EfI/P{8(6,E>I»(1]*+£)/lTj.lX} <00,
and by the Borel Cantelli Lemma P[{8(6, E>l) > (1]*+£)/lTj.lx} infinitely often] = O.
Therefore,
for Xk=¥ and £k = min{
!' £*(xk)}' 3 a set Ak, such that:
P(Ak)=O and Vw~Ak'
lim 8(6,E>1) $ (1]*+£k)/lTj.lXk'
I I I ---'00
00
Consider then the set A= U A k • Then,
k=l
P(A)=O and Vw ~ A, [ lim 8(6, E>I) $
I I I ---'00
(1]*
+ ck)/lTj.lXk, Vk ~ 1].
Theorem 4i
H
1l:J(6) holds, then for any e > 0,
(T4)
P{III 6 .8(6,
6 I»e}
~
III
N1 (£).
sufficiently large so that
~
JO·Ic:rII·ezp{-K'iT2'p2.e2·III1-26},
for
III
Further, K is a universal constant, and iT & p are
constants for the problem.
Theorem §;.
H ~(O) holds, then for any 0 < z
< 1, and 0 < e ~ e*(z),
61) > (fJ* + e)/lTJJZ}
(T6)
P{8(6,
so that
I I I ~ N 2(e,z).
~ 10·
I c:r1 I . ezp{-K· e2 • Ill}
for
I II
sufficiently large,
Further, K is a universal constant, and iT & p are constants for the
problem.
Proofs of Theorem 1
&. Theorem 2:. Applying Lemma 3, Lemma 4, Lemma 5, and property
(2.f) of SII/, we have:
8(6, T)
> 'Y => I AI<6)-AI<T) I = [p(6)-p(T)]SII~6~:
i E 1)
> IT' 'Y' 67,
where 67:= E if! 6~/ Ill· Thus,
(1)
1P{8(6, E>j) > c I II- O} $ IP{ I AI<6)-A;r<E>j)
$IP{ I flol A J(6)-A J(E>j)
(2)
1P{8(6, E>j) >
(1]*
I > IT'C I fl-067}
I > IT' C' j.l/2}+1P{67 < j.l/2},
+ c)/lTj.lx} $ IP{ I AI<6)-AJ<E>j) I >
-63-
(1]*
and
+ £)67/ j.lx}
By Lemma 7, J-I
> O. The 1st probability on the r.h.s. of (1) [resp. (2)] is immediately handled
by Lemma 6, statement L6 [resp. L6']. The 2nd probability on the r.h.s. of (1) [resp. (2)] is
bounded by P{ IfJf-J-I1
Denote
fJf==6f( I
7$._
"
U I .- L.J
> J-I/2} [resp. IP{ I fJf-J-I I > J-I' (I-x»].
~$I
Ii
j~e
U
I -e I I
e I II I I I )+~r( I ~ I II I I I)·
I fJf-J-I I
:S
and
~e._ "
!!-I
.- L.J
so
j~.@.
that
Notice that:
6~ 11~:I-A(e)I+A(e)16r-J-IFI +~~ 11~:CA(~)I+A(~)I~r-J-IGI.
The first and third summands on the r.h.s. are handled using Lemma 1. Consider the second
summand on the r.h.s. (a similar argument holds for the fourth).
Hoeffding (1963), we have
ensures that
p{16r-J-IFI >J-I·(I-x)/4}
I e II eventually
exceeds
I IIA(e)/2
2.exp{-c,xleI I }.
:S
any fixed x, giving us (T6). Putting x == ~, we get (T4).
* - ~; T,
Define t'rI:=={a,
Lemma 1
(say); therefore the bound from Hoeffding
(1963) can be combined with the earlier bound from Lemma 6, for e
Lemma 1: Assume R.4.
By equation (2.3) of
> 0 sufficiently small, for
0
- - Ina,
-I, Tna,
Tn~,
In~:
TE'fJ}.
Then:
6
1114. sup A~t'r *I I.\(A)-IAIJ!IIII-o
111-00.
as
Proof: Consider A E lr~. Denote
GJlfiA):=={C E elc ~ A} ~ A and GJlfiA):={C E eI:c ~ AC} ~ A C,
so that {GJlfiA), GJlfiA), GJlI(A)} form a partition of
up of all cells completely contained in A,
e/l
the set of all cells. Thus GJlfiA) is made
and GJlfiA) is the union of all cells completely
contained in A c, and GJl[(A) is made up of all the cells that intersect both A and A c, There is
a one-to-one correspondence between
grid-node
Now, #{C E GJlfiA)}1
which
I II ==
el
and I, where each cell C is associated with a particular
IS
a
vertex
A(GJlfiA»:S A(A)= l-A(AC):S l-A(GJlJ<A»
=[#{C E GJlI(A)}+#{C E GJl/(A)}lIl
-64-
II.
of
C.
Also, #{C E ~,,(A)} $ I ~,,(A)II$ I A 11= III-I AC11$ I II-I ~,,(A) II
$
I II-#{C E ~IA)}=#{C E ~,,(A)}+#{C E ~,,(A)},
. so that I'\(A)-IAIIIIIII $I'\(A)-#{CE~,,(A)}/IIII+
I#{C E ~,,(A)}I I II-I A III 11 11 $ 2· #{C E ~,,(A)}I I II = 2'\(~,,(A».
Thus, sets A of the form 0 and l' are handled immediately by RA. Sets A of the form
§ and 1: are also handled by RA, since ~,,(AC)=~,,(A). For the remaining sets A in fi'~, note
We need some notation, making the dependence upon I explicit. The data will now be
denoted {X[:i E I}, and we write:
ii~x):= L j~TI{X~ $ x}1 1l' II' h~x):= L j~II{X~ $ x}1 11:11'
df= I ii~x[)-h~X[) I, D,,(T):= (I 1'11/1 1 1)( 11:1
where
SII~')
Ii 111 )SII~df i E 1),
is a mean-dominant norm with I II arguments. Also define:
7j r<x):=['\(T n 0)F(x)+'\(T n §)G(x)]1 '\(1'),
!l r<x):= ['\(1: n 0)F(x)+'\(1: n §)G(x)]f'\(1:),
Ch:= 17jr<X[)-!lr<X{) I, Llfi,T):='\(T)'\(1:)SIIPh: i E 1).
Lemma
e:
Assume
~(6).
Then,
6
2
P{III 4'SUPT~c:J)DAT)-~,,(T)1 >e} $ 8·I~II·exp{-Cl·e .111
for
I I I sufficiently large so that I I I ~ N 1 (c).
Proof: Define
D/T):=>.(T)>'(I)S II/ d
r
1-26
4},
The constant C 1 is independent of (8; F,G).
i E f), and note that
I DIT)-Dfi,T) 1= 1('\(1')-1 1'1 II 111 )>'(1:)+(>'(1:)-11: I II I II) 11'111 I III SII~dh:
$1 (>'(1')-1 1'1111 II )'\(1:)+('\(1:)-11: III I II) I l' III
i E 1)
IIII '
since SII~dh: i E 1) is at most 1, by properties (2.e) and (2.d) of S/ll' Now, Lemma 1 applies
whenever I II is sufficiently large so that
II I
~
prove that the inequality of Lemma 2 holds with
-65-
N O(e-;c 4 ), and hence it will suffice for us to
DI in place of DI .
T
T cT
d cT
T dT
d Ii
:::; e Ii+ u Ii an U Ii :::; eIi+ Ii'
Then 15 fiT)-AfiT) :::; >.(T)>.(I)S/I/(eE: iE/), by virtue of
properties (2.e) and (2.c) of S/I/' The same bound applies to AfiT)-15 fiT), yielding
115 fiT)-AfiT)
>'(T)S/I/(fI~: i E /)
I :::;
+ >'(I)SIIfH~:
iE/), by property (2.c).
Now observe that
- Ii <
H
T -
L: J'tTn 13 O{X~ :::; Xl} F(X)I
I Tne I I
-, i
The
P~=sup nlR
1st
modulus
L: 'tTn 13
J
I
I
.
I{X j :::; x}
=I
Tne I
r.h.s. is bounded by 1.
on
the
r.h.s.
is
bounded
by
.
- F(x) , and the factor F(Xf) in the 2nd summand on the
The 31'd and 4th summands on the r.h.s. are similarly bounded.
Substituting these bounds into the r.h.s., we obtain fI~:::; fI~ (say), where fI~ does not depend
upon i. Use an analogous argument to obtain H~:::; H~ Therefore, applying properties (2.e),
(2.b), and (2.d), we find:
The r.h.s. of the above equation is comprised of four analogous summands.
We shall deal
explicitly with only the first one of these, i.e.,
(
e
e
>.(T)- I T I I IT n I I IT n IT pT
III
ITIT +
III
T+
For the 2nd modulus, note that Lemma 1 applies -- uniformly in T E <:T T'
Also Lemma 1
applies to the 1st term inside the 1st modulus. It now suffices to consider
IP{ sup Tt<:T I ( IT n
e ITI III ) p~ > c III -64 }
:::;
L: Tt<:T TIP { p~ > c 111
1 64
- 1 ITn
e IT}'
1
Each probability in this summation is bounded by 2· exp{-2'c 2 1 11 - 264 } (see Dvoretzky,
Kiefer, &. Wolfowitz (1956):
their Lemma 2 and the discussion following their Theorem 3;
Massart (1990) has established the constant to be 2). Dealing in the same way with the other
-66-
three summands, we get the result.
0
Lemma 9: We can write LlJ<T)=p(T). SI[J6~: iE/), where
p(T):= I A(T n 6)A(I)-A(I n 6)A(T)
I.
Proof: Observe that 6J:=p(T)8~/A(T)A(I). Now apply property (2.b) of S/ll'
Lemma ,/: For every T E GJ"/, we have LlJ<T)
~
Proof: By Lemma 3, it suffices to show p(T)
0
Ll[(8).
~
p(0). In the definition of p(T), consider the
expression within the modulus. If this expression is positive, then
p(T) =A(T n 0)A(I n ~)-A(I n 0)A(1' n~) < A(0)A(~).
The negative case is handled similarly.
0
Lemma 5: For 7 > 0,
(L5)
8(8,T) < 7
=*
p(8)-p(T) < 7,
aDd 8(8, T)
> 7 =* p(8)-p(T) > u, 7 > 0,
where u=min[A(9), A(e)], is a constant for the problem. It follows that:
[P(8)-p(T)] ~ lJ(8, T) ~ u -1. [P(8)-p(T)].
(L5')
Proof of L!dili Note that
(3)
p(0)-p(T) = minp(~ n 1')'\(0)+A(0 n I)A(~), A(0 n 1')'\(~)+'\(~ n I)A(0)}.
Comparing this expression to the definition of 8(0, T), the first implication is clear. For the
second implication, by hypothesis we have that,
8(0,T)=minp(~n 1')+'\(0 n I),A(0 n 1')+'\(~ n I)}
> /.
We can write
8(0,T)=min{x + x',y + y'}, & p(0)-p(T)=min{re(x, x'),r e(y, y')},
where r e(x, x')= '\(6)x + A(El)X'.
Then,
-67-
8(e, T) > ,~ mini x+x', y+y'} > , ~x + x' >, & y + y' > ,
~p(e)
the
Further,
bounds
p(e) - p(T)=<T' 8(e, T).
Proof of
!.I&'t
in
the
- p(T) ><T .,.
inequalities
are
attainable,
as
when
0
Fix T E ':r. Let d=8(e, T), and r=[p(e) - p(T)]. By (L5),
d < d+x ~ r < d + x
&
Taking limits as xl0, we get r:::; d, and r
~
d > d-x ~ r > <T' (d-x)
<T . d.
'r/ x > O.
0
Lemm46:
(L6)
IT R3 ( 6) holds then,
P{III 6 'IAAS/)-AA6)1 >e} :::; 8· 1':r/I.exp{-C2 .e2 .1I1 1 -
for
II I
(L6')
sufficiently large so that
I I I ~ N 2(e).
26
},
The constant C 2 is independent of (6; F,G).
IT ~(O) holds then,
P{IAAS/)-AA6)I >fJ*+e} :::; 8·1':T'/I·exp{-C;.e2 ·1I1l
for
II I
sufficiently large so that I I
I ~ Ni(e). The constant C; is independent of (6;
Proof: By definition, e'! E ':r / is the maxi miser of p(
0
),
F,G).
and hence by Lemma 3, the maximiser
of A 1(·) over ':r/o Then, by Lemma 4, we have ~/e) ~ ~/e,!) ~ ~AT) 'r/T E ':r/. And, by
definition, we have D/e[) ~ D/T) 'r/T E ':r[. Now,
The second modulus on the r.h.s. is bounded by sup T!':r [I D AT)-~ AT)
I,
because either
DAe[) ~ AAe'!) ~ ~Ae[) or Ale'!) ~ Dl.e[) ~ Die'!). The same bound applies to the first
modulus on the r.h.s. of (4).
Using Lemmas 4 and 3, and properties (2.e) & (2.d) of SI//' on
the third modulus on the r.h.s.,
[p(e)-p(6'!)]. Therefore,
(5)
-68-
By definition of 01'
for
III
[p(0) - p(0'l)] is no bigger than [p(0) - p(T[)].
sufficiently large, R.2 ensures that a(0,T/) <e/3.
second term in (5)
is
111-5.
111- 5 •
deterministically bounded by e/3'
Under R 3 (6),
Thus, by Lemma 5, the
For the first term in (5),
apply Lemma 2. This yields equation (L6).
To obtain (L6'), notice that for
[p(0)-p(0[')]<TJ* + e/3.
II I
sufficiently large, by the definition of TJ*,
Thus the second term of the r.h.s. of equation (5) is again
deterministically bounded, this time by
7]* +e/3.
Combining this with the bounds for the first
term by applying Lemma 2, as before, we get equation (L6'). 0
Lemma 7:
(7.1) 0 < P ~ 1
(7.2) 0 < PF + Pa ~ ~ always.
Further, 0 < PF + Pa ~ 1 if F and G are
continuous.
(7.3) 0 < CT' P ~ ~
Proof of 7.1: Clearly, each one of J1.F & J1.a is bounded by 1. Since J1. is a convex combination
of JJ F & JJa' JJ is also bounded by 1. We will now establish that JJ F + JJa >0. That J1. > 0
will follow since JJ is a non-trivial convex combination of JJF & J1.a •
By assumption we have A:={x E IR:
either
J AdF(x) > 0 or J AdG(x) > O.
I F(x)-G(x) I
> O}
# ¢J.
It suffices to show that
The case where A contains a discontinuity point of F or
G is trivial, so we will now presume that F and G are continuous at each x E A.
Select Xo E A with (say) F(xo) > G(xo)'
'fix E (y, xoD is non-empty, by continuity.
xo] ~ A and therefore
JAdF(x)
also (Yo, xoJ ~ A, yielding
Then
Denote yo:=inf{y E u}.
2:: F(xo) > G(xo) 2::
J AdF(x)
u :={y E (-00,
o.
xo):
F(x) > G(x)
If yo=-oo, then (-00,
If Yo> -00, then F(yo)
::5 G(yo) and
2:: F(xo)-F(yo) 2:: F(xo)-G(yo) > G(xo)-G(yo) 2:: O.
Proof of 7.2: It remains to prove that J1.F + J1.a ::5~. Some of the arguments used here were
suggested by Robert Lund.
Define L(·): =min{F( . ), G( .
Then, Land U are distribution functions, dL+dU
-69-
= dF+
n,
and U(·): = max{F( . ), G( .
dG, and U-L= I F-G
I.
n.
Thus,
given any two distribution functions F and G, we can construct a pair of distribution functions
Land U such that IJ F + lJa=1J L + lJu, and L ~ U. Then,
IJF + lJa
=IJL + lJu=
I (U-L) (dU+dL)
=IUdU- ILdU + IUdL- ILdL ~ 2- ILdL,
I UdL
since
&
I UdU
I LdU
are both bounded by 1, and
#21.11(d), in Billingsley (1986),
I LdL ~ t
can be ignored.
By Problem
and it follows that IJF + lJa ~~.
If F and G are continuous, so are the corresponding Land U. Again using Problem
f UdU = I LdL =~.
#21.11(d) cited above,
By
It follows that IJF + lJa ~
IJ ~ maxp(6),
definition,
From (7.2) above and since 0 < (T < 1, (7.3) follows.
d
o
E
Proposition
<II
1=1
£.
I
d
~[
A(~)}(IJF + lJa),
II (z.-£.)]
i=l
I
and
0
therefore
0
d
II z· -
i=l
I UdL ~ 1.
I
I
~
d
E £i <- d· maz~£i'
i=1
I
where
£tgQfr Some of the arguments used here were suggested by Allen Roginsky.
d
II (z .-£.)
i=l
I
1
= xl'
=
=.
II (x .-£.) -
i~2
1
(1'
II (x ._{.)
j~2
1
1
x1 ·x 2 • II (x._(.) X '(2' IT (x._(.) - ( I ' II (x._(·)
1
i>3 1
1
;>3 1
1
j>2 1
1
d
-d
II x· X1 ·X2 . . . . x '-1'( " .
(X'-(j),
) } 1~}+1
j=l
where we write, xo=
j
d
n
2:
,=1 '
Therefore ,
1
II
> d +1
d
(Xj-(i)=l.
d
II x·I - . -II 1 (x I._(.)
= ~
l.~
.- 1
1-
Xl' X2' , .• X· l ' (
}-
J=l
1-
d
Now, r.h.s ~.2: (j' since
) =1
Also, r.h,s.
.
J'
Xj, (.l:j-(j) ~
d
!,
> II. + 1 (x I._(.)
I'
I_}
~ Xl • X2 ' ••• xd-1 • (d ~ j ~ 1(j ,
Proposition.5.;, Consider vectors ~.
~
I
since
a:< a·I<-a:+
1I
(j ~ Xj'
! + in CUd'
is denoted by ail etc. Assume that 'Vi:
(6)
1 'Vi.
< 6:- < 6· < 6:+ and
-1-
-70-
I-I
'Vi,
0
where the i th component of
min{(I1'11:-)
(11:+
-11.)
(6.-b:-)
(b.+
-b.)}
>.1.
I
I'
I
I'
I
I'
I
I
ni
(7)
fr.22.f:.
For any sets Band Q, Q=(Q n B) U (Q nBC). Further given A
(Q n B)
~
B
C and (Q n BC) ~ BC~ Ac. Therefore in order that Q
~
that Q n B ~ AC and Q n B C
~
~
B
~
C, it follows that
ACn C, it is sufficient
C.
ef.2+,
By (6) and (7),
~
!-]~e~, !])~e~-,
!+].
By the above observation,
we
need to prove that:
(8) ~I(ef.2,.eJ)
~ I( ef.2
nef.2,.eJ ~ {ef.2+, rn c , and
' ! ]) n {ef.2 ' ! n
C
~
ef.2 -,! +].
We shall establish the first. The proof of the second assertion is similar.
Consider any £. E l.h.s. of (8). Then, .£ E ef.2
£. E ~I(e~, !]):::} 3! such that Yi
Now if Yi < ai ' then, xi :5 Yi
Yi
> bi ' then,
xi ~ Yi-~i
+ ~i < ai + ~i
But lim
II I - 0 0
bi Vi and
but IYi-xil :5 ~i "Ii.
:5 a/ :::} £.
't e~ + ,!-].
On the other hand if
> bi-'~i ~ bi:::} £. 't ef.2 +,r]· 0
Proposition §.;. Condition R.2. holds
Proof:
't [ai,b i] for some i,
' !] :::} ai :5 xi:5
¢>
'1=0.
"=>": R.2. holds :::} lim 8( e, T 1)=0 for some sequence TIE cr I'
III ---'00
8(e, el ):5
",¢::": JT>:=Olim
I I I -00
lim
I I I -00
8(e, T I)' by definition of el , and hence "1= lim
8( e, el )=O:::}
II I - 0 0
lim
I I I -00
a( e, el )=0, since lim ace, el ) ~ 0,
II I
0
Furthermore, all the bounds in the in(."qualities are
attainable.
Proof: By definition "1:=
lim B( e,
II I --00
el ),
always
-00
:::}R.2. holds with 62 =0 for T I=e[.
Proposition 7: 0:5 '1* :5 '1 :5 u -1 '1*.
8(e, el )=0.
where el:=arg min B( e, Tj, and
T E
-il-
cr I
1]*:= lim [p(8) - p(8[')], where 8[':=arg min [p(8) - p(T)]
111-00
TeGJ 1
Since 8 l , 8[' E GJ l' using the definitions above and fl'Om Lemma 5, it follows that,
o$; [p(8) and taking
lim
111--00
p(8[')] $; [p(8) - p(8[)] $; 0(8, 8[) $; 0(8,8[') $;
throughout completes the proof.
By Proposition 6, the bounds are attainable.
-i2-
0
(1-1.
[p(8) - p(8[)]
REFERENCES
Asatryan, D. and Safaryan, I. (1986). Nonparametric methods for detecting changes in the properties of random sequences.
In Detection of Changes in Random Processes (L.
Telksnys, ed.), 1-13. Optimization Software, New York.
Basseville, M. and Benveniste, A., editors, (1986). Detection of Abrupt Changes in Signals and
Dynamical Systems, Springer-Verlag.
Beasley, J.D. and Springer, S.G. (1977).
The percentage points of the Normal distribution.
Applied Statistics, 26, 118-120
Besag, J. (1986). On the statistical analysis of dirty pictures. Journal of the Royal Statistical
Society, B, 48, 259-302.
Bhattacharyya, G.K. and Johnson, R.A. (1968). Nonparametric tests for shift at an unknown
time point. Annals of Mathematical Statistics, 39, 1731-43.
Bhattacharya, P.K. (1987). Maximum likelihood estimation of a change-point in the distribution of independent random variables: General multi-parameter case.
Journal of
Multi'llariate Analysis, 23, 183-208.
Bhattacharya, P.K. and Brockwell, P.J. (1976). The Minimum of an Additive Process with
Applications
to
Signal
Estimation
and
Storage
Theory.
ZeitschriJt
fur
Wahrscheinlichkeitstheorie und verwandte Gebiete, 37, 51-75.
Bhattacharya, P.K. and Frierson, D. (1981).
A non parametric control chart for detecting
small disorders. A nnais of Statistics, 9, 544-54.
Billingsley, P. (1986). Probability and measure, p.288. John Wiley.
Brodskij, B. and Darkhovskij, B.S. (1986). The a posteriori method of detecting the disruption
of a random field.
In Detection of Changes in Random Processes (L. Telksnys, ed.),
32-38. Optimization Software, New York.
Carlstein, E. (1988).
Nonparametric change-point estimation.
Annals of Statistics, 16,
188-197.
Carlstein, E. and Lele, S. (1990).
Nonparametric change-point estimation for data from an
ergodic sequence. Department of Statistics, Chapel Hill, North Carolina, Mimeo Series
#2020.
Chernoff, H. and Zacks, S. (1964).
Estimating the current mean of a normal distribution
which is subjected to changes in time.
Annals of Mathematical Statistics, 35, 999-
1018.
Cliff, A.D. and Haggett, P. (1988). A tlas of disease distributions: A nalytic approaches to
epidemiological data. Oxford: Basil, Blackwell Inc.
Cobb, G.W. (1978). The problem of the Nile: Conditional solution to a change-point problem.
Biometrika, 65, 243-51.
Csorgo, M. and Horvath, L. (1987). Nonparametric tests for the changepoint problem. Journal
of Statistical Planning and Inference, 17, 1-9.
Csorgo, M. and Horvath, L. (1988).
Handbook of Statistics,
Nonparametric methods for changepoint problems. In
Volume 7 (P. Krishnaiah and C.R. Rao, ed.s), 403-425.
Elsevier, The Netherlands.
Darkhovskij, B.S. (1976).
A non parametric method for the a posteriori detection of the
"disorder" time of a sequence of independent random variables.
and Its Applications, 21, 178-83.
-74-
Theory of Probability
Darkhovskij, B.S. (1984). On two estimation problems for times of change of the probabilistic
characteristics of a random sequence.
Theory of Probability and its Applications, 29,
478-487.
Darkhovskij, B.S. (1985). A nonparametric method of estimating intervals of homogeneity for
a random sequence. Theory of Probability and its Applications, 30, 845-849.
Darkhovskij, B.S. (1986).
A general method for estimating the instant of change in the
probabilistic characteristics of a random sequence.
In Detection of Changes an
Random Processes (L. Telksnys, ed.), 47-52. Optimization Software, New York.
Darkhovskij, B.S. and Brodskij, B. (1980). A posteriori detection of the "disorder" time of a
random sequence. Theory of Probability and its Applications, 25, 624-628.
Deshayes, J. and Picard, D. (1981).
Convergence de processus
a double
indice:
application
aux tests de rupture dans un modele. C.R. Acad. Sci. Paris, 292, 449-452.
Diimbgen, 1. (1988).
Die konvergenzrate der nichtparametrischen changepointschatzer von
Edward Carlstein. Diploma thesis, University of Heidelberg.
Dvoretzky, A., Kiefer, J., and Wolfowitz, J. (1956).
Asymptotic Minimax Character of
theSample Distribution Function and of the Classical l\tlultinomial Estimator. Annals
of Mathematical Statistics, 27, 642-669.
Elliott, T.J. (1988).
A proposed non parametric test statistic for the location of a single
change-point. Masters Thesis, Uni'versity of North Carolina, Chapel Hill.
Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7,
1-26.
Gastwirth, J .L. and Mahmoud, H. (1986).
An efficiency robust nonparametric test for scale
change for data from a gamma distribution. Tecll1lometrics, 28, 81-84.
-75-
Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian
restoration of images.
IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. PAMI-6, 721-741.
Haccou, P., Meelis, E. and van de Geer, S. (1988). The likelihood ratio test for the changepoint problem for exponentially distributed random variables.
Stochastic Processes
and Their Applications, 27, 1-19.
Haccou, P. and Meelis, E. (1988). Testing for the number of change points in a sequence of
exponential random variables. Journal of Statistical Computation and Simulation, 30,
285-98.
Haralick, R.M. and Shapiro, L.G. (1985). Image segmentation techniques. Computer Vision,
Graphics and Image Processing, 29, 100-132.
Hawkins, D.L. (1986). A simple least-squares method for estimating a change in mean. Communications in Statistics, B15, 655-79.
Hawkins, D.M. (1977). Testing a sequence of observations for a shift in location. Journal of
the A merican Statistical Association, 72, 180-86.
Henderson, R. (1986). Change-point problem with correlated observations, with an application
in material accountancy. Technometrics, 28, 381-90.
Hinkley, D.V. (1970).
Inference about the change-point in a sequence of random variables.
Biometrika, 57, 1-16.
Hinkley, D.V. (1972). Time-ordered classification. Biomet1'ika, 59, 509-23.
Hinkley, D.V. and Hinkley, E.A. (1970).
Inference about the change-point in a sequence of
binomial variables. Biometrika, 57, 477-88.
Hoeffding, W. (1963).
Probability Inequalities for Sums of Bounded Random Variables.
Journal of the American Statistical Association, 58, 13-30.
-76-
Hsu, D.A. (1979).
Detecting shifts of parameter in gamma sequences with applications to
stock price and air traffic flow analysis.
Journal of the American Statistical Asso-
ciation, 74, 31-40.
Kandel', Z. and Zacks, S. (1966). Test procedures for possible changes in parameters of statistical distributions occurring at unknown time points. Annals of Mathematical Statistics, 37, 1196-210.
Lombard, F. (1983). Asymptotic distributions of rank statistics in the change-point problem.
South African Statistical Joumal, 17, 83-105.
Lombard, F. (1987). Rank tests for change-point pl'Oblems. Biometrika, 74, 615-24.
Massart, P. (1990). The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality. Annals
of Probability, 18, 1269-83.
Miao, B.Q. (1988).
Inference in a model with at most one slope change-point.
Journal of
Multivariate Analysis, 27, 375-91.
Page, E.S. (1954). Continuous inspection schemes. Biometrika, 41, 100-115.
Page, E.S. (1955).
A test for a change in a parameter occurring at an unknown point.
Biometrika, 42, 523-26.
Page, E.S. (1957).
On problems in which a change in a parameter occurs at an unknown
point. Biometrika, 44, 248-52.
Perez-Abreu, V. (1989). Estimacion nonparametrica del tiempo de cambio para distribuciones
discretas. Aportaciones Matematicas, Societe Matematicas Mexicana, 6, 289-301.
Petitt, A.N. (1979).
A non-parametric approach to the change-point problem.
Applied
Statistics, 28, 126-35.
Petitt, A.N. (1981).
Posterior probability for a change-point using ranks.
443-50.
-77-
Biometrika, 68,
Picard, D. (1985). Testing and estimating change-points in time series. Advances in Applied
Probability, 17, 841-867.
Riggan, W., Creason, J., Nelson, W., Manton, K., Woodbury, M., Stallard, E., Pellom, A.,
and Beaubier, J. (1987).
Volume IV:
U.S. Cancer AIortality Rates and Trends, 1950-1979,
Maps, U.S. Environmental' Protection Agency, Washington, D.C.:
U.S.
Government Printing Office.
Ripley, B.D., and Rasson, J. (1977).
Finding the Edge of a Poisson Forest.
Journal of
Applied Probability, 14, 483-491.
Rudemo, M., Skovgaard, 1., and Stryhn, H. (1990). Maximum likelihood estimation of curves
in images.
Report 90-4, Department of Mathematics and Physics, Royal Veterinary
and Agricultural University, Denmark.
Scariano, S.M. and Watkins, T.A. (1988).
Nonparametric point estimators for the change-
point problem. Comm'unications in Statistics Theory and AIethods, 17, 3645-75.
Schectman,
E.
(1982).
A
non parametric
test
for
detecting
changes
III
location.
Communications in Statistics. All (13), 1475-82.
Schectman, E. and Wolfe, D. (1985). Multiple change-point problem - nonparametric procedures for estimation of the points of change. Communications in Statistics, B 14, 61531.
Schectman, E. and Wolfe, D. (1988).
Distribution free test for the change-point problem.
American Journal of Mathematical and Management Sciences, 8, 93-119.
Sen, A. and Srivastava, M.S. (1975).
On tests for detecting changes in mean.
Statistics, 5, 1107-23.
-78-
Annals of
Sen, P.K. (1980). Asymptotic theory of some tests for a possible change in the regression slope
occurring at an unknown time-point.
Zeitschrijt filr Wahrscheinlichkeitstheorie und
verwandte Gebiete, 52, 203-218.
Sen, P.K. (1983). Some recursive residual rank tests for change points. Recent Advances in
Statistics: Papers in Honor of Herman Chernoff on His Sixtieth Birthday (editors:
Rizvi, M.H., Rustagi, J.S., Siegmund, D.), 371-91, Academic Press, New York.
Shaban, S. (1980).
Change-point problem and two-phase regression:
An annotated biblio-
graphy. International Statistical Review, 48, 83-93.
Siegmund, D. (1986). Boundary crossing probabilities and statistical applications. A nnals of
Statistics, 14, 361-404.
Smith, A.r.M. (1975). A Bayesian approach to inference about a change-point in a sequence
of random variables. Biometrika, 62, 407-16.
Smith, A.r.M. (1980).
Change-point problems-approaches and applications.
Trabajos
Estadistica, 31, 83-98.
Snow,
J.
(1855).
On
the
Mode
of
Communication
of
Cholera
(2 nd
ed.),
London: John Churchill.
Srivastava, M.S. and Worseley, K.J. (1986). Likelihood ratio tests for a change in the multivariate normal mean. Journal of the American Statistical Association, 81, 199-202.
Telksnys, L., editor, (1986).
Detection of Changes
In
Random Processes, Optimization
Software, New York.
Wolfe, D.A. and Schectman, E. (1984).
Nonparametric statistical procedures for the change-
point problem. Journal of Statistical Planning and Inference, 9, 389-96.
Yakimovsky, Y. (1976). Boundary and object detection in a real world image. Journal of the
Association for Computing Machinery, 23, 599-618.
-79-
APPENDIX
Source Code For The linear Bisection Problem (Simulation)
c
We shall start out by assigning parameters:
c
NREP = # replications, NN = # data points, NC = # of Columns,
c
NR = # of Rows, NTN = # candidate boundaries
c
PROGRAM BISECT
PARAMETER (NREPLIC =1000, NRDATA =05, NCDATA =05, NNDATA =025)
PARAMETER (KTHETA = 5, THETA1 = 0.25, THETA2 = 0.85)
c
c
The
matrix
IND(NNx2)
keeps
track of
the
original
indices.
IND(j,l)
c
and IND(j,2) values denote the original row and column indices of
c
the corresponding X(j)s. As the X(j)s are generated, they will be
c
sorted and the original indices stored in IND(j,l) and IND(j,2).
c
c
Type and dimension declarations
c
COMMON X(NNDATA),ITAR«NNDATA+1),2), XN
COMMON jINDEXj IND«NNDATA+l),2)
COMMON JENSj NC, NR, NN
COMMON jTHETAj KTU, TU1, TH2
COMMON JABj A,B
COMMON jCOUNTj NCOUNT, V(NNDATA)
DIMENSION
DT(3), DELTA(NREPLIC,NNDATA)
REAL DELTA-MEAN(3), DELTA_SD(3), DELTA_LOW(3),
DELTA-HIGH(3), MAXD(3)
INTEGER T(3,3), ICON(NRDATA,NCDATA)
c
c
Now for some routine initialisations:
c
TZERO
= CPUTIME(O.O)
NR
= NCDATA
= NRDATA
NN
= NNDATA
NC
NREP
= NREPLIC
KTH
= KTHETA
TH1
= THETAhNR
= THETA2*NC
NCOUNT = 0
NLESS = 0
TH2
ISEED
= 15
DO 1=1,3
DELTA_MEAN(I)
DELTA-SD(I)
DELTA_LOW(I)
DELTA_HIGH(I)
= 0.0
= 0.0
= 0.5
= 0.0
ENDDO
DO IR
= 1,NR
DO IC
= 1,
NC
-81-
ICON( IR, IC) =
a
ENDDO
ENDDO
DO 100 IREP = 1, NREP
ISEED = 15 + IREP
DO 50 I
= 1,3
DELTA(IREP, I) = 0.0
MAXD(I) = 0.0
50
CONTINUE
c
cGENERATE THE RANDOM NUMBERS
c
CALL RANV(ISEED,NN,V)
N
=a
IF (KTH.LE.4) CALL TRIARM(KTH,TH1,TH2)DO 200 IR= 1, NR
DO 200 IC= 1, NC
N =N + 1
XN = GEN(IR,IC,N)
IF (N.EQ.1) THEN
J
=1
GO TO 181
ELSE
DO 150 J = N, 2, -1
IF (XN.GT.X((J-1))) GO TO 179
150
CONTINUE
ENDIF
-82-
c
c Here XN is inserted as the jth element into the sequence
c X(1).le ..... X(J-1).le.X(J).le ... X(NN-1).
XN has original index N.
c It also updates the corresponding indices In
c { IND(J,l) : 1.le.J.le.NN } so that IND(J,l) is the original index
c of the jth ranked X(J).
c
179
IF (J.LT.N) THEN
DO 180 IJ
= N,
X(IJ)
IND(IJ,
(J+1), -1
= X(IJ-1)
1) = IND«IJ-1),
1)
= IND«IJ-1),
2)
IND(IJ, 2)
180
CONTINUE
ENDIF
181
X(J)
= XN
IND(J,l)
= IR
IND(J,2)
= IC
200
CONTINUE
IF (NCOUNT*(NN-NCOUNT).EQ.O) THEN
write (6,51)ncount
51
format (i4,"
True boundary is trivial")
STOP
ENDIF
c
c
At this point we have data x(1).le.x(2).le.le.x(nn). We
c
will now calculate ITAR(.,l) : ITAR(j,l) is the number of
-83-
c
of x(i)'s above x(j) in the sorted order
c
In the case of no ties ITAR(j,1)
=0
for all j.
c
ITAR(NN,1)
DO 300 J
=0
= (NN-1),
1, -1
IF (X(J).EQ.X((J+1))) THEN
ITAR(J,1)
= ITAR((J+1),1)
+ 1
ELSE
ITAR(J,1)
=0
ENDIF
CONTINUE
300
TIME
= CPUTIME(TZERO)
c
c
Now the rank of X(j) (defined as the the number of X(i)'s in
c
the data that are
less
than or equal
to X(j)
),
is
(j +
ITAR(j,l)).
c
Its original index in the data is (IND(j,1), IND(j,2)).
c
c
Enter the candidate boundaries T:
c
ITAR(j,2), will for each boundary IT keep track of the rank of
X(j)
c
within IT .
c
DO 400 K = 1, 6
DO 400 IT1
= 1,
NR
DO 400 IT2
= 1,
NC
-84-
RT1
RT2
= float(IT1)
= float(IT2)
IF (K.LE.4) THEN
CALL TRIARM(K,RT1,RT2)
IF «A*B).EQ.O) THEN
write (6,71) k, it1, it2
71
format (5x,"The cando bndry.:(",3i3,") is trivial")
Go to 400
ENDIF
ELSEIF «IT1*IT2).EQ.(NN*NN»
THEN
write (6,71) k, it1, it2
Go to 400
ENDIF
ITAR(0,1)
=0
ITAR(0,2)
=0
DO 350 J
= 1,
NN
IF (ITAR«J-1),1).EQ.0) THEN
ITAR(J,2)
= ITAR«J-1),2)
+ MOD_ID(K,RT1,RT2,J,-1)
IF (ITAR(J,1).GT.0) THEN
DO 370 JJ
370
ITAR(J,2)
= 1, ITAR(J,1)
= ITAR(J,2) + MOD_ID(K,RT1,RT2,
ENDIF
ELSE
ITAR(J,2)
= ITAR«J-1),2)
ENDIF
350
CONTINUE
-85-
(J+JJ),-1)
IF(ITAR(NN,2)*(NN-ITAR(NN,2».EQ.0) THEN
wri te (6,71) k, it 1, i t2
Go to 400
ENDIF
c
c
5i (l.le.i.le.3), is the mean-dominant norm of the d's for this
IT.
(i.e) 5i = 5 ( d(l,T), d(2,T),
c
, d(nn, T».
From 5i we
will
c
calculate the D(T)'s.
Notice that IND(nn,3)
= #(T).
c
51
= 0.0
52
= 0.0
53
= 0.0
DO 380 J
= 1,NN
D = AB5(float(ITAR(J,2»/float(ITAR(NN,2»
- float(J+ITAR(J,1)-ITAR(J,2»/float(NN-
x
ITAR(NN,2»)
52
= 51
= 52
53
= MAX
51
380
+ D
+ D*D
(53,D)
CONTINUE
RLAM
= 5QRT(float(ITAR(NN,2»*float(NN-
ITAR(NN,2»/float(NN*NN»
DT(l)
= RLAM
* 51/NN
DT(2)
= RLAM
* 5QRT(52/NN)
-86-
•
DT(3)
= RLAM
• 83
c
c
We now update MAXD(i)'s and T(i)'s so that:
MAXD(i) = max(D(T) : T considered upto now), and T(i) = argmax
c
MAXD(i).
c
DO 390 I = 1, 3
IF(DT(I).GT.MAXD(I»
THEN
T(I,l) = K
T(I,2) = IT1
T(I,3) = IT2
ENDIF
MAXD(I)
= MAX
(MAXD(I),DT(I»
write(6,74)irep,t(i,1),t(i,2),t(i,3),maxd(i)
format (5x, ".** irep=", 414,"
74
390
400
",f8.6)
CONTINUE
CONTINUE
c
c
At the end of loop 400, we have our estimators T(i)
c
estimating theta, using the different mean-dominant norms.
c
We shall now calculate, for the jth replication (l.le.J.NR)
c
{delta(i,J): i= 1,2,3}. delta (i,J) = dist (theta, T(i»
c
the jth replication.
c
DO 500 N = 1, NN
IF (KTH.LE.4) CALL TRIARM(KTH,TH1,TH2)
-87-
1= 1,2,3
for
J = MOD_ID(KTH,TH1,TH2, N,-l)
•
DO 500 I = 1, 3
RT1 = float(T(I,2))
RT2 = float(T(I,3))
IF (T(I,1).LE.4) CALL TRIARM(T(I,1),RT1,RT2)
IF (J.EQ.MOD_ID(T(I,1),RT1,RT2,N,-1)) THEN
DELTA(IREP,I)= DELTA(IREP,I)+1.0
ENDIF
500
CONTINUE
DO I =1, 3
DELTA(IREP,I) = DELTA(IREP,I)jNN
DELTA(IREP,I) = MIN (DELTA(IREP,I), (l-DELTA(IREP,I))
write(6,83)IREP,T(I,1),T(I,2),T(I,3),DELTA(IREP,I)
83
format(I4,2x,3(14,2x),f8.6,)
ENDDO
write(6,83)IREP,T(1,1),T(1,2),T(1,3),DELTA(IREP,1)
c
c
We will now calculate some summary statistics.
c
DO 600 I =1, 3
DELTA-MEAN(I)
= DELTA_MEAN(I) + DELTA(IREP,I)
DELTA_SD(I)
= DELTA_SD(!) + DELTA(IREP,I)**2
DELTA_LOW(I)
= MIN (DELTA_LOW(I), DELTA(IREP,I))
DELTAJIIGH(I)
600
100
= MAX (DELTA_HIGH(I), DELTA(IREP,I))
CONTINUE
CONTINUE
-88-
...
wri te (6, 85 )
format(70(" "))
85
write (6,81) kth, th1, th2
write (6,82) nr, ne, nn, nrep
DO 700 I =1, 3
write (6,85)
DELTAJlEAN(I)
= DELTA-JIEAN(I)/NREP
DELTA-SD(I)
=
SQRT«DELTA-SD(I)-
NREP*DELTAJlEAN(I)**2)/NREP)
write(6,86)i,delta-mean(i),delta_sd(i),delta_low(i),delta_high(i)
86
format ("DELTAS :FOR NORM ",Il,"
88
format (20x," MEAN_REP
700
CONTINUE
SD-REP
",4(f8.6,"
LOW_REP
"))
HIGH_REP")
write(6,92)nrep,nn,delta_mean(1),delta_mean(2),delta_mean(3)
write(6,93)delta_sd(1),delta_sd(2),delta_sd(3)
do i=1,3
delta_sd(i) = delta_sd(i)/sqrt(float(nrep))
enddo
write(6,93)delta_sd(1),delta_sd(2),delta_sd(3)
91format (2I6,3x,f19.11,3x,f16.14)
92format(2I6,2x,3(f8.6,"
93format(15x,3(f8.6,"
"))
"))
e
STOP
END
-89-
Function GEN(.,.,.)
REAL FUNCTION GEN(Ll, L2, N)
PARAMETER (NNDATA =025)
COMMON /THETA/ KTH, THl, TH2
COMMON /AB/ A,B
COMMON /COUNT/ NCOUNT, V(NNDATA)
XX
= YeN)
IF (MOD_ID(KTH,THl,TH2,Ll,L2).EQ.l) THEN
GEN = SIGN(l.O,XX -.5)*((ABS(XX-.5)/.232376)**(1.0/3.0))
NCOUNT = NCOUNT + 1
ELSE
GEN = GAUSS(XX)
ENDIF
RETURN
END
-90-
Subroutine MOD _10(.,.,.,.,.)
cldentification function for the two-dim bisection problem
INTEGER FUNCTION MOD_ID(K, IR, IC, IRANK, IDUMMY)
PARAMETER (NNDATA =025)
COMMON /INDEX/ IND((NNDATA+l),2)
COMMON /ENS/ NC, NR, NN
COMMON /AB/ A,B
INTEGER XA, VB, ZI, Z2
REAL IR, IC
cThis function returns "1" if the Jth ranked X(J) is "inside"
cthe boundary defined by (K, IR, IC), and "0" otherwise.
c Here for the bisection problem, "inside" is defined as
c the CORNER/LEFT/UPPER side of the line formed by
c connecting IR and IC,
c in the direction suggested by K. The original index of X(j)
c is (IND(J,I), IND(J,2».
c This subroutine may be accessed by gen. f to determine the mod_id
with
c respect to the true boundary
THETA, or by the main routine to
classi fy
c
points according to
the candidate
boundaries.
accessed by
c THETA and -1 if accessed by the candidate boundaries.
IF (IDUMMY.LT.O) THEN
Zl = IND(IRANK,I)
-91-
IDUMMY > 0
if
Z2
= IND(IRANK,2)
ELSE
Z1
Z2
= IRANK
= IDUMMY
ENDIF
cThe cases K = 1,4 can be reduced to the same problem by
=0
MOD_ID
IF (K.LE.4) THEN
IF (K.EQ.1) THEN
XA
= NR
YB
= Z2
- Z1
ELSEIF (K.EQ.2) THEN
XA
= NC
- Z2
YB
= NR
- Zl
ELSEIF (K.EQ.3) THEN
YB
= Z2
= Zl
XA
= Zl
= NC
- Z2
XA
ELSE
YB
ENDIF
IF«XA.LE.A).AND.(YB.LE.(B-XA*B/A»)
MOD_ID
=1
ELSE
IF (K.EQ.5) THEN
IF(Z2.LE.(IC+(IR-IC)*float(Zl)/float(NR»)
~IOD_ID
= 1
-92-
Subroutine Triarm( ., .,.)
SUBROUTINE TRIARM(K,Tl,T2)
COMMON
COMMON
IENSI NC,
IABI A,B
NR, NN
IF (K.EQ.l) THEN
A = NR - Tl
B
= T2
ELSEIF (K.EQ.2) THEN
A = NC - T2
B
= NR
- Tl
ELSEIF (K.EQ.3) THEN
A = T2
B
= Tl
ELSE
A = Tl
B
= NC
- T2
ENDIF
RETURN
END
-94-
Function Gauss
[Algorithm due to Beasley and Springer (1977)]
FUNCTION GAUSS"(U)
DIMENSION A(4),B(4),C(4),D(2)
DATA A/2.50662823884EO,-18.61500062529EO,41.39119773534EO,
#-25.44106049637EO/
DATA B /-8. 47351093090EO,
23.08336743743EO, -21.06224101826EO,
#3. 13082909833EO/
DATA C /-2.78718931138EO,
-2. 29796479134EO ,
4.85014127135EO,
#2. 32121276858EO/
DATA D / 3.54388924762EO,
IFAULT =
1.63706781897EO/
°
Q = U - .5
IF (ABS(Q).LE.SPLIT) THEN
R = Q**2.0
GS= Q*«(A(4)*R+A(3»*R+A(2j)*R+A(1)/
#
««B(4)*R+B(3»*R+B(2»*R+B(1»*R+l.0)
ELSE
R
=U
IF (Q.GT.O.O) R = 1.0-U
IF (R.GT.O.O) THEN
R = SQRT(-LOG(R»
GS=«(C(4)*R+C(3»*R+C(2»*R+C(1»/«D(2)*R+D(1»*R+l.0)
GS = SIGN(1.0,Q)*GS
ELSE
IFAULT = 1
-9.5-
WRITE (6,*) "IFAULT
GS
="
,IFAULT
= -25.0
ENDIF
ENDIF
GAUSS
= GS
RETURN
END
-96-
Source Code For Lipschitz Program (Data)
PROGRAM LIPSCHITZ
PARAMETER (NCDATA =64, NRDATA =40, NNDATA =1574)
PARAMETER (MCCAND =64, MRCAND =40)
PARAMETER (LIPCON=2, LENBLOCK=7)
DATA Iwri te /26/
DATA NORM /1/
DIMENSION
TSTAR(3,MCCAND),T(MCCAND),
NAME(MCCAND),
IND(NNDATA,2)
DIMENSION X(NNDATA),Y(NRDATA,NCDATA),ITAR(NNDATA,2),DT(3)
INTEGER Y, XN, X, FLAG, TSTAR, T
REAL
MAXD(3)
INTEGER CANDID
COMMON /CANDY/ MC, IND, T
COMMON /DEETY/NN, ITAR, DT, 51, 52, 53, NODT
NC = NCDATA
NR = NRDATA
NN = NNDATA
MC = MCCAND
MR = MRCAND
N
=0
LBLOCK = LENBLOCK
LPCON
= LIPCON
NEGLPCON = (-l)*LPCON
read(15,1) (T(I), 1=1,32)
-97-
read(15,1) (T(I), 1=33,64)
DO 10 IR = NR, 1, -1
10
read(15,2) (Y(IR,IC), IC=l.NC)
1
format(32(i2))
2
format«NC>i1)
write(Iwrite,1000)
wri tee Iwri te, *) "THE RUNTIME PARAMETERS ARE :"
write(Iwrite,73) nr,nc,nn
write(Iwrite,74) Iblock,lpcon
write(Iwrite,77) (t(i), i=1,32)
wri tee Iwri te, 76)
write(Iwrite,77) (t(i), i=33,64)
write(Iwrite,999)
write(Iwrite,*)"THE DATA GIVEN :"
write(Iwrite,999)
do 20 ir=nr,l,-l
nir = nr - ir + 1
20
write(Iwrite,521) nir,(y(ir,ic),ic=l,nc), ir
DO 200 IR= 1, NR
DO 200 IC= 1, NC
XN = Y(IR,IC)
If(XN.eq.O) goto 200
N =N + 1
IF (N.EQ.1) THEN
J
=1
GO TO 181
-98-
ELSE
= N,
DO 150 J
2, -1
IF (XN.GT.X«J-1») GO TO 179
150
CONTINUE
ENDIF
179
IF (J.LT.N) THEN
= N,
DO 180 IJ
(J+1), -1
= X(IJ-1)
X(IJ)
IND(IJ, 1)
= IND«IJ-1),
1)
IND(IJ, 2)
= IND«IJ-1),
2)
180
CONTINUE
ENDIF
181
= XN
X(J)
IND(J,l)
= IC
IND(J,2)
= IR
200
CONTINUE
IF(N.NE.NN) then
write (Iwrite,64) n, nn
64
format ("N and NNDATA do not match. N = ",i5, ",NNDATA
stop
endif
ITAR(NN,l)
=0
DO 300 J
= (NN-1),
1, -1
IF (X(J).EQ.X«J+1») THEN
ITAR(J,l)
= ITAR«J+1),1)
+ 1
ELSE
-99-
= ",i5)
ITAR(J,I) : 0
ENDIF
300
CONTINUE
CALL DEETEE
DO 5 I : 1,3
MAXD(I) : DT(I)
do 5 k : l,mc
TSTAR(I,k) : T(k)
5 CONTINUE
DO 125 ISRIFT : 2, (LDLOCK+l)
INDBLOC : 0
99
ISHFLAG: 0
DO 100 MLOW : ISHIFT, (ISRIFT+56),LBLOCK
INDBLOC : INDBLOC + 1
MHIGH : MLOW + LBLOCK - 1
IF (MHIGH.GT.MCCAND) MUIGR : MCCAND
DO 310 I : MLOW,
~1HIGII
NAME(I) : NEGLPCON
310
CONTINUE
NAME(MHIGH) : NAME(MHIGH) - 1
311
FLAG: 0
DO 330 I : MHIGH, MLOW,-l
IF(NAME(I).LT.LPCON) THEN
FLAG:I
GO TO 331
ENDIF
-100-
330
CONTINUE
331
IF (FLAG.EQ.O) GO TO 400
J
= FLAG
NAME(J)
= NAME(J)
+ 1
IF (J.EQ.MHIGH) GO TO 341
DO 340 I
NAME(I)
= (J+1), MHIGH
= NEGLPCON
340
CONTINUE
341
CONTINUE
DO 343 I
= MLOW,
MHIGH
IF( 1. NE. 1) THEN
T(I)
= T((I-1»
+ NAME(I)
ENDIF
IF ((T(I).LT.O).OR.(T(I).GT.MR»
THEN
GO TO 311
ENDIF
343
CONTINUE
DO 344 1= 2, MCCAND
ITMOD
= T(I)
- T((I-1»
IF ((ITMOD.LT.NEGLPCON).OR.(ITMOD.GT.LPCON»
GO TO 311
344
CONTINUE
NODT
=0
CALL DEETEE
IF (NODT.EQ.1) THEN
write (Iwrite,71) (T(k), k=1,mc)
-101-
71
format ('The candidate boundary
IS
trivial. T is ',<MC>I3)
GO TO 311
ENDIF
IF(DT(NORM).GT.MAXD(NORM)) THEN
ISHFLAG = 1
DO 394 I = 1,3
MAXD(I) = DT(I)
DO 395 k = 1,mc
395
TSTAR(I,k) = T(k)
394
CONTINUE
ENDIF
GO TO 311
400
CONTINUE
DO 410 KK = 1,MCCAND
410
T(KK) = TSTAR(NORM,KK)
write (Iwrite,*) "MLO\¥ =
",1111010."
MIIIGH = ",mhigh
write(Iwrite,77) (tstar(l,kk), kk=1,32)
wri tee Iwri te, 76)
write(Iwrite,77) (tstar(1,kk), kk=33,64)
write(Iwrite,78) maxd(1)
100
CONTINUE
IF (ISHFLAG.GT.O) THEN
INDBLOC = 0
GO TO 99
ENDIF
125 CONTINUE
-102-
write(Iwrite,999)
write(Iwrite,*) "THE FINAL PICTURES ."
write(Iwrite,999)
do 515 i = 1,3
write(Iwrite,*) "CRITERION
"
I
do 520 k = 1,mc
t(k) = tstar(i,k)
520
continue
do 530 j = 1,nn
y(ind(j,2),ind(j,1)) = candid(j) + 1
if(y(ind(j,2),ind(j,1)).eq.2) then
y(ind(j,2),ind(j,1)) = 4
endif
530
continue
write(Iwrite,77) (tstar(i,kk), kk=1,32)
write(Iwrite,76)
write(Iwrite,77) (tstar(i,kk). kk=33,64)
wri tee Iwri te, 78) maxd( i)
do 540 ir=nr,l,-l
nir = nr - ir + 1
write(Iwrite,521) nir,(y(ir,ic),ic=l,nc), lr
540
continue
write(Iwrite,1000)
515
continue
73
format("NR=",i4,", NC=",i4, ", NN=",i6)
74
format("LBLOCK = ",i2,3x,"LPCON = ",i2)
-103-
76
format (32(" ."»
77
format (32(i2»
78
format (5x,'MAXD for this estimator :',2x,f15.12)
521
format(i2,'**',<NC>i1,'**',i2)
999
format(x)
1000
format(70("*"»
STOP
END
-104-
Subroutine Candid(.)
INTEGER FUNCTION CANDID(k)
PARAMETER (NCDATA =64, NRDATA =40, NNDATA =1574)
PARAMETER (MCCAND =64, MRCAND =40)
INTEGER T(MCCAND), IND(NNDATA,2)
COMMON jCANDYj MC, IND, T
CANDID=O
I = IND(K,1)
J = IND(K,2)
IF (I.GT.MC) I = Me
IF (1. GT . 0) THEN
IF (J.LE.T(I»
CANDID=1
ENDIF
RETURN
END
-105-
Subroutine Deetee
SUBROUTINE DEETEE
PARAMETER (NCDATA =64, NRDATA =40, NNDATA =1574)
PARAMETER (MCCAND =64, MRCAND =40)
DATA Iwrite/16/
DIMENSION T(MCCAND), ITAR(NNDATA,2), DT(3) , IND(NNDATA,2)
INTEGER CANDID, T
COMMON /DEETY/NN, ITAR, DT, 51, 52, 53, NODT
COMMON /CANDY/ MC, IND, T
ITAR(1,2) = CANDID(1)
IF (ITAR(1,1).GT.0) THEN
DO 365 JJ = 1, ITAR(1,1)
365
ITAR(1,2) = ITAR(1,2) + CANDID«l+JJ))
ENDIF
DO 350 J = 2, NN
IF (ITAR«J-1),1).EQ.0) THEN
ITAR(J,2) = ITAR«J-1),2) + CANDID(J)
IF (ITAR(J,1).GT.0) THEN
DO 370 JJ = 1, ITAR(J,l)
370
ITAR(J,2) = ITAR(J,2) + CANDID«J+JJ))
ENDIF
ELSE
ITAR(J,2) = ITAR«J-1),2)
ENDIF
350
CONTINUE
-106-
IF«ITAR(NN,2)*(NN-ITAR(NN,2»).EQ.0) THEN
NODT
=1
RETURN
ENDIF
51
= 0.0
52
= 0.0
= 0.0
53
DO 380 J
= 1,NN
D = AB5(float(ITAR(J,2»/float(ITAR(NN,2»
- float(J+ITAR(J,1)-ITAR(J,2»/float(NN-
x
ITAR(NN,2»)
51
= 51
52
= 52 + D*D
= MAX (53,D)
53
380
+ D
CONTINUE
DT(l)
= float(ITAR(NN,2»*float(NN-ITAR(NN,2»/float(NN*NN)
= RLAM * 51/NN
DT(2)
= RLAM
* 5QRT(52/NN)
DT(3)
= RLAM
* 53
RLAM
RETURN
END
-107-
Source Code For The Bootstrap Calculations (Simulation)
(For Subroutines Gauss, Triarm and
Mod-id see under BISECTION)
PROGRAM MAIN
PARAMETER (NREPLIC=13l,NRDATA=15,NCDATA=15,NNDATA=225,NNBOOT=1000)
COMMON /AB/ NCOVNT, A, B
COMMON /CEST/ NORM
COMMON/CBOOT/ISEED,NVPP,NLOW,EFF(NNDATA),GEE(NNDATA)
COMMON /ENS/ NC, NR, NN
COMMON /INDEX/ IND«NNDATA+l),2)
DIMENSION X(NNDATA),DT(3),ICON(NRDATA,NCDATA),Y(NRDATA,NCDATA)
DIMENSION V(NNDATA)
REAL RTHETA(3), DELTA(NREPLIC,NNDOOT)
REAL DELTA-JIEAN(3), DELTA_SD(3), DELTA_LOW(3), DELTA-"IGH(3)
INTEGER THETA(3), THAT(3). T2HAT(NNBOOT,3), TEMP(3), JT(NNDATA)
INTEGE.. LCON, VCON
DATA RTIIETA/5.0, 0.40, 0.70/
DATA NORM/l/
DATA Iwrite/16/
DATA Iwrite2/26/
DATA ISEED/60/
DATA ALPHA/0.10/
NC = NCDATA
NR = NRDATA
NN = NNDATA
NBOOT = NNBOOT
-108-
NREP = NREPLIC
write (Iwrite,81) (RTHETA(I), 1=1,3)
write (Iwrite2,81) (RTHETA(I), 1=1,3)
81
format
(~True
boundary
~,
f3.1, 2(f10.6,"
H))
write (Iwrite,82) nr, ne, nn, nrep, nboot
82
format("nr,ne,nn nrep and nboot
~,
5(14,"
H))
write (Iwrite,85)
THETA(l) = INT(RTHETA(l))
THETA(2) = INT(RTHETA(2)*NR)
THETA(3) = INT(RTHETA(3)*NC)
e
cSHOV THE ORIGINAL THETA:
e
write(Iwrite,*)""
write(Iwrite,*)""
write(Iwrite2,*)~"
write (Iwrite,83) (THETA(I), 1=1,3)
write (Iwrite2,83) (THETA(I), 1=1,3)
83
format ("THE ORIGINAL BOUNDARY ON THE GRID"
CALL TRIARM(THETA)
DO IR = 1,NR
DO IC = 1,NC
ICON(IR,IC) = MOD_ID(THETA,IR,IC)
ENDDO
ENDDO
DO JR = NR,l,-l
-109-
3(13,"
H))
write(Iwrite,199) JR,(ICON(JR,JC),JC=l,NC)
write(Iwrite2,199) JR,(ICON(JR,JC),JC=l,NC)
199
forrnat(I2,"** ",<NC>(i1," "))
ENDDO
DO 100 lREP = 1, NREP
NLESS
DO I
=0
=1 ,
THAT(I)
3
= 0
ENDDO
write(Iwrite,86)
86
forrnat(50("="))
CALL RANV(ISEED,NN,V)
IF (THETA(1).LE.4) CALL TRIARM(TIIETA)
NCOUNT = 0
N
=0
DO 110 IR= 1, NR
DO 110 IC= 1, NC
N =N + 1
XX
= YeN)
IF (MOD_ID(THETA, IR, IC). EQ. 1) THEN
X(N) = SIGN(1.0,XX -.5)*«ABS(XX-.5)/.232376)**(1.0/3.0))
NCOUNT
= NCOVNT
+ 1
ELSE
X(N)
= GAUSS(XX)
ENDIF
Y( IR, IC) = X(N)
-110-
110
CONTINUE
IF ((NCOUNT*(NN-NCOUNT».EQ.O) THEN
write(lwrite,*) NCOUNT,"The original boundary is trivial"
STOP
ENDIF
write(Iwrite,*)""
write(Iwrite,*) "The Original Data Is :"
do jr
= nr,l,-l
write(Iwrite,99) jr, (y(jr,jc),jc=l,nc)
99
format(i2,"**",<NC>(f7.2»
enddo
CALL ESTIMATOR (X, THAT)
c
cSHOW THE ESTIMATOR THAT:
write (lwrite,84) irep, (THAT(I), 1=1,3),
DEL(THETA,THAT)
write (lwrite2,84) irep, (TIIAT(I), 1=1,3),
DEL(THETA,THAT)
84
format ("THE ESTIMATOR, TIIAT: ",i3,3(I3,"
call triarm(that)
do jj
= 1,nn
icon(ind(jj,1),ind(jj,2»
= mod_id(that,jj,-l)
enddo
do jr = nr,l,-l
write(Iwrite,199) jr,(icon(jr,jc),jc=1,nc)
write(Iwrite2,199) jr,(icon(jr,jc),jc=l,nc)
-111-
"),3x,f5.3)
enddo
NUPP = 0
NLOW = 0
CALL TRIARM(THAT)
DO 120 J= 1,NN
IF (MOD_ID(THAT,J,-1).EQ.l) THEN
NUPP = NUPP + 1
EFF(NUPP) = X(J)
ELSE
NLOW = NLOW + 1
GEE(NLOW) = X(J)
ENDIF
120 CONTINUE
IF((NUPP + NLOW).NE.NN) THEN
write(Iwrite,*) " NN is not Nupp + Nlow"
STOP
ENDIF
c**************************TIIETA_I1AT FN AND GN ARE NOW IN PLACE.
DO IR
= 1,NR
DO IC
= I,NC
ICON( IR, IC) = 0
ENDDO
ENDDO
DO 200 IBOOT
DO I
= 1,NBOOT
= 1,3
TEMP(I)
=0
-112-
ENDDO
CALL RANV(ISEED,NN,V)
NCOUNT = 0
N
=0
IF (THAT(l) .LEA) CALL TRIARM(THAT)
DO 210 IR= 1, NR
DO 210 IC= 1, NC
N =N + 1
XX
= YeN)
IF (MOD_ID(THAT,IR,IC).EQ.l) THEN
X(N) = EFF(AINT(NUPP*XX)+l)
NCOUNT = NCOUNT + 1
ELSE
X(N) = GEE(AINT(NLOW*XX)+l)
ENDIF
Y(IR,IC) = X(N)
210
CONTINUE
CALL ESTIMATOR (X, TEMP)
DO 220 I = 1,3
T2HAT(IBOOT,I) = TEMP(I)
220
CONTINUE
IF (TEMP(l).NE.TIIAT(l)) THEN
NLESS = NLESS + 1
ELSE
IF (TEMP(1).LE.4) CALL TRIARM(TEMP)
DO 500 N = 1,NN
-113-
IF (MOD_ID(TEMP,N,-l).EQ.l) THEN
ICON(IND(N,1),IND(N,2))=ICON(IND(N,1),IND(N,2))+1
ENDIF
500
CONTINUE
ENDIF
DELTA(IREP,IBOOT) = DEL(THAT,TEMP)
write(Iwrite,89) IBOOT, TEMP,
DELTA(IREP,IBOOT)
89
format(i4,2x,3(i4),3x,f5.3)
200
CONTINUE
LCON = AINT«NBOOT-NLESS)*ALPHA)
UCON = AINT«NBOOT-NLESS)*(l.O-ALPHA))
write(Iwrite,*)"NBOOT
ALPHA
LCON
UCON"
write(Iwrite,191)nboot,alpha,lcon,ucon
191
format(i4,4x,f4.2,2x,i4,2x,i4)
ACON
= 0.0
DO 600 IR =1,NR
DO 600 IC =l,NC
IF (ICON(IR,IC).LE.LCON) THEN
ICON(IR,IC) = 0
ELSEIF (ICON(IR,IC).GE.UCON) THEN
ICON( IR, IC) = 2
ELSE
ICON( IR, IC) = 1
ACON = ACON + 1.0
ENDIF
-114-
600
CONTINUE
ACON = ACON/NN
c
c
SHOW THE BOOTSTRAP CONFIDENCE REGIONS:
c
write(Iwrite,*)"NBOOT= ",NBDDT,"
write(Iwrite2,*)"NBDDT= ",NBDDT,"
NLESS= ",NLESS
NLESS= ",NLESS
write(Iwrite,*)""
do jr = nr,l,-l
write(Iwrite,199) jr,(icon(jr,jc),jc=l,nc)
write(Iwrite2,199) jr,(icon(jr,jc),jc=l,nc)
enddo
write(Iwrite,87) ACON
write(Iwrite2,87) ACON
87
formate"
100
Propn. of the GREY(=l) area is ",f4.2)
CONTINUE
write (Iwrite,85)
85
format("")
STOP
END
-115-
Function Del(.,.)
REAL FUNCTION DEL(Sl,S2)
COMMON IABI NCOUNT, A, B
COMMON IENSI NC, NR, NN
INTEGER Sl(3), S2(3)
DEL
= 0.0
DO 10 IR
= 1,NR
DO 10 IC
= 1,NC
IF (Sl(1).LE.4) CALL TRIARM(Sl)
10
= MOD_ID(Sl,IR,IC)
IF (S2(1).LE.4) CALL TRIARM(S2)
IT
= MOD_ID(S2,IR,IC)
IF (IO.EQ.IT) DEL
= DEL
+ 1.0
10 CONTINUE
DEL
DEL
= DEL INN
= MIN(DEL,
(l-DEL»
RETURN
END
-116-
Subroutine Estimator(.,.)
SUBROUTINE ESTIMATOR(X, EST)
PARAMETER (NRDATA =15, NCDATA =15, NNDATA =225)
COMMON /AB/
NCOUNT, A, B
COMMON /CEST/ NORM
COMMON /ENS/ NC, NR, NN
COMMON /INDEX/ IND«NNDATA+1),2)
DIMENSION X(NNDATA), ITAR«NNDATA+1),2)
REAL DT(3), MAXD
INTEGER T(3), EST(3)
DATA Iwri te/16/
MAXD = 0.0
N
=0
DO 200 IR= 1, NR
DO 200 IC= 1, NC
N
=N +
1
XN = X(N)
IF (N.EQ.1) THEN
J
=1
GO TO 181
ELSE
DO 150 J = N, 2, -1
IF (XN.GT.X«J-1») GO TO 179
150
CONTINUE
ENDIF
-117-
179
IF ( J.LT.N) THEN
DO 180 IJ
= N,
(J+1), -1
= X(IJ-1)
X(IJ)
= IND«IJ-1),
= IND«IJ-1),
IND(IJ, 1)
IND(IJ, 2)
180
•
1)
2)
CONTINUE
ENDIF
181
X(J)
= XN
IND(J,l)
= IR
IND(J,2)
= IC
200
CONTINUE
c
c
X(NNDATA) has now been sorted.
c
ITAR(NN,1)
DO 300 J
=0
•
= (NN-1),
1, -1
IF (X(J).EQ.X«J+1») TUEN
ITAR(J,1)
= ITAR«J+1),1)
+
1
ELSE
ITAR(J,1)
=0
ENDIF
300
CONTINUE
c
c
c
Now the rank of X(j) (defined as the the number of X(i)'s in
the data that are less than or equal
ITAR(j,1».
-118-
to X(j)
),
is (j +
c
Its original index in the data is (IND(j,l), IND(j,2)).
c
c
Enter the candidate boundaries T:
c
ITAR( j, 2), wi 11 for each boundary IT keep track of the rank of
X(j)
c
within IT .
c
DO 400 K
= 1, 6
DO 400 IT1 = 1, NR
DO 400 IT2 = 1, NC
T(l)
=K
T(2) =IT1
T(3) =IT2
IF (K.LE.4) THEN
CALL TRIARM(T)
IF ((A*B).EQ.O) THEN
write (6,71) k, it1, it2
71
format (5x,"The cando bndry.:(",3i3,") is trivial")
Go to 400
ENDIF
ELSEIF ((IThIT2).EQ.(NN*NN)) THEN
write (6,71) k, it1, it2
Go to 400
ENDIF
ITAR(O,l) = 0
ITAR(0,2) = 0
-119-
DO 350 J
= 1,
NN
•
IF (ITAR((J-1),1).EQ.0) THEN
ITAR(J,2)
= ITAR((J-1),2)
+
MOD_ID(T,J~-1)
IF (ITAR(J,1).GT.0) THEN
370
DO 370 JJ
= 1,
ITAR(J,2)
= ITAR(J,2)
ITAR(J,1)
+ MOD_ID(T, (J+JJ),-1)
ENDIF
ELSE
ITAR(J,2)
= ITAR((J-1),2)
ENDIF
350
CONTINUE
IF(ITAR(NN,2).(NN-ITAR(NN,2».EQ.0) THEN
write (6,71) k, it1, it2
Go to 400
•
ELSE
write (6,72) k,it1,it2,itar(nn,2)
72
format (5x, 4i3)
ENDIF
c
c
Si (l.Ie.LIe.3), is the mean-dominant norm of the d's for this
IT.
(Le) Si = S ( d(1,T), d(2,T),
c
, d(nn, T».
will
c
calculate the D(T)'s.
Notice that IND(nn,3)
c
S1
= 0.0
-120-
= #(T).
From Si we
S2
S3
= 0.0
= 0.0
DO 380 J
= 1,NN
D = ABS(float(ITAR(J,2»/float(ITAR(NN,2»
x
- float(J+ITAR(J,1)-ITAR(J,2»/float(NN-ITAR(NN,2»)
S2
= S1 + D
= S2 + D*D
S3
= MAX
S1
380
(S3,D)
CONTINUE
RLAM
= SQRT(float(ITAR(NN,2»*float(NN-
ITAR(NN,2»/float(NN*NN»
DT(1)
= RLAM
* S1/NN
DT(2)
= RLAM
* SQRT(S2/NN)
DT(3)
= RLAM
* S3
c
c
We now update MAXD(i)'s and T(i)'s so that:
c
= max(D(T)
MAXD(i)
: T considered upto now), and T(i)
MAXD(i).
c
IF(DT(NORM).GT.MAXD) THEN
= DT(NORM)
390 I = 1, 3
MAXD
DO
EST(I)
390
= T(I)
CONTINUE
ENDIF
write(6,74)irep,t(i,1),t(i,2),t(i,3),maxd(i)
-121-
= argmax
Preliminary Estimates of the Lipschitz Boundary:
Different Lipschitz Constants.
NUMBER OF ROWS=40j NUMBER OF COLUMNS=64
Lipschitz Constant
= 1/40
1**
11
**40
2**
1111111
**39
3**
111111111111
**38
4**
111111111111111111
11 **37
5**
1111111111111111111111111111111
111 **36
6** 1111111111111111111111111111111111111
1111**35
7** 111111111111111111111111111111111111
1111**34
8** 111111111111111111111111111111111111 111 11
1111 **33
9** 1111111111111111111111111111111111111111111
1111111 **32
10**111111111111111111111111111111111111111111 111
111111
**31
11**111111111111111111111111111111111111111111 111
111114
**30
12**111111111111111111111111111111111111111111 111
1111111444 **29
13**111111111111111111111111111111111111111111 11111
111111444
**28
14**111111111111111111111111111111111111111111 1111
111111444
**27
15**111111111111111111111111111111111111111111 111 1111111444
**26
16**11111111111111111111111111111111111111111111111111111144444
**25
17**1111111111111111111111111111111111111111111111141144144444
**24
18**1111111111111111111111111111111111111111111111444444444444
**23
19** 111111111111111111111111111111111111111111114444444444444
**22
20** 111111111111111111111111111111111111111111144444444444444
**21
21** 111111111111111111111111111111111111111111444444444444444
**20
22** 111111111111111111111111111111111111111114444444444444444
**19
23** 111111111111111111111111111111111111111144444444444444444
**18
24**
1111111111111111111111111111111111111444444444444444444
**17
25**
11111111111111111111111111111111111444444444444444444
**16
26**
111111111111111111111111111111111444444444444444444
**15
27**
11111111111111111111111111111111444444444444444444
**14
28**
1111111111111111111111111111444444444444444444
**13
29**
11111111111111111111111111444444444444444444
**12
30**
111111111111111111111114444444444444444444
**11
31**
1111 111111111111114444444444444444444
**10
32**
111111111111144444444444444444444
** 9
33**
11111111141444444444444444444444
** 8
34**
11111114444444444444444444444444
** 7
444
** 6
35**
111 144444444444444444
444
** 5
36**
44444444
444
444
** 4
37**
444444
38**
4444
4444
** 3
444
** 2
39**
44
44
** 1
40**
444
-123-
Lipschitz Constant
= 2/40
•
40**
11
** 1
39**
1111111
** 2
38**
111111111111
** 3
37**
111111111111111111
11 ** 4
36**
1111111111111111111111111111111
111 ** 5
35** 1111111111111111111111111111111111111
1141** 6
34** 111111111111111111111111111111111111
1141** 7
33** 111111111111111111111111111111111111 111 11
1114 ** 8
32** 1111111111111111111111111111111111111111111
1111114 ** 9
31**111111111111111111111111111111111111111111 111
111114
**10
30**111111111111111111111111111111111111111111 111
111114
**11
29**111111111111111111111111111111111111111111 111
1111111444 **12
28**111111111111111111111111111111111111111111 11111
111111144
**13
27**111111111111111111111111111111111111111111 1111
111111144
**14
26**111111111111111111111111111111111111111111 444 1111111144
**15
25**11111111111111111111111111111111111111111111444111111141144
**16
24**1111111111111111111111111111111111111111111444441144114144
**17
23**1111111111111111111111111111111111111111114444444444144444
**18
22** 111111111111111111111111111111111111111444444444444444444
**19
21** 111111111111111111111111111111111111111444444444444444444
**20
20** 111111111111111111111111111111111111114444444444444444444
**21
19** 111111111111111111111111111111111411414444444444444444444
**22
18** 111111111111111111111111111111114411444444444444444444444
**23
17**
1111111111111111111111111111144444444444444444444444444
**24
16**
11111111111111111111111111144444444444444444444444444
**25
15**
111111111111111111111411114444444444444444444444444
**26
14**
11111111111111111111144414444444444444444444444444
**27
13**
1111111111111111144441444444444444444444444444
**28
12**
11111111111111114444444444444444444444444444
**29
11**
111111111111144444444444444444444444444444
**30
10**
1111 111144444444444444444444444444444
**31
9**
111144444444444444444444444444444
**32
8**
14444444444444444444444444444444
**33
7**
44444444444444444444444444444444
**34
6**
444 444444444444444444
444
**35
5**
44444444
444
444
**36
4**
444444
444
**37
3**
4444
4444
**38
2**
44
444
**39
h*
444
44
**40
-124-
•
•
Lipschitz Constant=3/40
1**
11
**40
2**
1111111
**39
3**
111111111111
**38
4**
111111111111111111
11 **37
5**
1111111111111111111111111111111
111 **36
6** 1111111111111111111111111111111111111
1141**35
7** 111111111111111111111111111111111111
1441**34
8** 111111111111111111111111111111111111 111 11
1414 **33
9** 1111111111111111111111111111111111111111111
1141414 **32
10**111111111111111111111111111111111111111111 111
114444
**31
11**111111111111111111111111111111111111111111 111
114444
**30
12**111111111111111111111111111111111111111111 111
1111444444 **29
111144444
**28
13**111111111111111111111111111111111111111111 11111
14**111111111111111111111111111111111111111111 1114
111114444
**27
**26
15**111111111111111111111111111111111111111111 444 1111114444
16**11111111111111111111111111111111111111111111444411111144444
**25
**24
17**1111111111111111111111111111111111111111411144444144114444
18**1111111111111111111111111111111111111111414444444444144444
**23
19** 111111111111111111111111111111111411111444444444444144444
**22
20** 111111111111111111111111111111111411114444444444444444444
**21
21** 111111111111111111111111111111111411114444444444444444444
**20
22** 111111111111111111111111111111114441414444444444444444444
**19
23** 111111111111111111111111111111114441444444444444444444444
**18
24**
1111111111111111111111111111144441444444444444444444444
**17
25**
11111111111111111111111111144444444444444444444444444
**16
26**
111111111111111111111411114444444444444444444444444
**15
27**
11111111111111111111144411444444444444444444444444
**14
28**
1111111111111111114441444444444444444444444444
**13
29**
11111111111111114444144444444444444444444444
**12
30**
111111111111144444444444444444444444444444
**11
31**
1111 111144444444444444444444444444444
**10
32**
111144444444444444444444444444444
** 9
33**
14444444444444444444444444444444
** 8
34**
44444444444444444444444444444444
** 7
444 444444444444444444
444
** 6
35**
444
** 5
36**
44444444
444
444
** 4
37**
444444
4444
** 3
38**
4444
39**
44
444
** 2
40**
444
44
** 1
-125-
Lipschitz Constant=4/40
•
1**
11
**40
2**
1111111
**39
3**
111111111111
**38
4**
111111111111111111
11 **37
5**
1111111111111111111111111111111
111 **36
6** 1111111111111111111111111111111111111
1441**35
7** 111111111111111111111111111111111111
1441**34
8** 111111111111111111111111111111111111 111 11
1414 **33
9** 1111111111111111111111111111111111111111111
1141414 **32
414444
**31
10**111111111111111111111111111111111111111111 141
414444
**30
11**111111111111111111111111111111111111111111 141
12**111111111111111111111111111111111111111111 114
1114144444 **29
111444444
**28
13**111111111111111111111111111111111111111111 11411
111444444
**27
14**111111111111111111111111111111111111111111 4444
15**111111111111111111111111111111111111111111 444 1111144444
**26
16**11111111111111111111111111111111111111111111444411111144444
**25
17**1111111111111111111111111111111111111111411144441144114444
**24
18**1111111111111111111111111111111111111111414444444444144444
**23
19** 111111111111111111111111111111111411111444444444444144444
**22
20** 111111111111111111111111111111111411111444444444444144444
**21
**20
21** 111111111111111111111111111111111411114444444444444444444
22** 111111111111111111111111111111111411414444444444444444444
**19
23** 111111111111111111111111111111114441444444444444444444444
**18
24**
1111111111111111111111111111144441444444444444444444444
**17
11111111111111111111111111144444444444444444444444444
**16
25**
26**
111111111111111111111111114444444444444444444444444
**15
11111111111111111111114411444444444444444444444444
**14
27**
28**
1111111111111111111441444444444444444444444444
**13
29**
11111111111111111144144444444444444444444444
**12
30**
111111111111141144444444444444444444444444
**11
**10
31**
1111 111144444444444444444444444444444
32**
111144444444444444444444444444444
** 9
33**
11444444444444444444444444444444
** 8
34**
14444444444444444444444444444444
** 7
35**
144 444444444444444444
444
** 6
444
444
** 5
36**
44444444
37**
444444
444
** 4
4444
4444
** 3
38**
44
444
** 2
39**
444
44
** 1
40**
-126-
•
•
•
Lipschitz Const&nt=5/40
1**
11
**40
2**
1111111
**39
3**
111111111111
**38
4**
111111111111111111
11 **37
5**
1111111111111111111111111111111
141 **36
6** 1111111111111111111111111111111111111
1441**35
7** 111111111111111111111111111111111111
1441**34
8** 111111111111111111111111111111111111 111 11
1414 **33
9** 1111111111111111111111111111111111111111111
1141414 **32
10**111111111111111111111111111111111111111111 141
414444
**31
414444
**30
11**111111111111111111111111111111111111111111 141
12**111111111111111111111111111111111111111111 114
1114144444 **29
13**111111111111111111111111111111111111111111 11411
111414444
**28
14**111111111111111111111111111111111111111111 1414
111144444
**27
15**111111111111111111111111111111111111111111 444 1111144444
**26
16**11111111111111111111111111111111111111111111444411111144444
**25
17**111111111111111111111111111111111111111141114444~144114444
**24
18**1111111111111111111111111111111111111111414444444444144444
**23
19** 111111111111111111111111111111111411111444444444444144444
**22
20** 111111111111111111111111111111111411111444444444444144444
**21
21** 111111111111111111111111111111111411114444444444444144444
**20
22** 111111111111111111111111111111111411414444444444444444444
**19
23** 111111111111111111111111111111114411444444444444444444444
**18
24**
1111111111111111111111111111144441444444444444444444444
**17
25**
11111111111111111111111111144444444444444444444444444
**16
26**
111111111111111111111411114444444444444444444444444
**15
**14
27**
11111111111111111111144411444444444444444444444444
28**
1111111111111111114441444444444444444444444444
**13
29**
11111111111111111444144444444444444444444444
**12
30**
111111111111141444444444444444444444444444
**11
31**
1111 111114444444444444444444444444444
**10
32**
111114444444444444444444444444444
** 9
33**
11414444444444444444444444444444
** 8
34**
14144444444444444444444444444444
** 7
444
** 6
35**
144 444444444444444444
36**
44444444
444
444
** 5
37**
444444
444
** 4
38**
4444
4444
** 3
39**
44
444
** 2
44
** 1
40**
444
-127-