ON THE BOOTSTRAP AND CONFIDENCE INTERVALS 1
by
Peter Hall
University of North Carolina, Chapel Hil1 2
c
ABSTRACT.
We derive an explicit formula for the first term in an unconditionaZ
Edgeworth-type expansion of coverage probability for the nonparametric bootstrap
technique applied to a very broad class of "studentized" statistics.
The class
includes sample mean, k-sample mean, sample correlation coefficient, maximum
likelihood estimators expressible as functions of vector means, etc.
•
We
suggest that the bootstrap is really an empiric one-term Edgeworth inversion,
with the bootstrap simulations implicitly estimating the first term in an
Edgeworth expansion.
This view of the bootstrap is reinforced by our discussion
of the iterated bootstrap, which inverts an Edgeworth expansion to arbitrary
order by simulating simulations.
•
AMS (1980) SUBJECT CLASSIFICATION:
KEY WORDS AND PHRASES:
Primary 62E20, Secondary 62G05 .
Bootstrap, central limit theorem, Edgeworth expansion,
rate of convergence, Studentized statistic.
SHORT TITLE:
Bootstrap.
1Research partially supported by AFOSR Contract No. F49620 82 C 0009.
20n leave from Australian National University
1.
Introduction and summary.
In this paper it is argued that Efron's
bootstrap [8,9] is in effect an empiric one-term Edgeworth inversion, at
least from the point of view of its role in constructing confidence
intervals and hypothesis tests.
The bootstrap argument uses the empiric
distribution to estimate the first term in an Edgeworth expansion, and
correct for it.
That task may be performed directly, as described in
[10], although the bootstrap provides a correction which is smoother
(i.e. uniform) and automatic (i.e. does not require a prelim,nary theoretical
calculation of the expansion's first term).
There exist other smooth
tnversions [11], although we know of none other which is automatic.
We shall show that the bootstap may be iterated to yield an
approximation with an error of arbitrarily small order, in the same
•
way that direct Edgeworth inversions were iterated in [10].
Bootstrap
iteration involves simulations of simulations, and although it is
almost prohibitively laborious to implement in practice, its theoretical
properties provide new insight into the nature of the bootstrap
algorithm.
We shall obtain an explicit formula for the first error term after
the bootstrap correction, in a very wide class of "Studentized" statistics
including the mean, variance, correlation coefficient, maximum likelihood
•
estimators expressible as explicit or implicit functions of vector means,
etc.
The formula for the error term may be used to predict the
performance of the bootstrap; see the comments following display (3.2)
below.
Our expansion is very different from those described by Singh
[13] and Babu and Singh [2] in the care of k-sample Studentized means.
-2-
The expansions studied by Babu and Singh hold with probability one along
a sample path, and are of interest if a statistician wishes to know the
order of approximation for his particular sample.
In contrast, our
expansions are not conditional on a sample path, and so are useful
in describing coverage probabilities or significance levels in the
classical frequentist sense.
Our general formulation of the problem
includes k-sample means as a particular case.
The basic result is described in Section 2 and proved in Section 5.
Its implications, and issues arising from its proof, are discussed in
Section 3.
These matters lead naturally to bootstrap iteration,
described in Section 4.
Several ideas in our proofs are borrowed from [5,6,10], and in
those places we omit many details.
In addition to the works of Babu
and Singh cited above, the reader is referred to Beran [3,4] and Bickel
and Freedman [7] for an asymptotic account of the bootstrap.
2.
Notation and basic result.
Let
~'Y1'Y2".'
be independent and
identically distributed d-dimensional random vectors with mean~= E(Y).
n
~
Define Y::{Y 1, ... ,Y} and Y=n- 1 I Yr' Denote the ilth elements of
~
-n
r=1 Y,Y ,V and]1 by Y"Y "V" and ]1", respectively. Let f be a real-valued
- ~r , rl
function on ffid with at least one continuous derivative. We shall consider
the problem of interval inference based on Y1 (Y) :: f(~) - f(~).
- e is
f(Y)::
•
Typically
an estimate of a parameter f(]1):': O.
-
Under appropriate regularity conditions, n*Y1(Y) is approximately
normally distributed with zero mean and variance given by
-3-
a2~E
(2.1)
were
h
0i' fi(~)fJ'(~) ,
E
i j
a . . ::
1J
J
Usually the value of 0 2
cov (Y ., Y)
. an d f . ()
~
. ( d / 'dP l' )f()
~.
1
J
1 - -
would not be known, and would be replaced by an estimate such as
•
(2.2)
where o .. =n -1 ~L (Y i - -Y. )( Y . - -Y. ) .
lJ
r=l
r
1
rJ
J
A
We would focus attention on
_
Y2(Y):: a-I Yl (Y), rather than Yl (Y) •
Let us adjoin to the vector
~r
the set of those products YriY rj ,
1..:: i"::j~d, which do not already appear in ~r' Thus, ~r is expanded to
o
yO , say, of length d < d(d+3)/2. Let yO and ~o = E(Y ) denote the
-r
0-
-
-
corresponding lengtheningsof ~ and ~, and let ~ = n-
-
1
n
r~l ~~.
We may
g(rl~)={f(~)-f(~)}/a.
Clearly g(~ole) = O. The asymptoti·c variance of n1 g(rl~) is unity,
which is reflected in the fact that if we construct the quantity 0 2 at
write Y2(Y) as a function of r
(2.1) for g(o
I~)
alone:
instead of f(o), we obtain precisely 1.
The bootstrap argument runs as follows.
Y, and let
Condition on the sample
~'~l""'~n
be independent and identically distributed with
n
the n-point distribution P(Z=Y IY)=n- 1 , 1<r<n. Set Z=n- 1 I Z .
- -r
- r=1 - r
We may work out the distribution of g(ZOIV), conditional on Y, to arbitrary
accuracy using simulation.
for 0 <a < 1 .
The distribution ;s discrete.
Define
Then t a is the (nonparametric) bootstrap approximation to
the upper (l-a)-level critical point of n' g(~I~), and may be used in the
-4-
construction of confidence intervals or hypothesis tests.
Theorem 2.1
below describes the accuracy of the approximation.
Before stating the theorem we list notation and technical conditions.
Given any vector ~o= (u1, ... ,ud,ull,u12, ... ,udd)T = (u 1 , ... ,u
dO
)T of length
dO, define ~:= (u 1 ' ... ,u d )T (the first d elements of ~o),
g(uolv}:= {f(u} - f(v}} / {l(uo}}!
""'
.......
and g,.
1'"
.....,
"'"
" (uolv):: (aP/au,
p ~
~
'1'''
;)U,')
P
g(uolv}.
(Recall from the definition
,J
of yO that only those products Y'Y ' not already occurring in Y appear in
~
the extension yO.
If Y.Y.=Yk€Y,
we define u.J.=u
, J
,
k .) Let B(u,p}
denote the open ball in Rd centered at ~ and of radius p, and define
~
B(uo ,p} analogously.
~
Given an extended mean vector pO with a 2 (j.I0}
.......
"',
> 0,
"V
assume that on some set B(1!°,P} x B(~,p} of vectors (~o,~), o2(~O»0
and g(~ol~} has four continuous derivatives with respect to uO .
Let
h(~ol~} denote any of these derivatives of order one, two or three.
Assume that h(uolu} has four continuous derivative with respect to UO ,
~
~
~
for each choice of h and all ~o € B(~o ,p}.
satisfies E(yo} = ~o and E(llrol~+c}
< 00,
Suppose the distribution of ~
for some
£
>
O.
Further. assume
that yO satisfies Cram~r's continuity condition:
(2.3)
(One consequence of (2.3) is that yO has nonsingular variance matrix.)
Note that by definition of g,
~
-5d
d
g(uolu) = 0 and
(u .. - u,.u .)g,.(uolu)gJ.(uolu) = 1
- "
i=1 j = I ' J
J
""
""
r r
(2.4)
for all uO e B(j..I° ,p).
Define
"
"
. =g.
. (j..I°lp), 9.. (j~ ::: (d/aJJ.)g,.
" (uo/u)1 ° °
'1 ... 'p
1··· p" " u =j..I
1·" p
1·" p
R,.
,
,
,
v
-
'
p
and
Ct.
_c
•
'1···'p
(2.5)
E{
n
j=1
(Y. -po )}.
'j
'j
tjJ(Z):: (z/4)
Define the odd, quintic polynomial
r r r
i
2
x {z (Q,p(X;j
j
P
r
Q, ••
{2
+ (z 2-3)(9. 2pCt pp - I)}
(k)
(Xk + 9. .. 9. p
'J
P ,J
k
2
+ (l / 3)( Z -1 )( 3R, i R, j R, p
r r
k
•
-3R,.R,.R,
, J p
a ..
lJ
}'
K
9.
k
a
kP
m
Ct 1jp)
(m)
\
R,k Ct" k Ct + R,.R,.9. L. 9. k
,J
mp
, J p k
Ct··
' J kP
)}.
Let t a be as in the previous paragraph, and let ¢ and
~
be the standard
normal distribution and density functions, respectively.
THEOREM 2.1.
Under the above conditions,
(2.6)
uniformLy in
Ct,
0< a < 1, where z(a) is the soLution of 4>(z) =a.
An alternative approach is to bootstrap the statistic g*(~I~o), where
g*(~I~o)::: {f(~) - f(~)} / {o2(vo)}! ,
-6-
instead of g(YOI~).
t~
=
In that case we use the critical point estimate
t~(Y):: inf{t: PEnt g*(~lf) ~ tl Y] ~ cd ,
instead of ta. Construct a new polynomial ~* whose coefficients are
' d by rep lac,ng
'
d (j ) * . ,
obta,ne
R,.
•
an d R,. (j ) • by R,.*
"
an~.
'1""p
'1""p
'1'" p
'1""p
o
respectively, in formula (2.5), where g~
. (ulv o ) =, (0 P/::lU.
dU. )g*(ulv ),
'1""p -,
'1'"
'p
R,~
.::
'1""p
g~
.
(l}I'l}0), R"
'1""p --
(j )*
.
'1""p
Instead of Theorem 2.1, we obtain:
THEOREM 2.2. Under the above conditions, with the smoothness constraints applied
to g*(~I~o) instead of g(~ol~) ,
(2.7)
•
uniformly in a, 0 < a < 1 .
Theorem 2.2 may be derived almost identically to Theorem 2.1, and so
the proof will not be given here.
The simulations required to calculate
are usually less time-consuming than those needed to derive t a , since
only the numerator in the formula g*(~lf) ::: {f(~) - fr~)} / a(!) is being
t~
simulated.
However, result (2.7) can only be used to construct confidence
intervals for f(~) when 02(~0) is known, whereas (2.6) can be used even
when a2(~0) is unknown.
-7-
3.
Discussion.
3.1.
Our discussion will be confined to Theorem 2.1.
A Cornish-Fisher' view of the hootstr'ap.
general "Studentized statistic.
ll
Let s=n1-(8-8)/0 be a
In a great many cases, s admits an
Edgeworth expansion:
uniformly in x, where
1j is a polynomial of degree 3j-1. Whenever it
exists, this expansion may be inverted to yield an expansion of
TI
(inverse) Cornish-Fisher type:
P{n1-(6-o)/~ ~ x +
k
I
j=1
n- j / 2TI o(x)} = ~(x) + 0(n- k/ 2 )
2J
uniformly on compact intervals, where
~
{TI
2j
} is a new sequence of poly-
nomials.
An alternative, equivalent form is
(3.1)
!
k
0/2
PEn (8-8)/; 2. z(a) + oL n- J 11 o{Z(ct}lJ
2J
J=1
=a+
o(n-
k/2
)
uniformly in a€(c,1-c), any c>O, where z is the solution of ¢(z)= ex.
The coefficients of the TI2j'S depend on the sampling distribution
through its moments.
Let IT 2j denote the version of TI 2j with each
population moment replaced by the corresponding sample moment. It is
not difficult to show that for each k (modulo regularity conditions),
and for each Ct€ (0,1),
t
a
as n-+ oo •
= z(a)
+
k
0/2.
L
n- J IT 0{z(ex)}
2
j=1
J
+
0
(nP
k/2
)
In this sense, the bootstrap critical point estimate t
ex
is
asymptotically equivalent to the critical point estimate obtained by
empiric inverse Cornish-Fisher expansion.
-8-
If we replace each 1T 2j in
{3.l} by II 2j , then that expansion fails for k>2. This follows from
the fact that the cumulants of 01{z):: n!(6-o)~-1 n- j / 2 IT ,(Z)
2J
j=1
coincide with those of 02{z):= n!{e-e)/a- 1 n- j / 2 1T2'(z) only to
J
j=l
order n-!. Even-order moments of 01 (z) contain terms of order n- 1
3.2.
The bootstrap and Edgeworth inversion.
I
I
which do not appear in the corresponding moments of ()2(z).
As a result,
with z=z(ex.):
P{n!{8-e)/~ -< t ex. }
= P{nt(e-o)/~ ~ z + n- t
1T
1
1
1
21 (z) + n- 1T 22(z)} + n- ~(z) ~(z) + o(n- )
= ex + n- 1 l/J(z)~(z) + o(n- 1 ) ,
where
~
has the meaning it did in Theorem 2.1.
This is essentially the
argument used in Section 5 to prove Theorem 2.1.
~ n- j / 2 "2J'{z) produce confidence intervals with
,J= 1
the same coverage probability, up to terms o(n- 1). The latter critical
Note that t
ex.
and z +
point estimate results from a direct Edgeworth inversion of the type
studied in [10].
3.3.
Tightening the bootstrap.
As an illustration, we shall consider use
of the bootstrap to set confidence intervals for a population mean.
There, f{u 1):::u 1 , representing the mean, and ~o=(ul,ull)T = (u 1 ,u 2)T, with
-9-
u representing mean square.
2
The "Studentized" mean is given by
2 t
g(ul,u2Ivl) = (u 1-v 1 )/(u 2-u 1 ) .
Assume without loss of generality that E(Y)=O and E(y 2)=I. Then £1=1,
_
(2)_
(1)_
(2)_3
_ (3)
£12--1, £1 - -1, £11 - 2, £12 - 4' and other terms are zero. let J..l 3-E Y
and J..l4 = E(y 4 ). It follows from the definition 'of ljJ at (2.5) that
Therefore ljJ(z) vanishes if kurtosis and skewness are both zero.
Formula
(3.2) suggests that the bootstrap approximation may be noticeably in
error for skew, 1eptokurtic distributions.
On the other hand, contributions
of skewness and kurtosis have some tendency to cancel in the case of
skew, p1atykurtic distributions.
It was shown in [10] that statistics which are representable as
functions of vector means admit directly invertible Edgeworth expansions.
See [1,12] for alternative approaches.
A simple way of tightening the
bootstrap approximation is to directly invert the expansion in Theorem 2.1.
To thi send, defi ne
" : : 0"-4 n-1
h
were
J..l4
n
n
n
\"I.. (Y r --y)4 , J.'".l =(}"'-3 n-1 I..\" (y --y)3, 0,,2 = n- 1 I..\" (Y r --y)2 .
r
3
r=1
r=1
r=l
Under the conditions of Theorem 2.1,
(3.3)
uniformly in ae(c,l-d, any £>0.
In a sense, this procedure is the
-10-
reverse of one suggested by Abramovitch and Singh [1]:
they first-order
corrected and then bootstrapped, while we bootstrap and then first-order
correct.
The end result is very similar.
A more detailed argument shows that under appropriate moment
conditions, the right-hand side of (3.3) may be written as
a + n-3/2
)
~1 (
z)
¢(z
+ O(n -2 ), where
~1
is an even polynomial.
This
observation is particularly relevant when using the tightening procedure
to construct a symmetric, two-sided confidence interval.
g(YOI~)
~
~
={f(V) - f(~)}/o, with
~
~
Recalling that
A
(J
defined at (2.2), we see that the interval
(f("!) - n- i ;[t 1_a / 2(Y) - n- 1 ~{z(l-a/2)}] ,
f(!) - n- i o[ta / 2(Y) - n- 1 ~{z(a/2)}])
covers f(~) with probability 1-a+ 0(n- 2), not just 1-a+'O(n- 3/2).
This property motivates bootstrap tightening:
a single correction which
improves the bootstrap approximation to order n- 3/ 2 , actually gives an
error as small as order n- 2 in the case of two-sided confidence intervals.
A disadvantage of the approach described by (3.3) is that it does
not apply uniformly in a.
approximation is smooth.
as a+ 0 or as a+ 1.
In this sense, it is not "smooth"; the bootstrap
Unless ~4-3-(3/2)~i = 0, (3.3) will fail either
The problems created by this discontinuity are
illustrated by simulations summarized in [10].
An alternative, smooth
bootstrap tightening may be constructed as follows.
A
A
2
A.=~4-3-(3/2)D3.
Calculate the value
Conditional on Y, construct a known, "comparison
distribution" with zero mean and unit variance, having its 3rd and 4th
-11moments lJo3 and lJo4 satisfying ~ =lJ04-3-(3/2)lJ~3' By simulation or
otherwise, calculate that value 13 = S(CI.) such that the bootstrap confidence
interval (_ro,t S) for the comparison distribution covers the comparison
version of n!g(~I~) precisely 100a % of the time. Then
.
(3.4)
uniform~y
in CI.€ (0,1). The right-hand side of (3.4) may be expanded
3 2
as CI. + n- / 1J.i2(z) <P(z) + 0(n- 2) for an eVen polynomial 1J.i . Therefore the
2
two-sided interval
covers f(~) with probability 1-ct+0(n-2), not just 1-Cl.+0(n- 3/ 2 ).
e
3.4.
Bootstrap 'n·l,thy'en. The arguments described above may be refined in
many ways, for example to yield approximations with errors of order n- k/ 2
for arbitrary k.
The idea of a "comparison distribution" in Subsection
3.3 is based on a tabular method for correcting normal approximations
[11]; in the present case, simulation rather than tabulation is used to
calculate the correction.
Of course, if simulation can be conducted
very rapidly and cheaply then even traditional statistical tables become
obsolete.
We suggest that the "parametric simulation" used to generate
statistical tables, and the "nonparametric simulation" used to construct
nonparametric bootstrap critical points, are just two extremes of a
vast array of techniques which use simulation to solve problems that
are essentially numerical.
The "comparison distribution" method is
only one example of a simulation approach intermediate between nonparametric
-12-
and completely parametric.
uniform in a
€
It is possible to construct a smooth (i.e.
(0,1)) simulation-based approximation, based on the sample
Y only through its first four moments, which corrects two-sided
confidence intervals to order n- 2 .
method was mentioned in [11].)
(The tabular analogue of this
Among these bootstrap brethren, the
bootstrap itself stands impressively tall because of its great flexibility
and its lI au tomatic" correction of normal approximations to order n- 1 .
Other, intermediate approaches may prove well suited to specific
problems.
3.6
Bias correction.
The bootstrap technique discussed in this paper
has been termed tile "percentile method" by Efron (1932).
proposed a bias correction for the percentile method.
Efron has
That correction
is intended for use only with two-sided confidence intervals, and so
Its aim is to "cen tre a twoll
has been omitted from our work so far.
sided interval; it does not appear to have a substantial effect on coverage
probabi 1ity.
Versions of Theorems 2.1 and 2.2 may be proved for the biascorrected percentile method, using arguments in Section 5.
illustrate in the case of Theorem 2.1.
First, we introduce necessary
~(z)
Notice that the definition of
notation.
We shall
at (2.5) includes a term
whose coeffi ci ent is (1/3)( z2 -1) .
Let TIl (z) be the vers i on of ~(z) in
which that coefficient is replaced
by
E,
==
-(1/3) L L L Q,·Q,·Q,k
i j k
1 J
(t.
(1/3)(z2+1).
'k'
1J
l)
-=
(E~2/2)
Define the constants
- E,
L L ~".
1J
i j
a ...
1J
-13-
and t a
l
THEOREM 3.1
= ta
I.
Under> the conditions of 'l'heorem 2.1,
unifo~Zy in a
e (0,1).
As expected, the bias correction introduces a skewness term of
order n- 12
The effect of that term vanishes when the bias correction
is used to construct a two-sided interval.
4.
The
it~~~~~~~~~.
In Subsections3.1 and 3.2 we demonstrated
that the bootstrap may be viewed as a first-order inversion of an
Edgeworth expansion.
In Subsections 3.3 and 3.4 we described relatively
simple corrections of second order.
We shall show now that an arbitrarily
high degree of correction may be obtained by iterating the bootstrap
argument.
Once again, we shall tailor our discussion to the more
general context of Theorem 2.1, where a2(~0) is not assumed known.
So as to clearly explain the procedure, we shall abbreviate our
earlier notation.
Distributions, empiric or otherwise, will be
represented by their distribution functions. Let G1 and G2 be any two
distributions on Rd , let ~1 and ~2 be their means, and define
Given a distribution Fi - 1 , draw a random n-sample from it and take F
i
to be the (empiric) distribution function of that sample. Repeat this
for all
i~l,
with Fo being the (non-random) distribution of Y.
-14-
In this notation the random sample Y introduced in Section 2 has distribution function Fl , and our aim is to determine approximate critical
points for the statistic h(FlIFo). Those points are calculated as
follows.
Given an (r-l)'th order approximation t~r-l)(Fl)' define t~r)(Fl)
by t(r)(F )::: t(r-l)(F ) + t
(F ), where for any i > 1,
a
ex
a,r l
-l
l
Assuming Cramer's condition (2.3) and appropriate moment conditions on
Fo ' we have
(4.1)
P{h(F IF) < t (r )(F )} = ex + n- (r+ 1 )/ 2
1
uniformly in
a€
0
-
1
a
IjJ ( r ){z(ex ) }¢ { z(ci)}
(c,l-E),any £>0, where ljJ(r) is a polynomial.
notation of Theorem 2.1, t(l)::: t
a
a
+
0(
n- (r+l )/ 2 )
In the
and 1jJ(l) -:: 1jJ.
Calculation of t~r)(Fl) requires simulation up to and including the
level of Fr +l . We shall illustrate methodology in the case
that t(r)(F l ) =
t .(F ); we shall show how to construct
a
j=l a,J l
t a, 2(F l ) and t a, 3(F l ). Define t ex, l(F l ) = t(l)(F
l ) using the
ct
I
bootstrap argument, i.e. by simulating conditional on Fl.
r=3.
t
0.,
Note
l(F l ),
usual
To calculate
t a ,2(F l ), first compute t a ,l(F 2 ) for each F2 derivable by sampling Fl ,
by simulating conditional on F2:
Next calculate t a ,2(F l ) by simulating conditional on Fl :
-15-
To derive t a ,3{F 1 }, first calculate t a ,1{F 3 }, then t a ,2{F 2 } {using t a )1{F 3 }},
and finally t a ,3{F 1} {using t a ,1{F 2} and t a ,2{F 2}}.
The time taken to construct successive approximations
increases rapidly and exponentially with r.
t~r}{F1}
We have introduced the
iterated bootstrap to clarify the role of the ordinary bootstrap as an
empiric one-term Edgeworth inversion.
It could not reasonably be
regarded as a general practical tool for the continuous case which is
the subject of this paper.
Result {4.1} may be proved along the lines of Theorem 2.1, and so
will not be derived here.
Briefly, the argument runs as follows - we
illustrate with the case r=2.
~
Let
z=z{(~}
be the solution of ¢{Z}=a.
A version of Theorem 2.1 in which the pair {F o ,F 1} is replaced by
{F 1 ,F 2}, reveals that
P{h{ F2 IF1 } 5 t a ,1{F 2} IF 1} = ¢{z} + n-1 ~1{zIFl}~{z} + n-3/2 ~2{zIFl}~ {z}
+ Op{n- 2} ,
where
~i{·IG}
denotes a polynomial whose coefficients are smooth functions
of means with respect to G, and where
~1{.IFo} =~{.},
defined at {2.5}.
Consequently, inverting as in steps {ii} and {iii} from Section 5,
and arguing as in steps {iv} and {vi} of Section 5,
P{h{F1IF o } ~ ta,1{F 1)+ t a ,2{F 1}}
= P{h{F1IFo}':' t cx ,1{F 1 }-n- 1 ~l{zIFl}} + n- 3/ 2 ~4{zIFo}~{z}+o{n-2}
-16-
= P{h( F1 IFo ) ~ t a ,1(F 1)-n- 1 ljJ1(zIF o )} + n- 3/ 2 ljJ5(zIF o )<P(Z) + 0(n- 2)
= P{h(F1IFo) ~ ta,1(F1)-n-1ljJ1(zIFo)} + n-3/2ljJ6(zIFo)~(Z) + 0(n- 2)
={a+ n- 1 ljJ1(zIF o ) + n- 3/ 2 ljJ7(zIF o )} + n- 3/ 2 ljJ6(zIFo)~(z) + 0(n- 2 )
= a+ n- 3/ 2{ljJ6(zIF o ) + ljJ7(zIFo)}~(z) + 0(n- 2).
The general proof uses induction over r.
5.
Proof of Theorem 2.1.
superscript on vectors
yo,
It is inconvenient to continue using the
~o, etc.
g(!I~) instead of g(~I~), and so on.
We shall drop it below, writing
This amounts to assuming that the
vectors of interest are already sufficiently long, so that further
extension is unnecessary. Write V = (v .. ) _= var(Y) (var(Y o ) in the old
_0
lJ
-
notation).
We begin by introducing polynomials TIij and
ij , for i,j=1,2.
Bhattacharya and Ghosh [5, Theorem 2] provide explicit conditions
IT
under which
uniformly in x, where
1j is a polynomial of degree 3j-1 whose coefficients
depend on the first j+2 moments of Y and the first j+1 derivatives of
g(~I~)
at
~=~.
lT
Applied to the bootstrap distribution, this would give
us formally
Notice that only derivatives of order three or less are involved; hence
the condition on derivatives of order three or less in the paragraph
preceding Theorem 2.1.
-17-
The polynomials TI 2 , are defined to be those polynomials which are
J
2
_ '/2
such that, for each ye:R, the quantity x=x(y)=y+.L n J TI .(y)
2J
J=l
2 -'/2
-3/2
satisfies ¢(x) +.L n J TIlj(x)¢(x) = <!J(Y) + O(n
) as n-+ oo
J=l
The coefficients of the TI 2j ' S are of course simple functions of the
coefficients of the TIlj'S; in fact,
21 = -TIll' The random-coefficient
polynomials IT 2l and IT 22 are defined analogously. Fortunately we do not yet
require explicit expressions for any of these polynomaials.
- ( Y .-Y.)
By way of notation, define V,,~n-1 ~/., (Y .-Y.)
and
1J
r=1 rl
1
rJ J
let
~=
11
(V ij ) be the usual estimate of
~o'
As n-+ oo ,
~-+~o
in probability.
Given a nonnegative integer-valued vector ~= (a 1 , ... ,ud )T, and a smooth
function f of d variables, let
Our proof of Theorem 2.1 contains six steps.
Let C1 ,C 2 "" denote
positive constants not depending on n, and C the class of all samples Y.
Write R for the probability measure on:R generated by n!g(~IY), conditional
on Y.
Step (iJ.
Here we expand the conditional distribution of n!g(~IY), and prove:
PROPOS IT ION 5.1.
exist
c~l) _=
Under the conel-i f;1:ons () r 'l'heoY'em 2. Z, there
C with
P(C~l)) =0(n- 1 ),
and conDtants C1 > 0 and q
such that the random variable
t:.. (x)::
n
f( -oo,x ]
d(R-ep-
2 -j/2
L n
IT·· ¢)
j=1
lJ
satisfies
(5.1)
sup
sup
Ye c(l) _00< x<00
n
<
C1. n
-1- ....c'l
,
€
(O,t),
-18-
and such that
In .. (x)l
sup
YeCO)
n
(5.2)
< C1(l+lxI3j-1)
-
lJ
for -oo<x<oo and j=1,2.
PROOF.
We begin by defining several classes Cnj of samples
The random coefficients of polynomials
Y of
functions of moments of
E(II~1I8)<00,
for each
P(C
n1
J[n
order 4 or less.
and 11
12
Y.
are continuous
Under the condition
1
each such sample moment satisfies P( IM-EMI >\1) = 0(n- )
n > O.
Therefore we may choose C > a and C t.:.-. C such that
n1
2
1
)=0(n- ) and the absolute value of each coefficient of TIll and
TI 12 is dominated by C whenever YeC .
n1
2
= n- 1
- -
n
c~l)'::'Cn1
we
For any p > 0,
may ensure that (5.2) holds.
E(IIZ-VII~Y)
By choosing
d
L {L
(V ._V.)2}p/2< (2d)P nr= 1 j =1
rJ J
-
1
d
n
L L
(IV ·I P + IV·I P),
r= 1 j =1
rJ
J
and so
P{ E(II Z- y II PI Y)
n
d
.:: p{
L
j=1
I
2:
L ( Iv
r=1
d
(2 d) p
j=l
. I p - EIv ·1 p)
rJ
rJ
d
.:: L
n-(8+E)/p E{I
j=1
provided 1 < P < (8+£)/2.
L (E IVI· Ip + lEV. Ip) + (2 d) p+ 1 }
J
I
J
d ·
+ n
L
j=1
IIV·I p J
n
L
IEVJ.I P I ~.
2dn}
d
L
(IV .IP-Elv ·I P)I(8+E)/p} +
EIV.-EV 18+£
r=1
rJ
rJ
j=1
J
j
Therefore we may choose £2 e
= {E(II~_YI14+2E2IY) ~
(a,!) and C3 > a such
1
C } has P(.C ) =0(n- ).
3
n2
Note that V= (V.J.)-+V = (v .. ) in probability as n-+ oo , and that V is
1
-0
lJ
-o
that the set C
n2
e
-19positive definite.
where a> O.
Suppose all eigenvectors of ~o lie within [2a,1/2a],
Choose C > 0 so small that any sYmmetric d x d matrix (u ij )
satisfying lu .. -v ..
lJ lJ
I -<
b for all i and j has all its eigenvalues within
Let Cn3 be the class of all samples Y such that \Vij-V ij I s. b
-1
-1
d
for all i and j. If Ye Cn3 then C4 11:11 s.11~ :11 s. C41~11 for all t eR ,
[a,l/a].
~o
where C depends only on
4
IV .. -v··1 < In
lJ lJ --
-1
and b.
n
I
r=l
The inequality
{Y .Y .-E{Y .Y ·)}I + IV·IIV.-E{V·)I
rl rJ
rl rJ
1
J
J
+ I E(V . ) 11
J
may be used to prove that P{C
Let
?r= {~-i(?r - y)
o
Rd generated by n- t
t
if
) = 0{n- 1 ) .
IIY.-!{~r -~) 112 n1/ 2
otherwise
and Z-l":.::Z*_ E{z*IY).
-r -r
-r
on Y.
n3
v.1 - E(V1. ) I
Define Q and Qt to be the probability measures on
n
n
Z and n- 1 I zt, respectively, both conditional
r=l -r
r=l -r
I
t
t
1J
\)
Let V ::: (V .. ) be the variance matrix and {X } the cumulant sequence
-
of ill", again conditional on Y.
-
Observe that Vt-+-I:: (8 .. ) in probability
-
.~
1J
as n-+-<JO.
Choose c'O so small that any symmetric dxd matrix (u;j)
satisfying
IU ij - oijl 2
[1/2,2].
c for all i and j has all its eigenvalues within
t
Let Cn4 be the class of samples Y such that IVij-Oijl2 c for
all i and j, and define
V·l~·J· = 0 ..
lJ
W= y-1U-y).
The identity
- E{{W) .(W). I(II'WII> n1 ) IY}
- 1 - J
-
-20-
-
together with Markoy's inequality, may be used to proye that p( C n C )
n3
n4
1
= 0(n- ). For example, for any n > 0,
P[C n3 ; IE{(~)i(~)j I(11~Il>n!)IYJI > n]
2. P[C~ E{II~_YI12 I(C41l~- YII> nt)IYJ > n]
< C2 n- 1 (c- 1 n t )-3 E( IIZ-Yll5) = 0(n- 3/ 2 ) .
-
4
4
- -
Define C ==C n C n C n C .
nS
n1
n2
n4
n3
Thenp(C
ns
)=0(n- 1 ).
remainder of Step (i) we shall consider only samples YeC
Throughout the
nS
.
d
Let A':'R , random but measurable in the a-field generated by Y.
Define At ==
r 1 A + n1 E(~il~)
and, in notation of [6, pp. 53-54],
_
d+2 -r/2
H = Q- L n
Pr (-<PO Vt :
r=l
'
t
{x)).
Write {X } for the cumulant sequence of Z, conditional on
v
-
Y.
The argument
preceding (20.15) of [6, p. 209] yields:
(5.3)
IfA
-
2
d[ Q- rk n- r /2 Pr ( -<P 0, V: {X))]
f At
t
d[Q -
~ n-r/2 Pr(-<P V : {X~})]I'
I
O t
L
r=O
'
the last inequal ity following since Ye C
n2
(14.74) of [6, pp. 72, 133], we obtain:
n C
n3
.
Using (9.12) and
-21-
If
(5.4)
t
l d[Q -
A,
~L n-r/2 Pr(-<I.>O
r=O
again since Ye Cn2 nC n3 .
'
t
Vt : {x }) - H]I
v
Combining (5.3) and (5.4), we obtain on C :
n5
(5.5)
Let K be a probability measure on Rd with support confined to
B(2,1) and satisfying I(~~ K)(~)I 5.. Cg exp(-II!II!), all ,I~I 5.. d+5 and
t eR d . (The hat denotes Fourier-Sti el tjes transform.) Set
Ko(E) ::: K(o-I E) for 0< 0 < 1.
If l dHI 5.. CIa
(5.6)
A
Arguing as in [6, p. 210] we obtain:
sup
fl(D§-g~)(t)(Dg~8)(t)ldt
d
5
~5..~2.@, I ~ l2. +
~
~
d+2
+
J HAt AB(~t2o) ~ ltnlr~o
r 2
n- / Pr(-<I.>O,v t :
where A denotes symmetric difference.
do not depend on 6.
{X~})ld~,
The constants Cj here and below
Following the argument of [6, pp. 210-211], with
a little modification, we derive for a certain random variable An:
@
-1-£2
1f{1I~II5-An}I(D -gH)(~)(D~K8)(~)ld~ 5- Cn n
,
A
(5.7)
sup
A
~~~5-~, I~ l~d+5_
(5.8)
where K:::
sup
f {II ~ II >
95-~5-0' I~ I~d+5_
c n = n;/06
E(II~113IY)} .
C 11
IT
(D0-~Qt )(,~)( D~Kc;)(!) Id! and
-22-
Let
n
rll exp(i<
~n(!)= n- l
!'~r
» denote the (empiric) characteristic
function of Y, and ~(t)
the characteristic function of Y.
_ =E{~n (t)}
_
_
We
shall need:
LEMMA 5.1.
There exist constants C €(O,1), C and C such that whenever
i4
12
13
n >d+5, m€ [1,n-d-5], 0 < 6 < 1 and Y€
PROOF.
(5.9)
2n!.)
K
2. C9 (2n )
d+5
1~II>cn
11~-l~II>C15 nl
(5.10)
Then
IY}.
I(DyQt)(!) I.:::
0(:) = qn(:),
(2n) ly1Iq(:) :n-Iyl.
from
(Note that
Therefore if n > d+5,
If Y€C n5 then
that
~i' »
Set q(!) =: E{exp(n-! i <!,
which it may be deduced that
I~r I 2.
en5 ,
Iq(~)1 ~
.
J{II~II> cjq(~
) n-d-5
I
implies that 16
(
. l)
exp -1I6~1I d~.
c4Eql~I~IY)II~-l~ll>nl,
and hence
Now,
IE{exp(n-
l
i<~,~-l~
~ I~n(n-! ~-I~)1
»IY}I +
+ C n16
p(II~II>nlIY)
l
Results (5.9) and (5.10) together give:
(5.11)
K
~
1
-l
{Ilih(n-!t) I + C16 n- }
{II n ~ II > C1~
-
d 5
C n + J
17
x
eXP(-C41118~1I1)d~.
Let n3 =n3 k
)€[
2
sup !4J(u) Ii. max(nl,1-n2 ~ )
II ~ II > ~ sup 1l/J(u)l,l). If
Choose nl € [1,1) and n2> 0 such that
for all ~>O.
n-d-5
II~II>
r,
~.
-23-
•
l\jJn(~) I ~ n3(l+n4) for II~ II > E;,.
If l\jJn(~) - \jJ(~) I > n41\jJ(~) I then
l\jJn(~)ln-d-5 ~ {(l+n41)I\jJn(~)_\jJ(~)I}m. Finally,
if
1\jJ(~)1 ~ n3 then
-1 n-d-5
(
)
n-d-5
{IIJh (~ )1 + C16 n}
:5_ exp CI6 /'l3 n3
. Combining these results
we conclude that for any II~II >E;"
Lemma 5.1 follows from this inequality and (5.11), on taking
max H ,
sup
II~II>F,
I\jJ ( u) I} ,
-
= C15 , n3 =
n4 =! if n3 =! ,
n4 = (l/2)[{max(nl,l-n2 t.;2
if
E;,
n-l_ 1]
n3~i, and CI2 =max{(l/2)(1+n 1 ), 1-(l/2)n2
0
E;,2}.
Choose 8=n-\, where >-..>0 will be selected shortly.
uniformly in u, then by Markov's inequality, the set
Since
-24-
satisfies P(C ) =O(n d (H1)+7 n-m/2) =o(n- 1 ), provided m is chosen
n6
sufficiently large.
Ye C :: C n C .
n7
n5
n6
Henceforth we shall work only with samples
In that case, Lemma 5.1 implies
•
K
2. C18 n
d(A+1)+5
n
m
-2
-2
C + C C n
.:5. C19 n ,
13 14
12
and so by (5.6), (5.7) and (5.8),
(5.12)
IJ t
A
dH
I 2.
-1-£2
C20 n
+ ( ,
H2
where £,:::
L
£'r '
r=O
r 2
[,r=n- /
JI(~)IPr(-<l>o,vt:{x~})ld~
and I(x)==I{At==B(x, 2n- A)f0}.
An argument using (9.12) and (14.74) of [6, pp. 72 and 133] gives
Let
~ == nt ~t E(~il Y),
_ t
B( ~ ) = ~
B(~
and note that for Ye Cn7 '
-t ~,2n -A)
_ (
::- B ~, C22 n
-A)
.
Thus,
~ C23 JI{ (~+~) L'l B(~) 10}
exp( -0241 ~ 112)d~
2
~C23 J(oA)n exp(-C2411~+~11 )d~,
where n -= C
n-A •
22
h
An e1ementa ry argument shows tat
-25-
for all
~, 't. e ]Rd.
inequality.
3 2
If Ye C then II~II ~ C26 n- / , using Markov's
n7
Combining these estimates we conclude that
and so by (4.12),
(5.13)
We now restrict attention to sets A e {A(t): -oo<t<oo}, where
LEMMA 5.2.
P(C
nS
Take;\
=1+£2
and
-;\
T)
=C n .
22
'l'here exist8 CnSec
- n7' with
) =o( n-1), 8uch that
PROOF.
Suppose g(~Iv.) and its first four derivatives with respect to u
are continuous in
(~,~)
€ B(~,P1) x
not all tenns gi(~I~) can vanish.
B(~,P1).
In view of property (2.4),
Let P2€ (O,p/2J be such that for
some i, gi(~I~) is bounded away from zero in B(~,P2) x B(~'P2).
Cn9 be the class of all samples Y for which 11!-~II~P2/2.
~
-1
=C n Cn9 satisfies P(C nS ) =o(n
nS n7
C >0 and some io€{l, .... d},
29
C
inf
Ig· (YIV) I > 1/C 29 ·
Y€ C
'0 - '-'
nS
that Ye C .
nS
and
Let
The set
), and has the property that for some
We assume throughout the argument below
-26-
For any vector
T
~
d
= (xl' ... ,x d ) eR , define the random-coefficient
cubic polynomial
d
Pn(~):: ;~l
i
xig i (!IY) + (1/2) n-
gCy + n-i~IY) = Pn(~) +
~
d
L L
. I
1. = I J=
l
+ (1/6) n-
Then n!
d
d
x.x.g . . (VIV)
d
1 J lJ
~
-
d
L L L
;=1 i=1 ;=1
x.x.xkg .. k(VIV).
1 J
lJ
~ ~
L\nl (~), where
3/2 (log n)4 .
sup
lL\nl (x) I 2. C n30
e 8( 9, 1og n)
-
Therefore, rememberi ng that n = C n~
22
and A = I +£2:
i
n
g(! + n- I ~21!) = t}
.:. {AI (t)}u 8(0, log n)- ,
The lemma follows easily from this result and the statement within
quotation marks below:
IILet ~ have the
N(9,n
distribution on R d , let f3, b> 0, and let
_ d
_! d d
-1 d d d
r (x) - L c.x.+n
L L c .. x.x. + n L L L c. ·kx.x.xk
i=1 1 1
i=1 j=1 lJ 1 J
n i=1 j=1 k=1 lJ 1 J
be a cubic polynomial whose symmetric coefficients ci,c ij and c ijk
satisfy
su p
,. ,J. , k
(I c 1. I, Ic lJ.. I' Ic lJ.. k I)
<
-
b, and for some i,
Ic ,. I
>
lib .
-27There exists C> 0, depending only on
sup
p{
-oo<t<oo
•
band d, such that
11"
I~II ~ b log nand r (X) e [t-n-B , t+n-Sn < C(B ,b,d)n-B
n -
-
-
for all n > 1. 11
Our proof of this statement is by induction over d.
; s obviously true if d=l.
The statement
Suppose it; s true for d-1, some d> 2.
Without loss of generality, Icdl is the amaUeat Ic;l.
Let
~*=
(Xl'".,X _ )T,
d 1
~*= (x 1 ,.",x d _1 )T and
-1
2
d-1 ( - 1
c;+2 n
c;d xd + 3 n
c;dd xd ) x;
r~(~*lxd) =: i~l
+ n- i
+ n- i
d-1 d-1
L L
;=1 j=l
(c .. +3 n- 1 c"d xd)x.x,
lJ
lJ
1 J
d-1 d-1 d-1
L I
I
d-1
= I
;=1
say.
C"k x,x,x k
lJ
1 J
;=1 j=l k=l
d-1 d-1
ci Xi + n-!
I
1 d-1 d-1 d-1
I c~,
;=1 j=l
x,x. + nlJ 1 J
I
I
I
ciJ'k x;xjx k '
;=1 j=l k=l
For all n.::no(b), we have Icil, 1Cijl and 1Cijkl ~ 2b for all ;,j,k,
and Icil > 1/2b for some i, no matter what the value of Ixdl ~ b log n .
Consequently,
sup P{I~II.:.
-oo<t<oo
..
<
-
b log n
and rn(~)
€
[t-n- S , t+n- S ]}
sup
P{ I~* lI.::.b log n and r*(X*lx d )
n-oo<t<oo Ix I~b log n
.~
d
sup
€
n
[t-n- f3 .t+n- S
.::. C(B ,2b,d-l)n-S .
This proves the statement, and so completes the proof of Lemma 5.2.
0
-28-
Combining (5.5), (5.13) and Lemma 5.2, we conclude that if Ye C '
n8
(5.14)
sup
-oo<t<oo
The remainder of the proof, which only involves unravelling terms in the
polynomial expansion (5.14) to obtain the terms in (5.1), may be conducted
by modifying arguments used to establish Lemma 2.1 and Theorem 2(b) of
[5, pp. 443-444 and 445-446].
The cubic polynomial technique used to
prove our Lemma 5.2 may be employed to overcome minor technical difficulties.
The set c~l) is a suitably chosen subset of cn8 .
Step (ii). Here we develop an approximation to t .
ex
2
(5.15)
t
+ ==
a,_
inf{t: ¢(t) +
I
j=l
n-
j 2
/
Define
-1-£
f[lJ.(t)</>(t) + C1 n
1~
a},
where C1 and £1 are as in Proposition 5.1 and the +,- signs are taken
respectively. The left-hand side of the inequality within braces in
-1-£
(5.15) converges to 1 + C n
1 as t-+ oo • Therefore the set in (5.15)
1
is quaranteed nonempty, and t + well-defined, if we confine attention to
a,-1-£
-1-£1
a € T == [3C n
].
1, 1-3C 1 n
1
Notice that </>(log n) and l-¢(log n) are both O(n- A) for all A >0.
This observation and inequality (5.2) lead us to conclude that for some
C32 >0, sup It +1
aeT
a,-
~ 10gn provided
By Proposition 5.1, if
YeC(l) and
n
n~C32'
Yec~l) and P{n 1 g(~I!) ~ tlY} ~.a then
-1-£
P{n 1 g(~I!) ~ tlY} + C1 n
1 - 6n(t) ~ a.
-29-
Therefore, t a,- --< t a , and 1ikewise, t a, + -> t a , whence for Ye CO):
n
(5.16)
...
t (-\,- -< t a -< t a,+ .
Here we invert the expansion defining t a,±.. We begin
by discussing the deterministic polynomials TI 1j and TI 2j , which are
step (iii).
related to one another as follows.
(5.17)
x = x(y) = y +
Then ¢(x) +
~
j=l
uniformly in Iyl
(5.18)
¢(x) +
uniformly in Iyl
2
r
j=l
'/2
n- J
Let
TI .(y)
2J
n- j / 2 TIl ,(x)¢(x) = ¢(y) + 8 (y), where 1 8 1
1
1
J
~
log n.
2
r
j=l
n- J
~
log n.
./2
<
C n- 3/ 2
33
Thus,
TIl .(x)¢(x) ~ ¢(y) - C33 nJ
3/2
Let z=z(o,) be the solution of ¢(z)=a, and
write t=t(a) for the solution of
2
(5.19)
¢(t) +
r
j=l
j
n- / 2 TIlJ',(t)¢(t) = a.
Let T be the set of values of t such that the left-hand side of (5.19)
-1-£
-1-(
lies within Tl =[C l n
1, 1-C n
1]. If n is sufficiently large
1
then the left-hand side is a strictly increasing function of t for
teT, and so if ().eT l then t().) is uniquely defined.
-1-(
Assume aeT(.=.. T1 ),
and take y=z(a+ C1 n
1+C 33 n- 3/ 2 ) in (5.17) and (5.18). Let
-1-c
t l = t(a+ C1 n
1). For sufficiently large n, each of x(y), y and t l
is in T for all a€T. By (5.17) and (5.18),
-30-
<I>(x) +
2
n- j / 2
I
j =1
TI
.(x)¢(x) ~ (a+C
1J
-1-£
n
1 + C n- 3/2) C n- 3/ 2
1
33
- 33
n
= <I>(t l
Consequently, x
y +
~
I2
j=l
t
l
;
)
+
I
j=l
n-j/2
TI ••
lJ
(tl)¢(t l
).
that is,
'/2
n- J
TI
2 -j/2
() ( )
-1-c 1
~ n
lT · t ¢ t - C n
>cd.
1
2J,(y) -> inf{t: <I>(t) + j;1
1J
Translated to the random polynomials "lj and n2j , this argument
suggests the following, There exist positive constants C34 and C3S such
.
_
-1-c 1
-3/2
that, wlth y+ =z(a+ C1 n
+ C34 n
), we have
(
)
5.20
y+ +
~ n-j/2 TI , (y+)
2J
j=1
l
whenever a € T and
~
t a ,+
C3S . Techniques used early in the proof of Proposition
5.1 show that this translation is correct, provided we make the restriction
Ye C(2) for a suitable C(2)c eO ) with P(C(2)) =0(n- 1 ). The proof becomes
n
n - n
n
quite straightforward when it is noted that the constant C33 cited earlier
n~
may be taken to be a continuous function of the first four population
moments.
To obtain the constant C34 , we should ensure that the sample
moments are sufficiently close to the population moments with probability
1-0(n- 1 ) - see the second sentence of the proof of Proposition 5.1.
Of course, inequality (5.20) has a counterpart providing a lower
_
-1-£1
-3/2
bound to t a ,_. Let y_ z(a-C 1 n
- C34 n
) . Then
0
provided a € T, n2. C3S and Y € c~2),
-31-
Here we combine Steps (i) - (iii).
Step (iv).
Note from (5.16),
(5.20) and (5.21) that
2
y_ +
"
L
j=l
(~e T,
provi ded
n- j / 2 n .(y )
~
2J - .'5- to, .:: y + + .I..
J=l
n.:. C and Y e c~ 2 ).
35
n
-j/2 IT
( )
2J' y + '
Therefore
(5.22)
provided (~€ T and n~'C35'
If we prove that, with z=z(a),
(5.23)
uniformly in
Ct
€
Tl' then it is immediate from (5.22) that expansion (2.6)
holds uniformly in aeT.
below.
In the meantime, Step (v) extends (2.6) from a e T to a
Let
titep (v).
Since t a
The case
We shall establish (5.23) in Step (vi)
0.
=3C 1
1
-1-E
1 d
nan
0.
=1-3C 1
2
-1-E
n
1
.
Then T = [0.
is nondecreasing in a, and since (2.6) holds for a
0.
2 < (~ < 1 may be treated simi 1arly.
€
€
1
(0,1).
,0.
2
],
T, then
-32-
Step (vi).
Here we prove (5.23).
That result is of the type
established as Theorem 3 of [10], and may be proved in the same way.
As in [10], the argument rests heavily on ideas from [5].
The only
real difference here is that in (5.23), the expansion is to hold over
slightly more than just bounded z. However, the terms n- j/2 multiplying
the polynomials, and the fact that Iz(a)1 is not larger than const.
(logn)l fora€T 1 , ensure that this extra generality is easily
achieved. For example, the polynomial TI 22 depends on Y only through the
first four sample moments.
If Mis anyone of these moments and if
0< p < E:/2(8+E:), then for any c> 0,
Let Cn,10 be the set of samples Y for which the difference between any
p
coefficient I' of TI 22 and its limit y= p limI' satisfies Ir-yl < n- •
~
-1
In view of (5.24), P(C n ,10) = o(n ), and also for n:,- 2,
sup
n-11 n22 (z)-1T 22 (z) I
Y€C n ,10 z(a):aeT I
sup
~C36n -1
2
(-1) ,
n-p( logn )5/ =on
since TI 22 is of degree 5.
For these reasons we shall give only an identification of expansion
1
(5.23). Following [5,10] we work with Taylor expansions to order nj
in probability. (liTo order n- j/211 means that terms of order n- /2 are
included but smaller terms are excluded.)
notation and define L':.ri
c.::
Yri -
~i'
-1
L':.i- n
We shall use the summation
n
I
r=1
6 ..
rl
-33-
According to Lemma 1 of [10], 'n
21
(z) =-"Tn(Z) =a +( a /6)(i-1),
3
1
where a.:: (1/2) ~ .. a .. and a3=L£.~k a"k+3~'~'~k a ..a k .
,
'J 'J
, J
'J
, J m 'J m
But
by (2.4), ~i~jaij = 1, and so
"
1f
21 ( z) = (1/2) z
The sampl e vers ions of
2 to. a.. + (1/6)( z 2-1) ~ . Q, . ~ k a .. k .
'J 'J
, J
'J
~.
. and a.
. used to construct Il
are
21
'1""p
'1''''p
and
_ -1
W.
..:: n
, 1""p
n
L
r= 1
(l'lr,' .. ·l'lr,· p
1
0..
.) •
'1""p
Therefore to order n-!,
Il
21
(z),.~ 1T
+
•
21
(Z) + (l/2)z 2
(k)
a .. -l'lk+£" W•. -2Q, ..a.l'l.)
'J'J
'J'J
'J' J
(~ ..
(1/6)(z2_1)(3Q,.~·~k(m)a··k
Ii +Q,·Q,·~k
, J
'J
m , J
By definition, each ai=O.
To order 1, Il 22 (z) ~n22(z).
-1!
2
r 12
n , n g(YI~) - r~1 nIT 2r (z) equals
W.. - 3£'£'£k a ..
'J k
, J
'J
lik )·
Therefore to order
-34-
(
2( Q,..
(k) a .. -!'1 + 9, •• W.. )
U*-=U-n -! {1/2)z
1J
1J
k
1J
1J
(m)
- )
2
+ ( 1/6 ) ( z -l)(3~·~·9,k a·· k !'1 + Q,,9,·Q,k W" - 3~·Q,.Q,ko.·· !::,.k } ,
1 J
1J
m 1 J
1J k
1 J
1J
where U:::n 1{Q,. 6.+(1/2)L.
1 1
1J
6.6.+(1/6) L' 6. X. A } 1 J
1J k 1 J k
I
r= 1
n- r / 2TT
2r
{Z).
Let Ki and Ki denote the i'th cumulants of U and U*, respectively.
1
With approximations holding to order n- , we have Ki = Kl' Kj =K ,
3
K*2
_
=
2
(k)
- -
--
K - [z {L. Q, a .. E(!::,.k!'1 ) + 9, .. 9, E(W .. !::")}
2
1J P 1J
P
1J P
1J P
2
(m)
-- + (1/3)(z -l){39,·Q,.9,k 9, o"'k E(f-I f-I ) + Q"Q,.9,k Q, E(W. ·k!::,. )
1 J
P 1J
mp
1 J
P
1J P
* _
2- (k) 3
- -3
3
-3 )
K = K - 2n [zl.£.. Q, a .. E(!::,.k!::,. ) + 9, .. 9, E(W .. !::" }
4
4
1J P 1J
P
1J P
1J P
3
--3
- 3Q,·Q,·~k9, a .. E(f-I l'1 )}]
k P
1 J
P 1J
--+ 6[z 2 {9, (k)
.. Q, a .. E(l'1
f-I }
6 ) + 9, .. 9, E(W .. --)
k P
1J P 1J
1J P
1J P
- 3Q,. 9, .Q,k9, a.. E(EkE )}] .
P 1J
P
1 J
(
3 ) - 2 a·
Notice that E(W.
. -!::,..) = n-1 a·
. . , E W.
. 1'1. - 3n
. . a··
11 , .. 1P J
11 ... 1pJ
11 ... 1P J
11 ... 1pJ JJ
as n +00, and l'1 i = Wi'
4=K4 - 6n-
K
1
-1-1
Therefore to order n 'K2 ~ K2 -n
02(z) and
04(z), where ()2(z) :: ): L: ): Y.. (z)
.
1 J P 1JP
,
-35-
y .. (z):: z2(R,
lJP
..
P
a .. L
lJ k
R,~~)
lJ
a
kP
+ R, •• R, a .. )
lJ P
lJP
- 3R,iR,/,p a ij ~ R,k a kp )'
(These expressions are not be interpreted in the convention of summation
notation.)
(5.25)
Consequently,
P(U*2X) = P(U 2 x) - n- 1{(l/21)02(Z)(;)/;)X) + (l/41) 60 (z)(a/dx)3}<p(x)
4
+ 0(n- 1 )
2
1
= P(U2.x) + (x/4n){202(z) + 04(z)(x -3)}<P(X) + 0(n- )
uniformly in -oo<X<oo and z=z(Cx.) with a
7T
22 ensures that P{U
~z(a)}
€
Tl'
=0. + 0(n- 1 ).
1
expansion, to order n- , as n;
2
g(!I~) - r~1
The definition of
7T
21
and
Finally, u* has the same
r 2
n- / IT 2r (z).
Therefore
(5.23) follows on taking x=z in (5.25), and noting that
o
•
REFERENCES
1.
ABRAMOVITCH, L. and SINGH, K. (1984). Edgeworth corrected pivotal
statistics and the bootstrap. Ann. Statist.
2.
BABU, G.J. and SINGH, K. (1983).
Ann. Statist. 11, 999-1003.
3.
BERAN, R.J. (1982). Estimated sampling distributions:
and competitors. Ann. StatiDt. 10, 212-225.
4.
BERAN, R.J. (1984).
Inference on means using the bootstrap.
the bootstrap
Jackknife approximations to bootstrap estimates.
Ann. Statist. 12, 101-118.
•
e
5.
BHATTACHARYA, R.N. and GHOSH, J.K. (1978). On the validity of the formal
Edgeworth expansion. Ann. Statist. 6, 435-451.
6.
BHATTACHARYA, R.N. and RANGA RAO, R. (1976). Normal
Asymptotic Expansions. Wiley, New York.
7.
BICKEL, P.J. and FREEDMAN, D. (1981).
Ann. Statist. 9, 1196-1217.
Some asymptotics on the bootstrap.
8.
EFRON, B. (1979).
another look at the jackknife.
Bootstrap methods:
Ann. Statist. 7, 1-26.
9.
EFRON, B. (1982). The Jackknife, the Bootstrap and
Plans. SIAM, Philadelphia.
and
Resampling
Inverting an Edgeworth expansion.
11. HALL, P. (1985).
A tabular method for correcting skewness.
Ann. Statist. 11,
Math. Proc.
phil. Soc.
12. PFANZAGL, J. (1979).
Nonparametric minimum contrast estimators.
Selecta Statistica Canadiana,
13. SINGH, K. (1981).
•
othe~
10. HALL, P. (1983).
569-576.
Camb~idge
•
App~oximation
v.
In
On the asymptotic accuracy of Efron's bootstrap.
Ann. Statist. 9, 1187-1195 .