I
I.
I
I
I
I
I
I
I
I'~
I
I
I
I
I
I
I
,e
I
ALMOST SURE BEHAVIOUR OF U-STATISTICS AND
VON MISES' DIFFERENTIABLE STATISTICAL FUNCTIONS
By
Pranab Kumar Sen
Department of Biostatistics
University of North Carolina, Chapel Hill, N. C.
Institute of Statistics Mimeo Series No. 807
February 1972
I
I.
I
I
I
I
I
I
I
I_
ALMOST SURE BEHAVIOUR OF U-STATISTICS AND VON MISES'
DIFFERENTIABLE STATISTICAL FUNCTIONS l
BY PRANAB KUMAR SEN
University of North Carolina, Chapel Hill
For U-Statistics and von Mises' differentiable Statistical functions, when
the regular functional is stationary of order zero, almost sure convergence to
appropriate Wiener processes is studied.
A second almost sure invariance prin-
ciple, particularly useful in the context of the law of iterated logarithm and
the probability of moderate deviations, is also established.
AMS 1970 Classification No. 60BlO, 60F15, 62E20.
Key Words and Phrases.
Almost sure convergence, invariance principle, Wiener
process, U-statistics, von Mises' functionals, law of iterated logarithm and
probability of moderate deviations.
I
I
I
I
I
I
I
II
1) Work sponsored by the Aerospace Research Laboratories, Air Force Systems
Command, U.S. Air Force, Contract No. F336l5-7l-C-1927.
Reproduction in whole
or in part permitted for any purpose of the U. S. Government.
I
I.
I
I
I
I
I
I
I
I_
~.
distributed random vectors (iidrv) defined on a probability space
,e
I
(n~,p),
with
each Xi having a distribution function (df) F(x), XERP, the p(~l) - dimensional
Euclidean space.
Let g(Xl""'Xm), symmetric in its m(~l) arguments, be a Borel
measurable kernel of degreem, and consider the regular functional
8(F) =
(1.1)
where ~= {F: 18(F)
I
!-Rpm
..
<oo}.
fg(xl, ... ,x )dF(xl) ... dF(xm); FE~
m
The minimum variance unbiased estimator of 8(F) based
on a sample Xl""'Xn of size n is (the U-statistic)
(1. 2)
Un = (n)-lI c
g(X. , ... ,X.); C
= h<il< ... <i <n},
m
1. 1
1.m
n,m
mn,m
If we let c(u) to be equal to 1 if all the p components of u are
non~negative
and otherwise let c(u)=O, then on defining the empirical df
Fn (x) = n -II 1.=
. n lC(x-x.),
XERp ' n_>l,
1.
(1. 3)
I
I
I
I
I
I
I
Let {Xi,i~l} be a sequence of independent and identically
the corresponding functional
e(F ) = f .. ·Ig(xl, .. ·,x ) dF (xl)' .. dF (x )
n
Rpm
m
n
n m
(1. 4)
=n
-m\
\ n
Li =l'''Li =1 g(~ , ... ,~ )
1
m
1
m
is termed a von Mises' (1947) differentiable statistical function.
Asymptotic normality of
n~[Un -8(F)]
von Mises (1947) and Hoeffding (1948).
and
n~[8(Fn )
- 8(F)] are studied in
Under the same set of regularity condi-
tions, Loynes (1970) has shown that a process obtained by linear interpolation
1
from {n~[Uk-8(F)]; k~} weakly converges to a Wiener process, as n~.
Also,
Miller and Sen (1972) have shown that under the same set of conditions, pro_1
cesses obtained by linear interpolation from {n~k[Uk-8(F)], m<k<n} and
{n~k[8(Fk) - 8(F)], l~k~n} converge in distribution in the uniform topology on
I
I.
I
I
I
I
I
I
I
I_
2
the C[O,l] space to a common Wiener process, as
n~.
In the present paper, among
other results, these weak convergence results are strengthened to almost sure
(a.s.) convergence results.
For sums of independent random variables and martingales, Strassen (1967)
studied three a.s. invariance principles.
His first invariance principle, namely,
the a.s. convergence of sample partial cumulative sums to Wiener processes is
extended here (see Theorem 2.1) to a broad class of {U } and {8(F )}. In Theorem
n
n
2.2, we consider a result analogous to his third a.s. invariance principle, and
this is particularly useful in the context of the law of interated logarithm and
the probability of moderate deviations for U and 8(F ), for which we may refer
n
n
to Rubin and Sethuraman (1965) and Ghosh and Sen (1970) for earlier works.
The basic results along with the regularity conditions are stated in section
2.
The proofs of the theorems are presented in section 3.
are sketched in the last section.
Define for every h
I
I
I
I
I
I
I
I·
I
A few applications
(O~h~),
Eg(x l ··· ,xh,Xh +l ,··· ,XmL g0::::8 (F) ;
(2.1)
gh(xl,···,xh )
::::
(2.2)
I;;h (F)
::::
2
Eg~ (Xl' ... ,Xh ) _8 (F) , I;; 0 (F)::::O;
(2.3)
1;;* (F)
::::
max
l<i <... <i <m Eg 2 (X.1 ,H',X.1 ).
- 1- - mm
1
We term that 8(F) is stationary of order zero [cf. Hoeffding (1948)], if
(2.4)
0 < 1;;1 (F) <
00.
Let S :::: {set): O<t<oo} be a random process, where
(2.5)
S(k)
r
: : Sk
and Set) :::: Sk for k<t<k+l, k>O.
0,
O<k<m-l,
::::~k[Uk-8(F)], k~,
Similarly, let S* : :
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
3
{S*(t): O<t<oo} be a random process, where
fO,
(2.6)
S* (k) = Sk
and S*(t) = Sk for
k=O
= lLk [8(F )-8(F)],
k
k~t<k+1, k~O.
Alternatively, Set) [and S*(t)] can also be
defined by linear interpolation between (Sk' Sk+1) [and Sk' Sk+1)] when
tE[k, k+1],
k~O.
Consider now a positive and real valued function f(t),
tE[O,oo), such that
1
f(t) is t but t- f(t) is ~ in t: O~t<oo;
(2.7)
<
(2.8)
00
,
for every c>O, where gi(x) = gl (x) - 8(F) and leA) denotes the indicator function of a set A.
Finally, let
{~(t):
O<t<oo} =
~
be a standard Wiener (Brownian
motion) process, and we let
y2 = m2~1 (F) (>0 by (2.4)).
(2.9)
Then, the following theorem extends Strassen's (1967) first a.s. invariance
principle to {U } and {8(F )}.
n
n
THEOREM 2.1.
If 8(F) is stationary of order zero and
and (2.8), as
t~,
Set)
(2.10)
=
S* (t) =
(2.12)
Set) - S*(t) =
Let now ¢ = {¢(t):
{¢' (t)}, such that as
O~t<oo}
t~,
then under (2.7)
y~(t) + o(tf(t))~ log t) a.s.
Also, under (2.4), (2.7) and (2.8), if
(2.11)
~m(F)<oo,
~*(F)<oo,
then as
y~(t) + o((tf(t))~ log t)
t~,
a. s.;
o((tf(t))~ log t) a.s.
be a positive function with a continuous derivative
I
II
I
I
I
I
I
I
Ie
I
I
I
I
I
I
I
II
4
(2.13)
sit
-+ l~<j>'(s)/<j>'(t)
-+
1,
t-~ <j>(t) is t but t-h<j>(t) is ~ in t for some ~<h<3/5;
(2.14 )
oo
f1
(2.15 )
t
-3/2
<j>(t)exp{-~t
-1
<j>2(t)} dt <
00.
Then, as an extension of Theorems 1.4 and 4.9 of Strassen (1967), we have the
following theorem where we define {S } and {S*} as in (2.5) and (2.6).
m
n
THEOREM 2.2. If g(Xl, ... ,Xm) has a finite moment generating function in a
neighbourhood of 0, then under (2.4), (2.13), (2.14) and (2.15), as n+oo,
(2.16)
while, if in addition, g(X. , ... ,X. ) has a finite moment generating function
1
1
1
m
l~il~... ~im~m,
in a neighbourhood of 0 for every
then as n+oo,
(2.17)
The proofs of the theorems rest on the reverse martingale property of {U }
n
[See Berk (1966)], certain related results studied in Miller and Sen (1972) and
the basic theorems of Strassen (1967).
We shall see in section 4 that Theorem
2.2 strengthens some earlier results of Rubin and Sethuraman (1965) on U-statistics.
Proceeding as in Miller and Sen (1972), we
have for every
n~l,
(3.1)
8(F n ) = 8(F)
V(h)
(3.2)
and for every
(3.3)
n
+
Lh:l (~)V~h);
h
= I··
· fgh(x1""'~ ).nld[F (x.)-F(x.)],
RP h
n Jn J
J
n~,
U = 8(F)
n
m (m)U(h) U(l) = Vel).
Lh=l h n ' n
n'
+ \
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
5
U~h)
(3.4)
= n-[h]lp
n,h
I· p·h Igh(Xl, .•. ,Xh).Uld[C(X.-x.
R
where n-[h] = {n ... (n-h+l)}-l and Pn,h =
J
{12il~ ... ~ih2n},
We start with the proof of (2.10) in Theorem 2.1.
J
J. j
)-F(x.)],
J
h=l, ... ,m.
Let us define
(3.5)
Also, let .-?(l) = {S(l)(t)·. O_<t<oo} J.·s defJ.·ned by lettJ.·ng for t E[k , k+l) ,
s(l)(t) = S~l), k>O.
Further, let
\ m (m)U(h)
>
Un* = lh=2
h n ' n m.
(3.6)
Then, to prove (2.10), it suffices, by virtue of (2.5), (3.1), (3.5) and (3.6),
to prove that as
t~,
1
S (1) (t) = y~(t) + o((tf(t))~ log t) a.s.,
(3.7)
and as
n~,
(3.8)
Now, since
iidrv
S~l) = kU~l) = Li~l[gl (X i )-8(F)], k~l, involve the sequence of
{g~(X.) = gl(X )-8(F); i>l} where Eg*l(X,) = 0 and E[g*l(X.)]2 = sl(F) (>0),
J.J.
i
J.
J.
and (2.7) - (2.8) hold, the proof of (3.7) follows directly from Theorem 4.4 of
Strassen (1967).
1
(3.10)
For this, on,writing
1
~ = k[(kf(k)~ log k]- , k~l,
(3.9)
we note that
So, we need to prove only (3.8).
{~}
a sequence of positive numbers such that
ck is t in k [by (2.7)].
Now, {Uk' k>m} being a U-statistics sequence is a reverse martingale with respect
I
6
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
,.
I
to a non-increasing sequence of a-fields [cf. Berk (1966)].
Further, as in
Hoeffding (1948), it can be shown on using (3.6) that for every
E[U*-U*
(3.11)
n
2.
Finally, as in (3.11),
n+1
C(F)n
E[U~]
2
]2 = E[U*]2 _ E[U*
n
n+1
n~,
]2
-3 , where C(F) <00 when z:; (F)<oo.
m
~
C(F)n
-2
for every
n~m,
so that by (2.7) and (3.9),
(3.12)·
Consequently, by (3.10), (3.11), (3.12) and Theorem 1 of Chow (1960) [i.e., the
3 2
Hajek-Renyi inequality for sub-martingales], we obtain on noting that c n2 =0(n / )
[by(2.7) and (3.10)] that for every £>0,
(3.13)
Thus, (3.8) holds and the proof of (2.10) is complete.
of (2.12).
We next consider the proof
Proceeding as in the proof of Lemma 2.6 of Miller and Sen (1972), it can
be shown that for every
n~,
I
I.
I
I
I
I
I
I
I
I_
7
E{n[e(Fn ) - Un ]}2 -< C*(F)n- 1 ,
(3.14)
where C*(F) <
00
whenever
~*(F)
<
00.
Therefore, for every E > 0,
p{~~~[IS(t) - S*(t)I/[(tf(t)~ log t]]
(3.15)
~ P{~~~[k\e(Fk) - ukl/[(kf(k))~log k]]
00
~
I
k=n
> E}
> E}
1
p{k\e(Fk ) - ukl > E[(kf(k))~ log k}
~ C*(F)E- 2 Ik:nk-1[(kf(k))~ log k]-2
= C*(F)E -2 [fen)] -1/2 (log n) -2 [O(n -1/2 )]
-+
0 as n+oo.
Thus, (2.12) follows from (3.15), and (2.11) follows directly from (2.10) and
(2.12), and hence, the proof of Theorem 2.1 is complete.
I
I
I
I
I
I
I
••
J
llll....\
For the proof of Theorem 2.2, we first consider the following.
LEMMA 3.1.
For even
kn(2~kn~0(n~)), as n+oo, for every l~h~m,
E[nh/2u(h)]kn < [C (F)]knf (hk n )!
j{1+0(1)}
n
h
~hk
'
k
2 n(~kn)!
where E[lg(x , ... ,x )1 n] < oo::>Ch(F) < 00 for h=l, ... ,m.
m
1
( 3.16)
Proof.
We sketch the proof only for the case of h=2; the same proof holds for
every l<h<m.
(3.17)
By (3.4) and the Fubini theorem,
I
I.
I
I
I
I
I
I
I
8
where the summation 2~ extends over all 1 ~ ill
For a given set {in., j=1,2,l=1, ... ,k } of 2k
n
~J
n
+ i l2
~ n, l=l, ... ,k .
n
integers,
suppose that there are s
rk(~l)
distinct integers j1, ... ,j ,s >1, where jk occurs
n
s
nn
times, so that r +... +r
= 2k. Then,
1
s
n
n
k
2
..
IE{l~~ j~l d[c(x 1J
(3.18)
-X.
11j
) -
F(xo.)]}1
~J
= 0, if at least one of r , ... r
= 1,
k
1
n
k
2
< TIn TI dF(x ) otherwise,
- l=l j=l
lj'
for every set of {ilj' j=1,2,l=1, ... ,k }. Thus, the leading terms in (3.17) arises
n
k
from sets for which s n -- kn' r 1 -- '" -- r k = 2; there being [(2k )!/2 n. k !] such
n
n
n
sets, their total contribution in (3.17) is bounded by
I_
(3.19)
I
I
I
I
I
I
I
••I
k
k
= [C (F)]n.[(2k )l/2 n. k !],
2
n
n
k:
where C (F) < [sm(F) + e2 (F)] 2 < 00 whenever sm(F) < 00. The other sets with non2
zero contributions in (3.17) correspond to values of s n -< kn -1 with r -> 2 for
k
k=l, ... ,s.
n
If s
n
= k -u, u>l, and E\gll+U <
n
-
00
(as assumed), then the contribution
of the sets to (3.17) is bounded by (3.19) times a coefficient which is
(3.20)
O[(k 2 /n)u], for u=l, ... ,k -1.
n
n
Thus, the total contribution of these sets (with sn<kn -1) is bounded above by
(3.19) times a coefficient
(3.21)
I
I.
I
I
I
I
I
I
I
9
Hence, the lemma follows.
A direct consequence of (3.6) and Lemma 3.1 is the following.
!f Eigi
LEMMA 3.2.
k
n <
then for every even
00,
kn[2~kn=0(n~)], ~ n~,
k
k
k
k
E[nU*] n < (m ) n[C (F) n{(2k )!/2 n k !}{l+o(l)}.
n
- 2
2
n
n
(3.22)
[Note that in deriving (3.22), we make use of the fact that for every h>2,
k
[n-(h-2)/2] nn(h k )!/(2k )!][k !/(~hk )!]2-(h/2-l)kn }-+0 as n~.]
n
n
n
n
Also, by using (3.1) - (3.3), and proceeding as in Lemma 3.1, we have the
following.
LEMMA 3.3.
even k
k
!f E!g(X i
lor
n
1
""'X i ) I n <
[2<k =o(n )], as
-n
-
n~
00
for every l~il~"'~m~m, then for every
m
I_
--
I
I
I
I
I
I
I
I·
I
where S* and Sn(l) are defined by (2.6) and (3.5).
(3.23)
n-
LEMMA 3.4.
!f E(exp{ug (Xl' ... ,Xm) })
O<3a<4b-l (>O<a<1/3), then as
<
00
for
Iu I
<
= o(e- n
and, if E(exp{ug (X. , ... ,X. )}) <
-
1
1
1
00
a
),
for every l<il< ... <i <m, lui < E, then, as n~,
-
m
-
- m-
p{ISk-ms~l)1 > ~kb for some k>n} = o(e-
(3.25)
Proof.
(>0) and ~<b<~,
n~,
(3.24)
-
E
na
).
By (3.22) and the Markov inequality, for large n,
P{nIU*\ > ~b} < (~b)
(3.26)
n
< (~b)
so that on choosing k
n
-k
-
-k
k
n ElnU*1 n
n
k
k
k
n(;) n[C (F)] n{(2k )!/2 n kn!}{l+o(l)},
n
2
a
as the largest even integer contained in n , we obtain that
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
10
the right hand side of (3.26) is asymptotically (as
n~)
equal to
(3.27)
= [2m(m-l)C 2 (F)/n b-a ] n
a
e -n
a
{1+0(1).
}
Now, 3a<4b-l::>4(b-a) > I-a> 2/3::>b-a > 1/6.
Therefore the first factor on the
a
n . Consequently,
right hand side of (3.27) is bounded above by [2m(m-l)C (F)n -1/6 ]
2
by (3.26), (3.27) and the Bonferroni inequality, as n~,
p{k\ukl > ~kb for some k>n}
::. Lk:n P{k \ Uk \ > ~kb}
(3.28)
00
::. Lk=n[2m(m-l)C 2 (F)k
< e=
na
-1/6 k a _k a
]
e
[1+0(1)]
a
[1+0(1)] Lk:n[2m(m-l)C2(F)k-l/6]k
a
o(e- n ),
a
as \ 00 [2m(m-l)C (F)k- l / 6 ]k ~ 0 as n~.
L..k=n
2
using Lemma 3.3.
!f
THEOREM 3.5.
<
00
for
lu \ <
such that if
(3.29)
The proof of (3.25) follows similarly by
Q.E.D.
8(F) is stationary of order 0 and E(exp{ug(Xl, ... ,Xm)})
E(>O), then there is a standard Brownian motion ~ =: {~(t): O<t<oo}
~<b<~
and 0<3a<4b-l, then as
p{ ISet) -
y~(t)
I
> t
b
s~,
for some t>s} = o(e
and, if further, E(exp{ug(x.1 , ... ,x.1 )}) <
1
1::'i l ::'... ::'im::.m, then as
(3.30)
00
-s
a
),
for lul<E(~O), uniformly in
m
s~,
p{ls*(t) - y~(t)1 > t
b
for some t>s}
= o(e- s
a
),
I
I.
I
I
I
I
I
I
I
11
where Set) and S*(t) are defined as in Section 2.
Proof.
We only prove (3.29) as (3.30) follows on parallel lines.
(3.3) and (3.5), Sk
=
By virtue of
m~l) + kU k , so that the event [IS(t) - yl;;(t) I > t b for some
t > s] is contained in the union of the two events [\ms(l)(t) - y~(t)1 > ~tb for
some t>s] and [lkUkl > ~kb for some k>s].
Thus, the lefthand side of (3.29) is
bounded above by
P{lmS(l)(t) - ys(t)\ > ~tb for some t>s} +
(3.31)
p{ IkUkl > ~kb for some k>sL
Since S(l)(t) involves the iidrv {gl (Xi) - 8(F), i~l}, by Theorem 4.8 of Strassen
a
(1967), it can be shown that the first term in (3.31) is o(e- s ) as s~, while by
a
Lemma 3.4, it follows that the second term is also o(e -s ) as
s~.
Hence the
I_
theorem follows.
I
I
I
I
I
I
I
I·
I
along the same line as in Corollary 4.9 of Strassen (1967) where in his (204) and
Returning now to the proof of Theorem 2.2, we observe that the proof follows
(206), we need to use our Theorem 3.5, instead of his Theorem 4.8.
For brevity,
the details are therefore omitted.
~.
For {U } and {8(F )}, the law of iterated logarithm has
n
n
been studied in Sproule (1969) and Ghosh and Sen (1970), respectively.
The same
result follows directly from theorem 2.2 by letting
(4.1)
~(n)
=
1
[2n(1+E) log log n]~, E>O,
and noting that the right hand side of (2.16) or (2.17) is then asymptotically
r-:o-=
E -1
equal to [v4TI
E(log n)]
, and converges to 0 as
n~
(for every E>O).
Rubin and Sethuraman (1965) have shown that as n+oo,
(4.2)
I
12
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
On substituting ¢(n) = c[n log n]
~
2,
c>O, we obtain from Theorem 2.2 that as n+oo,
2
~ f 00
- (c/ 2v2n
) 1og n u- 1/2 e -1/2c u du
(4.3)
2
=
~ -l{ -1/2c
[cv2n]
n
(log n) -1/2 [l+O((log n) -1 )] } .
Thus, not only (4.3) specifies a better order in asymptotic expression, but also
strengthens (4.2) to the entire tail of {Uk' k>n}.
The same result holds for
{8(F ); k~n}.
k
Theorem 2.1 is of great help in the developing area of sequential procedures
based on U-statistics and {8(F )}, where the derived Wiener process approximation
n
simplifies the ASN and the OC functions in certain asymptotic sense.
be considered in a separate paper.
These will
I
II
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
II
13
REFERENCES
[1]
BERK, R.H. (1966). Liming behavior of posterior distributions when the
model is incorrect. Ann. Math. Statist. ~, 51-58.
[2]
CHOW, Y.S. (1960). A martingale inequality and the law of large numbers.
Proc. Amer. Math. Soc. 11, 107-111.
[3]
GHOSH, M., and SEN, P.K. (1970). On the almost sure convergence of von
Mises' differentiable statistical functions. Calcutta Statist. Assoc.
Bull. ~, 41-44.
[4]
HOEFFDING, W. (1948). A class of statistics with asymptotically normal
distribution. Ann. Math. Statist. ~, 293-325.
[5]
LOYNES, R.M. (1970). An invariance principle for reverse martingales.
Proc. Amer. Math. Soc. ~, 56-64.
[6]
MILLER, R.G. JR., and SEN, P.K. (1972). Weak convergence of U-statistics
and von Mises' differentiable statistical functions. Ann. Math. Statist.
43, 27-41.
[7]
MISES, R. von (1947). On the asymptotic distribution of differentiable
statistical functions. Ann. Math. Statist. !!, 309-348.
[8]
RUBIN, H., and SETHURAMAN, J. (1965).
Sankhya, Ser A. ~, 325-346.
[9]
SPROULE, R.N. (1969). A sequential fixed-width confidence interval for the
mean of aU-statistic. Doctoral Dissertation, Univ. N. Carolina,
Chapel Hill.
[10]
Probabilities of moderate deviations.
STRASSEN, V. (1967). Almost sure behavior of sums of independent random
variables and martingales. Proc. 5th Berkeley Symp. Math. Statist.
Prob. ~, 315-343.