•
'*
This rBsearach was partially supported by the National. Science Foundation
under Grant GU-2059.
IDENTIFYING PROBABILITY LIlfiTS '*
by
GORDON SIHONS
Department of Statistics
llniversi ty of North Carao Una at Chape l. Hi z.z.
Institute of Statistics Mimeo Series No. 681
April 1970
IIENTIFYING PRoBABILITY LIMITS
by
Gordon Simons 1
DepatMtment of StatiBtias
University of North Carolina
Chapel. Hil.l.# Nozoth CaroZina 27514
1.
INTRODUCTION AND SIJot\C\RY. The literature of mathematical statistics
is filled with theorems on (weakly) consistent estimators.
2
Even though
most statisticians want stronger evidence of an estimator's worth, these
theorems have provided some comfort for the applied statistician.
In this
paper. we begin an investigation into the concept of consistency and, more
specifically, investigate cne ex~n~ co wo1cna cons1s~ent sequence of
estimators identifies the parameter they estimate.
It will be recalled that for any sequence of random variables which
converge in probability to a limit, there is a subsequence which converges
almost surely to that limit.
This would seem to suggest that if one is
1
This researafz was partiaZZy supported by the NationaZ Science Foundation
undsp G:I'ant GU-2069.
2
We contrast weak consistency (or simply consistency) with stFong con-
sistency. The former requires a sequence of estimators to converge in
probability; the latter requ:Lres them to converge almost surely. This is
common terminology.
2
given a consistent sequence of estimators
(8€0) ,
say, then
ODe
A
A
4>1' 4>2'...
He)
converging to
can find a subsequence which converges almost surely.
That is, whenever there exists a weakly consistent sequence of estimators
there exists a strongly consistent sequence as well.
specific subsequence
may
Unfortunately, the
a.
depend upon the unknown parameter value
Still,
an applied statistician might be able to choose sequentially which observed
estimators to include in a (random) subsequence.
This is easily seen to be
equivalent to postulating the existence of functions
such that
A
A
8n(~l' ••• '~n)
converges to
~(e)
8n (Xl'" .,Xn )
with probability one
(n~l)
(9€0).
In Section 2, we will show that such functions do not always exis t.
It seems appropriate, therefore, to question whether the values of
the entire sequence
of
$1.$2'...
(always) allows one to determine the value
He) with probability one. We prefer the following mathematical re-
formulation of this question:
Let
(n,A,p)
be a probability space and
E denote the set of infinite dimensional random vectors
X· (Xl'X , ... )
2
defined on this space whose coordinates converge in probabi 'Li ty to a random variable (which we shall continue to denote as)
becomes:
Does there always exist a function
dimensional Euclidean space) into
f
p(X).
The question
which maps
GO
R
(infinite-
(the reals) such that for every
R
XfE,
the set
(1)
[ f (X)
~ P (X) ]
is contained in a null set of
We shall refer to any fwetion
f
which satisfies (1) for all
as a probabiZity l.imit identifiaation lunati-on (PLIF) on
justify the reformulation as follows:
vector of estimators
A A
(4)1.'2
•••• )
same probability space for every
A~
xe:Fc:E
F. We prutial.l.y
In Section 3, we will show that the
can be interpreted as defined on the
8e:0.
As such, consistent estimators are
3
equivalent to a family of vectors
F
D
{X
e-
6Ee} c: f.
Identifying
F.
becomes equivalent to showing that there exists a PLIF on
In Section 4 _ we show that there exists a PLIF on
a PLIP on
f·,
or 1).
With values of
corresponding to
E 1f there exists
the set of Xef whose coordinates are Bernoulli variables
and whose probability limit
o
cj>(6)
p(X),
p(X)
e
is almost surely a constant (necessarily
corresponding to vectors
XEf·
and
cj>(e)
the reformulation becomes complete.
We do not know whether a PLIF always exists far
tain elementary probability spaces.
f·
except for cer-
It is hoped that the current paper will
stimulate further research into this question.
If they do nbt always exist,
this would cast further doubt on the importance of consistency.
Breiman, Le Cam and Schwartz [2] have discussed an interesting problem whose formulation is closely related to the current one.
•
that they have a family of probability measures
on the same measurable space
cj>(e)
(0 _A)
(with points
{P e(·)' 6Ee}
WEn).
is measurable with respect to a a-field defined on
They assume
eac:b defined
They assume that
e
and find nec-
essary and sufficient conditions for the existence of an A-measurable estimator
~(w)
for which
(and also for a closely related condition).
of a measurable PLIF on
E*
The question of the existence
translates into the question of the existence
of a certain"zero-one set" in their context.
Skibinski [3] has connected
their work on zero-one sets with some work of Babadur [1].
4
2. WEAK AND STRONG CCWSISTENCY.
In this section, we demonstrate an
estimation problem in which weakly consistent estimators exist but strongly
consistent estimators do not.
Let
Y1'Y2""
be independent Bernoulli
P1' P2....
and
{~, k~l}
sequence of increasing positive integers.
Set
11
variables with arbitrary means
0
" 0
be an arbitrary
and
(2)
Finally, let
C denote the set of random vectors
X" (Xl ,X2 ' ••• )
which
arise from (2) and the condition
(3)
X takes the roll of
e
(XEC)
and
p(X) .. Umrr+a> Pn
"~1 • "~2" • •
He) • We create a sequence of estimators
each parameter
the values of
X,
the values of
~ ,X
2
'. • •
"~1,412""
A
respectively.
sistent estimating sequence exists.
functions
(4)
8n (xl"
• • ,xn )
..
by requiring, for
to agree with (be defined by)
Now assume that a strongly con-
Then. there exists a sequence of
taking values
n.f..!lr'f» gn (Xl"" ,Xn )
the roll of
0
or
klimco Pk
I
such that
almost surely for all
X E C.
Consider the situation
Pk" 0
exists a positive integer
quence
xl"",xj
m
j
for all
k > j
such that for
of zeros and ones,
m
(4) requires that there
1.
;?:
~
m
j
and any partial se-
gj+m(xI •••• ,xj,O,O, ••• ,O) .. 0.
That
is, independent of past bistory, a sufficiently long string of zeros will
force tbe current
g
long strings of ones.
1:;.1(l-Pk) .....
function to be zero.
A similar thing is true for
Next consider a situation in which
r;.lPk == f»
and
It follows from the Borel-Cantelli leDl1lla that the sequence
5
Yl'Y2'."
has infinitely many zeros and infinitely many ones.
by allowing the sequence
~ ,X
the sequence
2
, •••
{!it}
Clearly,
to grow rapidly enough, we can insure that
will have sufficiently large strings of zeros and
large strings of ones to bring about the nonexistence of the first limit
in (4).
This is a contradiction.
C.
We do not know if a PLIF exists for
(Note that
Cc:c· .)
We suspect
C would be a good class to work with in further research.
that the class
There is a somewhat contrived subset
C'c:C
for which a PLIF exists but
for which no strongly consistent estimators exist (for the same reason
given above).
Let
C·
X~C
be the set of
has finitely many even (odd) integers when
for which the sequence
Umk+c»Pk
{~}
equals zero (one).
The details are left to the reader.
3.
INTERPRETING ESTIMATORS N3 SUBSETS OF E.
Let
Xi,Xi,...
sequence of random variables defined on some probability space
(n,A,p)
and suppose that the probability space
admits a uniformly distributed variable
U.
. also admits a sequence of random variab les
n~l,
the laws of
Xi, ... ,X~
and
~,
(n:A:p')
(referred to in Section 1)
One can show that
~ ,X
2
' •••
••• , X · agree.
n
CO,A,P)
such that for each
Briefly, one begins
by showing that there exists a (one-to-one) measurable mapping of
a sequence
U 'U ,""
I 2
be a
U into
of independent uniformly distributed random variables
(since there exists a one-to-one mapping between a uniform variable and a
sequence of
Xl
ClI
in6{x:
iid
Bernoulli variables with mean
P' {XiSx} ~ U }
l
and then defines
conditional distribution functions.
1/2).
X (n>l)
n
One defines
recursively using
6
Xi ,xi,...
Now let
represent a sequence of consistent estimators of
Whereas the statistician often finds it convenient to view this
$(S), S€0.
sequence as one sequence of random variables and prefers to think of the
distr:f:butions of finite sets of them as depending upon the parameter
a,
the probabilist is inclined to say that the sequence of estimators represents a different sequence of random variables -- one for each
he views these random variable sequences as
spaces which can depend on
Xi'... 'x~
the sequence
€
E.
e --
since
defined on probability
e. Taking this latter view, we can identify
fop each S€0 with a random vector Xs •
(~'X2"')
As such, a consistent sequence of estimators becomes identified with
4.
s
E as
a subset of
ranges over
0.
PRoaABILITY LIMIT UENTIFlCATION Rf4CTIONS.
The main result of
this section is the following
THEOREM. Thezoe e:dstB a PLIF on E" if" and only if" there mats a PLIF
PROOF: Suppose that
arbitrary.
Set
(5)
X(a)
for each real
1
..
n
For
A€A,
a.
I
is a PLIF on
f
and
I[X <a]
n
E*. Let
X(a)
•
X· (X p X ,,,,)
2
(a) (a)
)
(Xl 'X2 , ...
1
A
" 1
or
0
as
A occurs or fails to occur.
€
f
be
7
For fixed
a
e > 0,
and
set
(6)
Then
p{y ~O}
s
n
and hence
set
Yt:E-
with
p{IX -p(x)/>e}
+
n
p(y) .. 0
0
almost surely.
as
n + ~,
It follows that within the
[p(X) > 8+£],
(7)
x(a) .. Y and
Letting
f{X(a»
.. fey) .. p{Y) .. 0
except for a subset
of a null set.
along a (countable) sequence, we obtain (7) for the set
E:-tO
[p (X) > a].
Similarly, (7) holds with
[p(X) < a].
Therefore, for arbitrary
defined analogously to
x(a),
g(x)
0
replaced by
1
~
X" (xl,~, ••• )€R
for the set
and
x
(a)
the function
.. in6{a:
f(x(a». OJ,
with a ranging over some countably dense set of the reals, is a PLIF for
E.
The converse is immediate.
We conclude this section with some elementary observations about PLIF's.
Let
on
F
n
Fn
be a nondecreasing sequence of subsets of
(n~l)
•
Then
lir+.4WJ
(8)
is a PLIF on
f(x)
E and
..
fn(x)
if finite
00
{
(x t: R )
o
othelWise
f
n
be a PLIF
8
Let
CD
T be a measurable transfromation of
a 'Limit presemng tPanSformation if
call T
into
TXeE
and
CD
R • We shall
p(TX)'" p(X)
Common examples include the simple shift
XeE.
almost surely whenever
R
transformation Tx· (x2 ,x , ... ) and, more generally, the subsequence
3
transformations Tx· (Xn ' Xn ' ••• ) for some increasing subsequence of
1
2
00
positive integers {~} (x-(x ,x , ••• )eR). Suppose that F is closed
l 2
under T. That is TXe F for every Xe F. Let f (x) be a PLIF on F.
O
Then
f (x) • f (Tux)
n
by (8), is such that
is also a PLIF on
F
(n~l)
and the PLIF
f,
defined
f(x)· f(T¥) (xER).
(8) can be used to show that a PLIF exists for any countable set
One simply orders the elements in
composed of the first
formation
Tn
for every
n
for which
XEF
n
(n~l).
elements of
on a subset
smallest set
f
F*cE
feE
(There exists a subsequence transthat converge almost surely
F.)
U
is closed under' subsequencing if for
T,
TXEF
containing
F·
for every XEF.
Many times a
which is closed under subsequencing.
F
are the set of
converge almost surely, and the set of
XeE whose coordinates
XeE* whose coordinates are inde-
In the latter case, the function
a PLIF on
be the (finite) set
naturally extends to become a fLIF on the
Typical examples of such sets
pendent.
F
n
Thus, it is easy to define a PLIF on each
every subsequence transformation
f
F.
TnX has coordinates
We shall say that a set
PLIF
F and lets
FcE.
f(x)·
0:__
-l~n
~'~UPn+=u
Lj=lxj
is
F because of (3) and the strong law of large numbers.
One might conceivable be able to use (8) (to find upper bowds of
linearly ordered sets) in an effort to demonstrate the existence of a PLIF
for
E· (by \Sing Zorn's lemma [4,page 39)). One can show that any maximal
element in the class
V... {FcE·:
sequencing}
E*.
must be
(D
F has a PLIF and f is closed wder sub-
is partially ordered by set inclusion.)
9
The only difficulty the author has found in developing such an argument
has been an inability to find upper bounds for uncountab Ze linearly ordered
subsets of V.
It seems reasonable to ask whether one should require a PLIF to be a
CD
measurabl.s mapping of R
into R.
This does not seem necessary and
adding such a restriction might prevent their existence.
ReFERENCES
[1]
Babadur, R.R. (1954). Sufficiency and statistical decision functions.
Ann. Math. Statist., 25, 423-462.
[2]
Breiman, L., Le Cam, L. and Schwartz, L. (1964). Consistent extimates and zero-one sets. Ann.. Math. Statist. , 35, 157-161.
(3]
Skibinski, Morris. (1969). Some known results concerning zero-one
sets. Ann. Inst. Statist. Math., 21, 541-545.
[4]
TayloT, Agnus E. (1964).
Wiley, New York.
Introduction to FunctionaZ AnaZysis.