USE OF MOMENTS IN DERIVING DISTRIBUTIONS. AND SOME CHARACfERIZATIONS
Norman L. Johnson
University of North Carolina
Chape 1 Hi 11. NC 27599
Samuel Kotz
University of Maryland
College Park. MD 20742
USA
USA
ABSTRACf
The utili ty of moment calculations as a simple way of establishing
distributional forms is illustrated with several examples. based on some
recent problems.
obtained.
KEY WORDS:
Characterization theorems
for beta distributions are
•
beta distribution; limit distributions; recurrence relations
- 1 -
1.
Introduction
Recently (Johnson and Kotz (1989» we have shown how moment methods
can
provide a way of obtaining distributions and characterizations. wi thout
recourse to characteristic functions. or solution of integral equations.
In
all cases. more sophisticated methods had been used to obtain the relevant
results.
In this note we briefly recapitulate the methods we have used. and
present further applications discussed by Assche (1986). Kennedy (1988) and
Chen
et
al.
In
(1984).
the
first
of
these
three
applications
a
characterization of the beta distribution is obtained.
2.
Methodology
(I)
It is required to find the distribution of
co
Y
where
=
the X· s are mutually
:I
j=O
j
(-l)j
rr x.
i=O
(1)
1
independent and have a
common beta (a. b)
distribution (density function
fX(x)
where B(a.b)
= {B(a.b)} -1
x
a-I
= f(a)f(b)lT(a+b)
(I-x)
b-1
(0
< x < 1;
a. b
> 0).
is a complete beta function).
The solution presented here is alluded to. briefly. in ..Solution.....
(1988) (page 564). but no details are given there.
the series in (1) converges almost surely. and 0
Now
~
Since
Y
~
1.
- 2 -
Y = Xo - XOX1 + XOX1X2 = Xo - XO{X1 - X1X2 +
.
}
= XO{l-Yif}
if
{2}
if
where X and Yare mutually independent and y has the same distribution as
o
y.
Using
~'{.}
s
to denote s-th moment. it follows from {2} that
s
E[Y ] = ~'{Y}
= ~'{XO}
~'{
s
s
s 1-Y} .
{3}
Since Xo has a beta {a.b} distribution
'{X} _ f(a+s)f(a+b)
0 - f{a}f{a+b+s}
{4}
~s
It
is natural to inquire whether Y has a beta {c.d} distribution. for
appropriate values of c and d.
If Y were to have a beta {c.d} distribution.
{l-Y} would have a beta {d.c} distribution. and {3} would require
f(a+s)f(c+d) _ f(a+s)f(a+b) • f(d+s)f(c+d)
f{a}f{a+d+s} - f{a}f{a+b+s}
f{d}f{c+d+s}
that is.
f(c+s)f(d) _ f(a+s)f(a+b)
f{c}f{d+s} - f{a)f{a+b+s)
This is satisfied by c=a. d=a+b.
(i)
for all s.
{5}
Furthermore. since
using (3) with s=1.2 •... sequentially. the
{~'{Y)}
s
can be determined
from the {~~(Xa)is. and
(ii) the moments of Y determine its distribution. because its support is
finite [0.1].
it follows that the distribution of Y must be beta (a.a+b).
The moments of the cODlllOn distribution of the X's can be determined
from those of Y. using (3) with s=1,2..... so the converse result holds.
that if Y has a beta (a.a+b) distribution. the cODlllOn distribution of the
X's must be beta (a.b).
These resul ts can be summarized in
•
- 3 -
Characterization Theorem 1
If XO.X •... are i.i.d. variables. then the distribution of
l
=
y
j
CXl
~
(-l)j
j=O
IT
X.
i=O
1
is beta (a.a+b) if and only if the common distribution of the X's is beta
(a. b).
(II)
Assche (1986) has considered the distribution of a random variable Z*
'uniformly distributed between two independent variables Xl and
~'
defined
by
o
z > max(xl'~)
if z < min(xl'~)
z-xl
if
1
Pr[Z* ~ zlx l
= Xl'
~
= x2 ] =
x2 -x l
For the special case when Xl and
~
Xl ~ z ~ ~
have identical distributions.
Johnson and Kotz (1989) have shown that the distribution of Z* is the same
as that of
Z
= WXl
+ (l-W)~
(7)
where W has a uniform [0.1] distribution and is independent of Xl and
Hence with ~~(X)
= ~~(Xl) = ~~(~).
~s'(Z*) = ~'(Z)
s
s
=
j:O
.
(~) ~j(X)~~_j(X) B(j+l. s-j+l)
~.
- 4 s
= (s+1}-1
~ ~j'(X)~'_j(X}
(s=1,2, ... )
s
j::O
Using (8) for s=1 ,2, ... , the moments {~. (Z*)}
can
s
the
moments
{~'(X}}
s
and
conversely.
If
the
(8)
be determined from
moments
determine
the
distributions, then the common distribution of the X's is characterized by
that of Z* , and conversely.
In particular, noting from (7), that if the
common distribution of the X's has finite support so does that of Z* , we
have
Characterization Theorem 2
If Xl and
~
are 1. i.d. variables with fini te support,
then their
common distribution is characterized by that of Z* , as defined in (6), and
conversely.
3.
(See Johnson and Kotz (1989) for some special cases.)
New Applications
(I) In Assche (1987), the following problem is posed.
sequence of stochastic
Consider the
~trices
}
nJ
1-Y
l-X
(n = 1,2,. .. )
n
formed by the recurrence formula
J
l-an
1-{3
n
(Xn - 1
(9)
Yn-1
with Xl = aI' Yl = {3l·
The a's and {3's are two sets of i.i.d. variables with support [0,1].
Assche shows (see below) that Y
=~
lim Y
n
What is the distribution of Y (or X)?
From (9)
and X
= n-tllO
lim X
n
exist and are equal.
- 5 -
xn
and
= an Xn- 1 + (l-a)Y
n n- 1
(10.1)
Yn = Pn Xn- 1 + (l-P)Y
n n- l'
(10.2)
xn-Yn
(11.1)
so that
n -Pn )(Xn-1 -Yn-1 )
= (a
Xn+Yn = (an + Pn )(Xn- 1-Yn- 1) + 2Yn- 1·
and
(11.2)
From (11.1), remembering that Xl = aI' Y1 = PI'
X - Y
n
n
=
n
IT (a j - P .)
J
j=l
(12)
and from (11.1) and (11.2)
(13)
Provided
max(Pr[a i
< 1], Pr[P i < 1]) > 0
for all
1,
then (using (12»
co
j-1
Y = lim Y = ~ P j IT
~ n
j=l
i=l
o
(with IT interpreted as 1)
i=l
almost surely exists and is equal to X = lim X (cf Assche (1987».
~i
n-P:»
n
Now
M
= PI + ~l Y
=a 1 Y*
"e
where aI'
PI
*
and Yare
distribution as Y"
Hence
+ P1(1-YM)
mutually
independent,
(14)
and
y*
has
the
same
- 6 -
(s=1,2 •... ).
~'(a).
where
s
~'(P)
s
(15)
are the s-th moments about zero of the distributions of
the a's and P·s. respectively.
Equation
(15)
~~-2(Y)' ... '~i (Y).
expresses
~'(Y)
s
as
a
linear
function
From this equation with s=1.2,...
of
~'
s-
l(Y)'
the values of all
moments of Y can be expressed in terms of the moments of the distributions
of a and
p. For example. with s=1. (15) gives
whence
~i (Y) =
~i(a) + ~i(P)
1-~i (a) + ~i (P)
This value is then used in (15) with s=2. from which a formula for
~2(Y)
is
obtained. and so on.
Assche (1981) considers the case when the a' s and p' s have a common
beta (a.a) distribution. so that
a _'
_ f(a+s) f(2a)
~s( ) - ~s(P) - f(a) f(2a+s)
I
Then (15) becomes
~'(Y) = f(2a)
s
{ f(a) }
2 s
f(a+j) f(a+s-j)
(s) - - - - - - E [ Y j (1-Y)s-j].
j=O j f(2a+j) f(2a+s-j)
I
(16)
We now show that (16) is satisfied if Y has a beta (2a.2a) distribution.
The left-hand side of (16) would then be
f(2a+s) f(4a)
(17)
f(2a) f(4a+s)
- 7 -
Since we would have
E[y j (1-y)s-j] =
f(2a+j) f(2a+s-j)
f(4a+s)
f(4a)
{f(2a)}2 '
the right hand side of (16) would be equal to
f(4a)
1
{f(a)}2 f(4a+s)
s
!
(~) f(a+j) f(a+s-j).
(18)
j::O
Now,
s
!
j::O
s
(~) f(a+j) f(a+s-j) = !
J
j::O
(~) f(2a+s) B(a+j, a+s-j)
J
2
•
= f(2a+s) B(a,a) = f(2a+s) {f(a)} /f(2a). (19)
From (18) and (19) we see that the right-hand side of (16) agrees with the
value (17) for the left-hand side.
(2a.2a) distribution.
Thus (16) is satisfied if Y has a beta
The distribution of Y must be beta (2a.2a) because
the equations (16) have only one solution.
It may appear that this derivation requires prior nknowledge of the
answer. n
However, one could be naturally led to examine the possibility of
a beta (2a.2a) distribution for Y by determining the first three or four
moments from (16) with s=I.2.3.(4).
(Formula (16) takes quite a simple form
when J.L'(a)
= J.L'(f3)
for all a and (j.)
s
s
We note that if J.L'(a)
= J.L'(f3)
= J.L'.
s
s
s say. for all s. then the values of
{J.L'}
can be determined (using (15» from the values of {J.L'(Y)}.
if these are
s
s
- 8 -
mown.
Our results can be summarized in:
Characterization Theorem 3
If {a .P } (n=1.2 •... ) are i.i.d. random variables
n n
l-Xn ] _
l-Y
n
[an
Pn
l-an ] [Xn- l
l-f3
n
Yn - l
l-Xn _ l ]
(n=2.3 •... )
l-Yn - l
with Xl = a l • Yl = Pl' then the common limiting distribution. as n
~~.
of
X and Y is beta (2a.2a) if and only if the common distribution of the a's
n
n
and p's is beta (a.a).
(II)
Kennedy (1988) has attacked the following problem. in connection with
a stochastic search model for global optimization:
"Let [An .Bn ] be random subintervals of [0.1]. defined recursively as
follows.
Let Al = O. Bl = 1 and take Cn' Dn to be the minimum and maximum
of k independent random points. each uniformly distributed on [A • B].
n
n
Choose [An+ l • Bn+ l ] to be [Cn' Bn ]. [An.Dn ] or [Cn.Dn ] with probabilities
p.q. r respectiv.ely.
(p+q+r=l)."
Figure 1 may help to clarify the procedure.
Since
we have
- 9 -
and lim E[Bn -An ]
n-lDD
Thus as n
~
= o.
00
the interval [A .B ] converges to a point. Z. say.
n
n
What
is the distribution of Z?
Let U
l
k from a
~
U2
~
...
Uk be order statistics from a random sample of size
~
standard [0.1] uniform distribution.
end-points
of
distribution
as
the
the
intervals
[A.B]
n
right-hand
n
end
{which
points}.
Consider the left-hand
have
the
Then
same
Z has
limiting
the
same
distribution as a mixture of
U + Z*{l-U } with proportion p
l
l
Z* U
k
and
•
where
*
~
..
..
q
..
..
r
has the same distribution as Z. and Z* and {Ul ..... Uk } are mutually
independent .
Hence
Now. from the distribution of the order statistics {U l ·· .• Uk } {see.
e.g .• David (l98l). Chapter 3}
- 10 E[~(U -U )h] = k(k-l) g!(k+h-2)!
1 k 1
(k+g+h)!
Inserting these values in (20), and collecting terms in J.L's (Z) on the
left-hand side, we obtain
s-1
[1 - ~ - ~ rk(k-l)
]J.L'(Z) _ ! (s) (s-j)!(k+j-2)!k
k+s
k+s
(k+s) (k+s-l) s
- j=O j
(k+s)!
x {(k+j-l)p+(k-l)r}J.L:(Z),
J
I.e.
J.L~(Z)
=
(
1)'k
s;1 (k+j-2)!
j!
{(p+r) (k-l)+pj}J.Lj (Z)
(k+s-2)~{k(l+r)+s-l} j=O
(21)
Equation (21) can be written as
= As
J.L'(Z)
s
(s-I)!k
with As -- (k+s-2)!{k(l+r)+s-l}
s-1
!
j=O
Tj
(21)'
T - (k+j-2)!{(p+r)(k-l)+pj} '(Z)
j j!
J.L j
.
•
Since, also,
s-2
J.L's- I(Z) = A
! Tj
s- Ij=O
we have
A
J.L'(Z)
s
= (----A
s )J.L's- I{Z)
s-1
A
= [___..s_
+
A
s-1
+
As Ts- 1
A (k+ j -2)!{(p+r)(k-l)+pj}], (Z)
s
j!
J.L s - 1
Inserting the values of As and As- I' the term in square brackets becomes
1
= {k+s-2){k(l+r)+s-l}
{{k+s-2)(s-I)+kr{s-I)+kp{s-I)+k{p+r){k-l)}
~ •
- 11 -
_ k(p+r)+s-l
- k(l+r)+s-l
So
'(Z) - k(p+r)+s-l • (Z)
- k(l+r)+s-l ~s-l
.
(22)
~s
Since
~O(Z)
= 1,
~'
s
(Z) =
~ k(p+r)+j-1
{k{1+r)+j-1}
j=l
which is the s-th moment about zero of a beta (k(p+r), k(l+r»
distribution.
This is, therefore, the distribution of Z, since the support of Z is finite.
We note that if k=l and
p::q~
(r::O), then we have a procedure in which
a random point Wn is uniformly distributed over (An,Bn ) and (An+1 ,Bn+1 ) is
The I imi t ing
chosen by taking (A ,W) and (W ,B) as equally likely.
n n
n n
distribution is then beta
(~,~)
- the 'arc-sine' distribution (see Chen et
al. (1981).
(III)
Chen et ale (1984) have considered a more general problem in which
the longer of the intervals (An,Wn ) and (Wn,Bn ) is chosen to be (An+1 ,Bn+1 )
with probability w and the shorter with probability (l-w).
It is easy to
show that the intervals (A,B) converge to a point Y •
n n w
What is the
distribution of Y ?
w
Clearly
Y~
has the same distribution as Z in the last paragraph of (II)
Here we outline a method of determining the moments of Y
w
of this section.
for general w.
If
After '1 has been obtained we have the following situations:
'1 ~ ~:
"
'e
If
'1 ~ ~:
_ {(WI,I)
(~,B2) =
(O"l)
_ {(WI,I)
(~,B2) =
(O,W )
1
with probability w
with probability (1-w);
with probability (1-w)
with probability
W.
- 12 -
Noting that the condi tional distribution of W1 , given W
1
~,
distribution of
~
~
is the
where U is a uniform [0,1] random variable; and the
conditional distribution of W , given W
1
1
is that of
~ ~,
~(l+U),
we see that
the distribution of Yw is that of a mixture of
with proportion
~ Y* U
w
~(l+U)
+
~
* (l-U)
Yw
~ Y*(l+U}
and
W
~;
with proportion
~(1-w);
with proportion
~(l-w);
with proportion
~;
*
where U and Yare
mutually independent and Y* has the same distribution as
U.I
W
Y.
U.I
Hence ~'(Y )
S
W
=~
w{E[{~ U+Y (1~ U}}s] + E{~ Y (l+U}}s}
W
W
(23)
Calculation of the first and last terms is facilitated by noting that
Y has a distribution on [0.1] which is symmetrical about
w
Y have the same distribution.
w
so (1-Y ) and
w
Thus
=
Similarly
~.
s
~
j=O
s
(j}(-l)
j 2j
j
+1-1
2 (j+1)
•
~j(Y ).
w
0,'
- 13 -
Inserting in (23) we obtain"
S
~'(Y ) = ~ (~)(-I)j j+~
s
j=O
W
+
2
(j+l)
j 1
{w(2 + -1) +
(l-w)}~j(Yw)
1
{w(2s +1_1) + (1-(,J»~' (Y ).
2s+1(s+l)
s W
(24)
Note that for s odd. the right hand side of (24) does not contain a
term in
~I
s
(Y ).
W
Taking s=1 we have
..
(as expected. because Y has a distribution synunetrical about
W
~).
Taking
s=2.
whence
I
.
3 (3-2c..l)
~2(Yw) = 2(11-6w)
The third moment could be derived from s=3. or using synnetry. since
the central moment
whence
~(Yw)
is zero. wee have
- 14 -
4.
Concluding Remarks
The
foregoing methods
determined by their moments.
depend on
the
relevant
distributions being
In all the cases we considered.
ensured. because the distributions had finite support.
this was
The condition may
also be true even when the support is unbounded. so that the method can be
applied in a wider field. subject to checking uniqueness.
5.
Acknowledgement
We are grateful to W. van Assche for drawing our attention to Assche
(1986) and Kennedy (1988). and for his illuminating comments thereon.
"
0
,._." ........A,.'
•
,
C"..
l'.....
,
Mj.... !
(A,..•• t 8
ft " , )
"
{~'
~
)'
1
I .....................................
.
~
...
:s.. .
,.
P
~
- 15 -
References
ASSCHE. W. van (l986)
Products of 2x2 stochastic matrices with random
entries. .1. Awl. Prob .• ~. 1019-1029.
ASSCHE. W. van (1987)
A random variable uniformly distributed between two
independent random variables. Sankhya. A29. 207-211.
""""'"
ClIEN. R.. GOODMAN. R. and ZAME. A. (1984)
Limiting distribution of two
random sequences • .1. Mvte. Anal.. 14. 221-230.
""'"
ClIEN. R.• LIN. E. and ZAME. A. (1981)
Another arc sine law. Sankhya. A43.
""""'"
371-373.
DAVID. H.A. (1981)
Order Statistics. 2nd edn. Wiley. New York.
JOHNSON. N.L. and KOO'Z. S. (1989) Randomly weighted averages:
and extensions. Submitted for publication.
Some aspects
KENNEDY. D.P. (1988)
A note on stochastic search methods
optimization. Adv. Awl. Prob .• 20. 470-478.
for
global
""'"
SOLurION TO PROBLEM 6524 (1988)
Amer. Math. Mthly .•
~.
562-564.
.'