•
ON UNBIASED ESTIMATION FOR RANDOMIZED RESPONSE MODELS
By
Pranab Kumar Sen
Department of Biostatistics
University of North Carolina, Chapel Hill, N. C.
Institute of Statistics Mimeo Series No. 867
APRIL 1973
ON UNBIASED ESTIMATION FOR RANDOMIZED RESPONSE MODELS I
PRANAB KUMAR SEN
University of North Carolina, Chapel Hill
ABSTRACT
For quantitative randomized response models, optimal unbiased estimation
of regular functionals of distribution functions is considered.
In this con-
text, distribution theory of Hoeffding's U-statistics and von Mises' differentiable statistical functions is extended to randomized response models.
Estimation of the basic distributions is also considered.
1. INTRODUCTION
In practical surveys, particularly, involving sensitive questions, the
.e
randomized response technique of reducing respondent bias has been found to
be quite effective.
Since the randomization affords protection to the respondent
in answering the question without revealing his personal situation, potential
embarrassments and stigma have been removed, and therefore, the primary reason
for either a refusal or an evasive answer does not exist.
An extensive amount
of work in this area has been carried out by the North. Carolina group; we may
refer to Greenberg et al (1970, 1971) where other references are also cited.
In a quantitive randomized response model utilizing two questions, the
respondent selects at random one of the two questions in such a way that the
interviewer does not know which question is being answered.
However, the
probabilities of selecting the two questions are set beforehand.
Thus, the
response distribution is a mixture of two basic distributions where the mixing
1) Work sponsored by the Aerospace Research Laboratories, Air Force Systems
Command Contract F336lS-7l-C-1927.
Reproduction in whole or in part permitted
for any purpose of the U.S. Government.
-2coefficients are known.
Mostly, the current literature deals with the situ-
ation where the functional forms of the basic distributions are assumed to
be specified, and one is interested in the set of parameters (algebraic
constants) associated with these distributions.
For such problems, standard
statistical procedures for mixture of distributions are usually adapted
without much problem.
For a broad class of problems, one may have very
little knowledge on the forms of the underlying distributions, and it may
be more reasonable to assume that the basic distributions belong to some
broad class of distributions.
In this framework, estimable parameters are
defined as suitable functionals of the distribution functions.
Our first
objective is to sketch this formulation of estimable parameters for randomized
response models and to provide unbiased estimators of these parameters.
In
this context, the theory of unbiased estimation of regular functionals of
.e
distribution functions, studied in detail by Halmos (1946), Hoeffding (1948)
and others, is extended here to randomized response models.
These develop-
ments along with the distribution theory of the related von Mises' (1947)
differentiable statistical functions are treated in Section 3.
The theory
is illustrated with the aid of some examples.
The characterization of randomized response distributions as mixtures
of two basic distribution provides clue for the estimation of the latter in
terms of the former, in a reasonably simple manner [See Section 4].
estimates are consistent and unbiased.
These
On the other hand, unlike the usual
estimates of distribution functions, these are (i) not necessarily nondecreasing everywhere, and (ii) do not necessarily assume values in the closed
interval [0,1].
These drawbacks call for certain modifications which are
considered in Section 4.
Certain confidence bounds for the basic distributions
are also provided.
The case of more than two questions in the basic model is treated briefly
in the last section.
-3Z. PRELIMINARY NOTIONS
Consider a randomized response model where a respondent selects at
random either of the two questions A and B, where A is usually sensitive.
Suppose, we have two mutually independent and non-overlapping random
samples of sizes n
l
and n •
Z
In the ith sample, a respondent selects the
two questions A and B with respective probabilities p. and q. (=l-p.),
1.
for i=l,Z.
1.
1.
The response is assumed to be quantitative in nature, and the
distribution functions (df) of the response (assumed to be a stochastic
k-vector for some k>l) for the two questions A and B are denoted by F (x)
l
and FZ(x), respectively, which are both defined on the common Euclidean
k
space R •
The random variables associated with the n. observations in the
1.
ith sample are denoted by X.l, ••. ,X.
1.
1.n.
1.
, so that these are independent and
identically distributed (iid) with a df denoted by G.(x), for i=l,Z.
1.
.e
Then,
we have
(Z ·1)
so that Fl,F
Z
are not expressible in terms of Gl,G •
Z
Equation (Z.l) leads
us to
The last equation leads us to estimate Fl,F
Z
as well as their parameters
from X , ••. , X
• We shall deal with this problem in the next two sections.
ll
ZnZ
In passing, we may remark that whereas G and G are convex combination of
Z
l
F
l
and F ' the converse is not true [as the coefficients in (Z.Z) are not all
Z
non-negative].
Section 4.
This creates certain problems which will be discussed in
-43. UNBIASED ESTIMATION OF PARAMETERS OF F , F .
2
l
We assume that both F
l
and F
2
belong to a family of df's, ~ which
contains all convex combination of Fl' F2' that is, if F lEY,' F2E'X then
aF
l
+ (l-a)F2E~for every O<a<l.
(3·1)
For example, if ~ is the family of all continuous df's or distributions
with finite moments up to the pth order for some p>O, then (3·1) holds.
Consider a functional 8(F) of F defined on~, such that for a sample
Zl, .•. ,Zn of size n from the df F, there exists a statistic ¢(Zl, ... ,Zn)
for which
(3·2)
Following Hoeffding (1948), 8(F) is then said to be regular over~.
m(~l)
be the smallest n for which (3.2) holds.
Then, ¢(Zl' ..• Zm) is called
the kernel of 8(F) and m, the degree of 8(F) over ~
.e
Let
Without any loss of
generality, we may assume that ¢(Zl, ... ,Zm) is symmetric in its m arguments,
so that by (3·2),
8(F) = !·k··!¢(zl' ... 'z )dF(zl) ... dF(z ), V F E ~.
Em
m
m
(3.3)
In the randomized response model, {Z.,i>l} are not observable.
].
-
Based
on the observable random variables {X .. ,1<j<n.,i=1,Z}, one is interested in
1J
-- 1
estimating 8(F ) and 8(F ), defined by (3·3) with F=F and F .
Z
l
2
l
We shall
consider the case of 8(F ) only, as the other case follows on parallel lines.
1
THEOREM 3·1.
.!f (3.1) holds and 8(F) is regular over
~ with degree m (~l),
then for the randomized response model, 8(F ) is estimable for all
l
Proof.
nl>m,n2~.
We have to show that under (3·1) and (3·3), for every Fl,FZE~, there
exists a kernel ~(Xll, ... ,Xlm'XZl' ... 'XZm) which unbiasedly estimates 8(F l ).
Let us denote by
(3·4)
for 02s<m, where for s=O or m, one of the product terms equals to one.
Since,
-5(3. 5)
we conclude that 8 s (F ,F ) exists for all O<s<m and F ,F EJ{
l 2
l 2
Hence
8 s (G l ,G 2 ) exists for every 02..s<m, Fl'F2e:~
(3'6)
Then, by using (2'2) and (3'3), we have
8(F l ) =
= (
J·k·J¢(zl'···'z ) ~
Em
m j=l
m
)-ro E (m)( l)s m-s s
Pl-P2
s=o s q2 ql
{(Pl-P2)-1{q2 dG l(z.)-q dG 2 (z.)}
J
1
J
J·KID't'
J~(
zl""
E
~s
)
()
,zm'~hdGl Zj
J
m dG (z )
'IT
2 f(,
f(,=m-s+l
(3. 7)
From the well-known results on generalized U-statistics [viz., Puri and Sen
(1971, Section 3,2)], it follows that 8 (G ,G ) is estimable, and
s l 2
(3'8)
E¢(Xll""'Xlm_s' X2l ,···,X 2s ) = 8 s (G l ,G 2 ),
for every O<s<m and Gl,G2EJ:(i.e.,F1E:¥,F2E~). Thus,if we let
.e
~(Xll""'Xlm,X2l""'X2m) =
(Pl-P2)
-m m m
s s m-s
E (s)(-l) qlq2 ¢(Xll""'Xlm-s,X2l"",X2s)'
s=O
(3.9)
it follows from (3'7), (3,8) and (3·9) that
E l/!(Xll""'Xlm'X2l""'X2m) = 8(F l ) for every
Gl,G2E~.
(3'10)
Hence the proof of the theorem is complete.
The generalized U-statistic corresponding to 8 (G ,G ) is
s l 2
Us (n l , n 2 ) =
(n~ ) -1 (n 2)-1 E*¢ (x lj
ms
s
s
1
, ..• ,X 1 .
1
m_ s
,x 2'J 1 ,···,X 2'J s ),
where the summation E* extends over all possible l<il< ..• <i
s
-
<n
m-S- l
(3'11)
and
l2..jl< •.. <js2..n2' for s=O,l, ••• ,m; (for s=O or m, one of the two sets is null).
As an unbiased estimator of 8 (G ,G ), U (n ,n ) possesses certain optimal
s l 2
s l 2
properties.
In particular, Us (n l ,n 2 ) is symmetric in ~l""'~nl as well
as in X2l ' ... ' X
' and hence, a function of the two sample order statistics.
2n2
Thus, if these order statistics are complete, then U (n ,n 2 ) is the minimum
s l
-6variance unbiased (MVU) estimator of 8 (G ,G ), for each s(=O,l, .•• ,m).
s 1 2
Also, jointly, the vector [Uo(n1,n2)"",Um(n1,n2)] is the minimum concentration ellipsoid unbiased estimator of [80(G1,G2), ..• ,8m(G1,G2)]'
so that if we define
(3'12)
it follows from (3·9) through (3·12) that under the completeness of the
two sample order statistics [viz., Fraser (1953)], U(n ,n ) is the MVU
1 2
estimator of 8(F ).
1
Let us denote the two empirical df's by
-1 n i
n
k
(x) = n. L. 1u(x-X.j), x£R , for i=1,2,
G
1
i
J=
1
(3·13)
where u(t) is 1 iff all the k components of t are non-negative, and 0, otherwise.
Then, following von Mises (1947), we define a (generalized) differentiable
.e
m-s
m
Tf
dG n1 (z')n
+l dG n (zn)
J N=m-s
2 N
J=l
J'k"mJCP(zl""'z m) ."..,.
E
for s = O,l, .•. ,m.
then, an alternative estimator of 8(F ) is
1
(3'15)
Whereas U(n ,n ) is an unbiased estimator of 8(F ), V(n ,n 2 ) is not, in general,
1
1
1 2
a strictly unbiased estimator.
by using the G
n.
By virtue of (2'2), we may estimate the df F1
in (3'13) by
1
F1 (x;n1 ,n 2)
= {q2 Gn (x)-Q1 Gn (x)}/(P1-P2)'
2
1
k
which unbiasedly estimates F (x) for every x£E
1
and
(3'16)
n1,n2~1.
Then, V(n ,n ) can also be written as
1 2
J~k~JCP(Zl,···,Zm)dF1(zl;n1,n2)···dF1
(zm;n 1 ,n 2 )
(3'17)
-7The expression (3·17) corresponds to the form in von Mises (1947), where
the empirical df is based on the basic random variables.
Using the results
in Puri and Sen (1971, pp.64-66) for the individual U (n ,n ) - V (n ,n ),
s l 2
s l 2
02s<m, we obtain by (3'12) and (3'15), that if n /(n +n ) is bounded away
1 2
l
from zero and one, when
then
nl+n2=n~ 00,
n~ [U(n ,n )-V(n ,n )]
l 2
l 2
~
0, in probability
(3'18)
Thus, for large n ,n , the two estimators U(n ,n ) and V(n ,n ) share the
l 2
l 2
l 2
common properties.
As such, we shall not discuss the case of V(n ,n 2 ).
l
Let us now denote by
¢ (Xli" .•• ,Xli'
,X ' , ••• ,X ., )},
2J '
1
m-s' 2j 1
s
.e
when (il, •.. ,i
m-s
) and (iI' , •.. ,i' ,) have exactly c (>0) indices in common,
m-s
and (jl, ... ,js) and
O<d.::.(s,s') and
(ji,· .. j~,)
O~s,s'<m.
have d (>0) in common, for O.::.c'::'(m-s,m-s'),
Then, it is well-known [cf. Puri and Sen (1971,
Section 3·2)] that for n ,n >m,
l 2
Cov(Us(nl,n2),Us,(nl,n2)]
u
=
(m-s) (m-s')r (
"F F)
sst
,
)
( -2)
•
n
~10 s,s , l' 2 + ~Ol (s,s ;F l ,F 2 + 0 n
l
(3'20)
Using than (3'12) and (3'20), we have
-2m m
Var[U(n l ,n 2 )] = (Pl-P2)
sEo
2m-s-s'
,
q2
Cov[Us(nl,n2)'Us (n l ,n 2 )]
= (p _p )-2mm2 ~
~ (_l)s+s'q s+s'q2m-s-S'{(m;1)
1 2
s=O s'=O
1
2
(m;t)~10(sls';Fl,F2)
1
-81
2 ).
+ (m-l)
s-l (m-l
s'-l ) ~Ol
(s,s';F l ,F 2 )}+ O(n
(3' 21)
Moreover, from the well-known results on the asymptotic normality of generalized U-statistics [viz., Puri and Sen (1971, Section 3·2)], it follows that
k:
asymptotically n2[{Uo(nl,n2)-80(Gl,G2)}""'{Um(nl,n2)-8m(Gl,G2)}] has a
(m+l)-variate normal distribution with null mean vector and dispersion matrix
whose elements are n times those in (3·20).
Thus, if we set
0<>"<1,
we conclude that asymptotically (as
n~
(3'22)
00),
~
2
~(n [U(n ,n )-8(F )]) ~ ){(o,y ),
l
2
l
where by (3'21) and (3'22),
2 = 2(
m
)- 2m m
~
~
(_l)s +'
s s +'
s 2m-s-s
Y
m Pl-P2
s~o sr=O
ql
q2
.e
+ 1='
(3'23)
{I
_(m-1) (m-l)r (
"F F)
>.. s
s' '-:>10 s,s , l' 2
(~:t) ~=t)1;Ol(s,s' ;F1 ,F 2 )}
(3·24)
In the above development, it is assumed that
max Ecjl 2(Xl1""'XlS'X2l, ... ,X2m_s) < 00 and y2 > O' (3'25)
O<s<m
0~sl,s2<m,
We may remark that all the scd(sl,s2;F ,F 2 ),
l
of G and G2 (E~, and hence, are estimable.
l
are regular functionals
Sen (1960) has obtained some
simple estimates of these functionals for the conventional one and two sample
problems.
His estimators remain good for the randomized response model too.
So, substituting these estimates in (3'24), one gets an estimator of y2, which
"-
we denote by y2
n n
.
Thus, noting that y
l 2
~
n n
y, in probability as
nl,n2~
l 2
and (3'23) holds, one obtains by using the well-known Slutzky theorem that as
1
"-
J.(n~[U(nl,n2)-8(Fl)]/y
n
n
)
----?>
J{(o,l).
(3'26)
l 2
The last result is useful in providing a large sample test or confidence
bound for 8(F ).
l
00
-900
We consider now some illustrative examples.
Let ~= {F:fxZdF(x)<oo},
_00
so that Fl,FzE~implies that (3'1) holds.
We desire to estimate
ai = aZ(F ) = JooXZdFl(x)-(JooxdFl(x))Z,
l
_00
-00
when {Xlj,l<j~ni,i=l,Z} are observed.
(3'Z7)
If we let ¢(zl'zZ) = ~(zl-zZ)Z'
then, in (3'3), 8(F) = aZ(F), so that aZ(F ) is estimable and m=Z. Then,
l
Z
Z
Z
by (3'4), 8(G l ,G l ) = a (G ), 8(G ,G )
a (G ) and 8(G ,G ) = ~[a (G ) +
z z
l
z
l
l Z
z
Z
00
a (G z ) + {~(Gl)-~(GZ)} ], where ~(Gi) = J xdGi(x),i=l,Z.
_00
1
nl
-
Hence, by (3·11),
Z
1
-
nZ
we obtain that Uo(nl,n Z) = ----1 ~. l(X' .-X.) ,UZ(nl,n Z) = ----1 ~. l[XZ'-X ]
nl J=
1J 1
n ZJ=
J Z
Z
nln Z
-1 ni
1
and Ul(nl,n Z)
----~, l~" l[X' ,-xz,,] , where X.
n. ~j lX'j,i=l,Z.
nln Z J= J =
1J
J
1
1
= 1
Z
Consequently, by (3'lZ),
.e
-Z
Z
Z
{qZUo(nl,nZ)-ZqlqZul (nl,n Z) + qlUZ(nl,n Z)} (3'Z8)
Z
is the MVU estimator of a (F ). Secondly, consider the case of bivariate
l
U(nl,n Z) = (Pl-PZ)
df's, and define
8(F) =
J J
ro
00
F(x,y)dF(x,y), FE~ all continuous df's.
(3.Z9)
_00_00
The sample measures for (3'Z9), known as the Kendall tau, can be easily
computed by using the kernel
¢(~l'~Z)
(where ~ means aj~bj,j=l,Z).
= 1 or 0 according as
These are Uo(nl,n Z) -_ (nZlJ
l
~l~Z
or not
),
~l~i~j<nl¢ (
Xli'X
ij
1
nl nZ
nZ -1
= ----- ~ ,~ ¢(Xl',X Z .) and U(nl,n Z) = (Z) ~l~i<j~nz¢(Xzl,XZj)'
nln Z i=l J=l
1
J
So that
-Z Z
z}
U(nl,n Z) = (Pl-PZ) {qZUo(nl,nZ)-zqlqzUl (nl,n Z) + qlUZ(nl,n Z) (3'30)
is the MVU estimator of 8(F ).
l
In either of these examples, V(nl,n )' defined
Z
by (3-15), will be different from U(nl,n )' and is biased.
Z
4. ESTIMATION OF THE BASIC DISTRIBUTIONS
In many practical situations, a complete knowledge of the distributions
F
l
and F is deemed to perform a more detailed study of the data.
Z
For example,
in the North Carolina Abortion study [1], Greenberg et al have estimated the
-10average number of abortions per female in the child bearing age group,
stratified by various socio-economic factors.
Instead of estimating these
averages, one may be more interested in comparing the distributions of the
number of abortions per woman over the different strata.
This may reveal
the proportion of women having at least one abortion as well as similar
other characteristics of the distributions.
In order to increase the
scope of our inference procedures, in such a case, we may be interested
in providing distribution-free estimates of F
any stringent assumption on the form of F
and F
l
Z
which do not require
and F '
Z
1
It is known that the empirical df's G
n
and G
n
l
,defined by (3'13),
Z
are unbiased estimates of G and G ' and they are sufficient statistics.
l
Z
Thus, as in (3'16), we have the estimates of F
l
and F
q2 Gn (x)-qlGn (x)
1
2
n.
1
as
PIGn (x)-P2 Gn (x)
2
l
, (4'1)
.e
Since G
2
unbiased estimates G , i=1,2, by (4'1) and
i
(2' 2)
k
A
EF (x;n ,n ) = Fi(x), xsR , i=1,2.
i
l 2
Also, by the Glivenko-Cantelli theorem, as n.
1
s~p
I Gn . (x)-Gi(x) I ~
a
(4' 2)
~ 00,
almost surely, i=1,2.
(4' 3)
1
Thus, by (4'1), (4,3) and (2'2),
s~p
I ;i(x;nl ,n 2 )-F i
(X)
I~
Consequently, the derived empirical df's F
consistent estimates of F
l
l
a almost surely, i=1,2
and F
and F , respectively.
2
2
(4'4)
are unbiased and strongly
On the other hand, by (4·1),
This introduces the
are not convex combination of G and G
n
nl
2
following undesirable properties of PI and P 2 · First, PI and P 2 are -not non-
F
l
and F
2
decreasing everywhere.
Second, F
l
and F
2
can assume negative values, and
-11third, they can also assume values greater than one.
To illustrate these,
we consider the case of k=l (i.e., X
real valued) and denote the ordered
ij
random variables of the ith sample by X. l< ... <X,
for i=1,2. Also let
1, - 1,n
i
=+00, for i=1,2. Then, by (3·13),
X. O=-ooand X.
1,
1,n +l
i
G (x) = (j-l)/n. for xi . l<x<x . . ,1<j<n.+l,i=1,2.
ni
1 , J- 1, J - - 1
(4· 5)
Thus, by looking at (4·1) and (4·5), we obtain that
A
[X l ,1>X 2 ,1] ===:- Fl (x;n l ,n 2 ) <0, \:J X2 , l<x<X l , l'
(4· 6)
°, V Xl , 1<x<X 2 , l'
(4' 7)
[Xl,l <X 2 ,1] ~ F2 (x; n l ,u 2 ) <
A
so that F ,F can be negative.
l 2
[X 1,n <: X2,n ]
l
2
[X
.e
~
2,n 2
~
=-
X
]
l,n
l
Similarly,
F1 (x; n l ' n 2 ) > 1, \f Xl ,n .::.x<X 2 ,n
1
2
(4' 8)
F2 (x;n ,n ) >1, 'V X
<x<X
l 2
2,n 1.n
2
1
so that F ,F 2 can be greater than one .
l
Again, by (4·1) and (4'5),
Q2/ n l(Pl-P2)' i f x=X l ,i,1'::'i<n l ,
,
-Ql/n 2 (Pl-P2)' i f x=X 2 ,1., 1<i<n
-- 2
dF1 (x;nl' n 2 )
0,
otherwise;
(4'10)
,
Pl/n 2 (P l -P 2 ), i f x=X 2 , 1. ,1<i<n
-- 2
dF (x;n ,n ) =
2
1 2
,
-P2/ n 1 (P1- P 2)' i f x=X 1 ,1. ,1<i<n
-- 1
so that F
l
and F2 can not be non-decreasing everywhere.
undesirable properties, F
l
and F
(4 ,11)
otherwise;
O.
2
Because of these
can not be regarded as distribution functions,
and hence, we need to consider some alternative estimators.
Definition.
An estimator of F.(x) having the fundamental properties that (i)
1
it is non-decreasing in x everywhere, (ii) it lies in the closed interval
[0,1] and (iii) it tends to
° or + 1 according as
x~
-
00
or +
00,
is termed
-12a characteristic preserving estimate.
We shall consider here characteristic preserving estimate of F
1
and F •
2
First, for simplicity of presentation, we consider the case of real valued
random variables for which both G and G are continuous.
2
1
As before, the
order statistics of the ith sarop1e are denoted by X. (=-oo)<X. l<.'.<X'
<
1,0
1,
1,n
i
Xi ,n.+1 (=00), i=1,2, where by virtue of the continuity of G1 ,G 2 , ties can
1
be neglected, in probability.
Since, by (4·10)-(4·11), dF. is positive
1
only at the n. order statistics of the ith sample, for i=1,2, we propose
1
our estimates of F ,F , using these n +n points.
1 2
1 2
For this, let
(4 '12)
(4 ·13)
where G
n
.e
and G
n
1
are defined by (3·13).
As mentioned earlier, F . . is not
1,J
2
necessarilyl'inj (q:t:.ni), i=1,2.
Our proposed estimators F i (x;n ,n 2 ),i=1,2,
1
are then defined by
F . ( x.. ) f or X. . <x<xi '+1' 0<'<
'-1 2
1 -1..,]
1, J,J
- J - n.,
1 1- , ,
(4'14)
where
1;
Fi(~,j_1)
,
i f F.
. <F. . l'
_1,J- 1,J-
A
F. (X . . )
1 1,]
(4 '15)
A
F..
1,J
i f F . . l<F . . <1,
1
i f F . . >1,
1,]-
1,J
(4'16)
A
1,]-
for 1<j<n.,i=1,2.
--1
Now, by (4'14)-(4'16), F.(x) is non-decreasing in x, lies in the closed
1
interval [0,1], and attains the lower and upper bounds for x<X.1,1 and x>X.
- 1,n ,
i
respectively, for i=1,2.
estimates of F
1
and F .
2
Thus, F
1
and F
2
are characteristic preserving
Note that (4'16) can be written as
-13F.(X . . ) = min{ max F. k,l},0<j<n.,i=1,2,
1
1,)
O.:::k<j 1,
- 1
(4'17)
Further, note that for Xl,j,:::x<Xl,j+l' G (x) remains stationary, whereas
nl
A
G (x) may increase,
n
Hence, by (4,1), F (x;n ,n ) is non-increasing for
l
l 2
2
F2 (x;n l ,n 2 )
Xl, j':::x<X l , j+l' j=O"."n l ,
Similarly,
X2,j<x<X 2,j+l' j=O " " , n 2'
Consequently,
is non-increasing for
A
max
for 0<j<n.,i=1,2,
--1
F. k = sup{F.(x;n ,n )
l 2
O.:::k.:::j 1,
1
x<X. .},
-1,)
(4 '18)
Thus, by (4'17) and (4'18),
F.(X . . ) = min[sup{F .(x;n ,n ) : x<X . . lol),
1
l 2
1
1,J
- 1,J
(4'19)
and by (4,14), (4'19) and a few standard steps, we get that
(4'20)
[Note that F i (y;n l ,n 2 )=O,
.e
vy <min(X l ,1,X 2 ,1)'
so that Fi(x)~O,) The last
definition is quite flexible and it readily suits the cases where G and G
l
2
(or F l and F2 ) are multi-dimensional distributions or are not necessarily
continuous everywhere,
k
F (x),XER
2
Thus, for general
k(~l)
variate distributions F (x),
l
on denoting by a<b, the coordinatewise inequalities, our proposed
estimates are
A
min[sup{F (y;n ,n ) : y.:::x},1),i=1,2,
i
l 2
A
where the df's F
(4'21)
A
l
and F
2
are defined by (3'16) and (4,2),
Actually, these
A
are obtained by Smoothing F
l
and F ,
2
If we let n=nl+n2,An=nl/n, and assume
that there exists a A : O<A <~, such that for all n,
o
c:r(4'22)
then we have the following,
THEOREM 4'1,
1£ Fl
and F
2
are continuous everywhere and (4,22) holds, then
I}->
as
n
~ 00,
for i=1,2,
o
almost surely
(4'23)
,
FIGURE 4.1.
The' empirical df's
1.0
E>
F1
and F
,.
0
0
1
@
~
..
oo@:
~
•
o
0
0.8
•.
"....,
x
•.
0.6
'-'
.....
II«
e
.
...
tl° •
0
"....,
>:
....,
.....
<I«
F
1
"
F
• 1
0
0.4
t
•
0=
F
1
F
1
0.2
o
2
4
6
10
8
+x
e.
.'
'O.
12
14
-14The proof of the theorem is sketched in the Mathematical appendix.
To illustrate the relative behavior of F and F, we obtain by the use of
random numbers and tables for standard normal random deviates the following
two samples, each of size 20, when F l and F are normal df's with means
2
5 and S and standard deviations 2 and 3, respectively, and Pl=q2=0.S.
The ordered variables for the two samples are, respectively, O·lS, 1·32,
1·46, 2·06, 2·40, 3·50, 3·62, 3·66, 4·52, 4·S0, 4·92, 4·94, 6·1S, 7·00,
7·1S, 7·52, 7·S5, 9·S9, 10·46, 11·45 and 2·9S, 3·9S, 4·11, 4·55, 5·00, 5·92,
6·5S, 7·37, 8·45, 8·93, 9·20, 9·41, 10·28, 10·46, 10·58, 10·71, 10·91, 11·15,
13·43,
l4·l2~
On using (4·1) and computing F and F , we immediately observe
l
2
that F (x)<0 for x<3·98 and F (x»l for x>10·46.
2
l
we plot Fl(x) and F (x), see figure 4·1.
l
Only the points of discontinuity
of Fl(x) or F (x) are spotted on the graph.
l
and negative jumps, F
l
is non-decreasing.
On the same graph paper,
Whereas F
l
can have both positive
The maximum displacement between
F and F , in this case is 0·067, and it occurs at x=9·4l.
l
l
hold for F
2
Similar conclusions
and F .
2
(Figure 4·1 goes here)
For discrete F and F , we have to impose the following condition that
l
2
the df's F
l
and F
2
both have the common jump points; otherwise, the identity
of the question A or B may be revealed by a look at the response.
If x be
a jump point of F and F (and hence, of G and G ), on noting that
2
2
l
l
it can be shown that as
nl,n2~oo,
(4·25)
so that Theorem 4·1 readily extends to this situation.
The same result holds
when the X.. are recorded on suitable interval scale, where the df's can only
1J
-15be estimated for the cell boundaries.
In the remaining of this section t we consider the case of continuous
and univariate F
l
and F2t and provide suitable confidence bands to them.
For this t let us define
(4·26)
We shall provide two alternative confidence bands.
The first one t analogous
to the confidence bands for p{X<Y}t based on two independent samples t considered
by Birnbaum and McCarty (1958), is based on the two Ko1mogorov statistics for
G
n
and G
n
l
The second one is based on the technique of Sen, Bhattacharyya
2
and Suh (1973, Section 4).
Note that from (2·2) and (4·1), we have
(4·27)
so that
s~p I ;1 (x)-F l (x)
=n-1D
where D
n
l
and D
n
n
+S-lD
l
n
= D
(Say),
n n
2
l 2
(4·28)
are the one-sample Kolmogorov statistics whose distributions
2
do not depend on G and G2t when these are continuous.
l
Let us denote the df of
D by ¢(x;n ), nl~l. Note that for small n lt extensive tables for ¢(x;n l ) are
l
nl
available [viz., Birnbaum (1952)t Owen (1962)], while for large n lt
¢(n
J""
1
2
x·n
' 1
for every fixed x(>O).
)~
00
L(x)=1-2L:.
J=l
j-l -2j 2 x 2
e'
(4·29)
From (4·28), we have
P{D
<x} = p{n-1D +S-lD <x}
n n n
n 2l
l 2
= P{D
•
so that for every a>O,
(-1)
n
+8D
l
<nx}
n -
2
J~x¢(nx-oy;nl)d¢(y;n2)'
(4·30)
-16-
p{S~pl;1(X)-F1(X)I~a}:f~a~(na-Oy;n1)d~(y;n2)'
(4'31)
which provides the desired distribution-free confidence bounds to F , by
1
equating the right hand side of (4'31) to the desired confidence coefficient
For large n ,n , we let [as in (4.22)]
1 2
1-a,0<a<1, by a proper choice of a.
n /n=A n and assume that
1
lim
n~oo
A =A exists and A <A<l-A .
n
0- -
0
(4' 32)
Let then
and D* =~D ,i=1,2.
n.1.
1. n.
1.
Then, by (4'29) and (4'30), under (4'32),
P{D*<a}=P{D* +6/A/(1-A)D* <anlX)
n-
~
_e
n
n -
1
2
f08a/1-A L(an rA- u6/A/(1-A))dL(u)
h
(4·33)
8a/1-A
= 1-{[1-L(8a/1-A)]+
l-2~~
f O {1-L(an~-u6/A/(1-A)>>dL(u)}
<_1)j-1e-2j282(1-A)2a2_
J=l
",00
(-l)j+£
2 ",00
~j=l~£=l
for every a>O.
~ 2
2 2
J0Sa~1-A.2
-2J >dna-6u/vl-A] 402 -2£ u d
e
ue
u,
N
The series approximation in (4'33) is usually quite rapidly
convergent, and for specific 8, n and a, only a few terms on the right hand
side of (4'33) gives an adequate approximation.
For the second procedure, we define
v =
(4.34)
Then, we have the following.
THEOREM 4·2.
Under (4'32), for every a>O,
n~ p{s~p ,mIF1 (x;n1 ,n2)-F1 (x) I.::.a}
> L(a/IV)
-
=
. 1 - 2.J 2 a 2V-1
J1-2~. 1(-1)
e
,
00
J=
where the equality sign holds when G =G .
1 2
(4' 35)
-17The proof of the theorem is sketched in the matematical appendix.
Since
n ,n 2 , PI and P2 are all specified, V is specified, so that by equating the
l
right hand side of (4.35) to our desired confidence coefficient I-a
one gets a confidence band for Fl'
The case of F
follow similarly.
2
(O<a<l),
Computa-
tionally, (4'35) is a lot simpler than (4'33).
5. THE CASE OF MORE THAN TWO QUESTIONS.
Suppose that Al, •.•
,At,t~2,
are the t questions, and in the ith sample,
a respondent selects (at random) the question A. with the probability P .. ,
J
for j=l, ... ,t,i=l, ... ,t.
questions Al, ..• ,A
t
1J
The actual distributions of the responses for the
are denoted by Fl(x), ..• ,Fl(x), respectively, while the
distribution of the response for the ith sample observations is denoted by
G.(x),i=l, ..• ,t.
1
Then
E'_-t l p .. F.(x), l<i<t.
G. (x)
1
.e
J
1J
J
(5 '1)
--
Let us denote by
(5' 2)
and assume that P is positive definite, so that pis positive definite.
needed.]
l
exists.
[For t=2, Pl>P2==7
But for t>2, more stringent condition on the
p ..
1J
may be
Also, let
G(x)
Gl
(X)J and ~ (x)
:.
[ Gt(x)
I~ (X)l
(5' 3)
~~(X)
Then by (5'1) - (5'3),
G(x) = PF(x)
i.e., F(x) = P-lG(x)
(5' 4)
The last equation provides the necessary clue for the estimation of F and its
functionals, and the results of Sections 3 and 4 can be readily extended for
t>2.
For intended brevity, these are not reproduced.
t
-18-
MATHEMATICAL APPENDIX
1. The proof of Theorem 4·1.
holds for i=Z.
We prove (4'23) only for i=l, as the same proof
Also, for simplicity of proof, we consider the univariate case
where Fl,F Z (and hence, G and G ) are defined on the real line (_00,00); the
Z
l
multivariate extension is straightforward, and hence, is not considered.
We
a.s. (almost surely). (A.l)
We make use of the elegant Bahadur representation of sample quantiles, as
extended to the case of non-identically distributed random variables [viz.,
Sen (1968)] along with the basic inequality in Theorem 4·Z of Sen and Ghosh
(1971), and following a few standard steps, obtain that under (4·22), as n
"-1
~OO,
"-1
Sup{!F l (F l (u);nl,nZ)-F l (F l (t);nl,nz)-(u-t)l:
.e
u,tE:[O,l] and lu-tl < n
almost surely.
As such, as n
----700 ,
J
=
-1 logn}
o(n- 3/ 4logn),
(A. Z)
for every €>O,
O(n- 3 / 4 logn),
(A.3)
almost surely, which implies that
"
"-1
F (F -1 (t+n J'1 £);nl,nZ»F
(F (t);nl,n ) a.s. for all O<t<l. ( A·4 )
l l
Z
l l
Therefore, by (4'Zl) and (A.4), as
"
-1
Fl(F l (t-n
J'1.
~
n~oo,
for every €>O,
-1.
A
-1
€),n l ,n 2<F l (F (t),nl,nZ)~Fl (F l (t+n
l
for all O<t<l, almost surely.
Consequently, as
J'1).
)
€ ,nl,n Z
(A' 5)
n~oo,
(A' 6)
-19Since
£(>0) is arbitrary and n
2. The proof of Theorem 4'2.
-1/4
10gn~0
as n
~OO,
the proof follows. Q.E.D.
By (4·27), (4-32) and standard results on the
weak convergence of empirical processes to Gaussian functions, it follows
1
1
A
that as n~oo, {n~[F1(F~ (t);n ,n )-t], O<t<l} converges in law to a Gaussian
1 2
function W = {W(t),O~t~l} where EW(t)=o, O~t~l, and for O~s~t<l,
(A' 7)
Using (4'34), (A'7) simplifies to
-1
-1
-1
-1
-1
-1
V{H(F 1 (s»[1-H(F (t»]-~(1-~)[G1(F1 (s»-G 2 (F (s»)][G (F (t»-G (F (t»]},
1
1
2 1
2 1
where
H(x) =
~G1 (x)+(1-~)G2(x)
for -oo<x<oo.
(A'8)
(A'9)
o
O
o
Let now {W (t) ,O~t~l} = W be a Gaussian function with EW (t) = 0 for 0.::.t.::.1,
.e
EWo(s)Wo(t) = VH(F~l(s»[l-H(F~l(t»].
Note that
every d>O,
(A'lO)
{VJ~Wo(F1(H-1(t»),0~t,::,1} is a standard Brownian bridge, so that for
pJ\.0 <t<l
Sup IWo ( ) I<J=
t-1
1
=
L(d/v~) =
(A' H)
where the last step follows from the well-known result on the Brownian bridge
on [0,1], viz., Billingsley (1968,p.
85
).
Also, note that for every
m(~l)
and arbitrary t = (t , ... ,t ) (with 0<t <···<t <1), if D (t ) and DO(t ) be
1
-m
m
- 1
m-m -m
-m -m
o
0
the dispersion matrices of [W(t ),···,W(t )] and [W (t ),"',W (t m)], respectively,
1
1
m
then by (A'8) and (A'10),
(A' 12)
where
~:
=
V~(1-~)«[G1 (F
positive semi-definite.
-1
-1
-1
-1
1 (t i »-G2 (F 1 (ti))][G l (F 1 (t j »-G 2 (F 1 (t j »]»
Consequently, by Lemma 4'4 of Sen,
Suh (1973), we have for every d>O,
is
Bhattachary~a
and
-20-
pf:';~mIW(tj)I<dfP f:';~mlw"(t j)1:+
where the equality sign holds when D*
-m
= O.
Since (A·13) holds for every m>l
and arbitrary O<tl<···<t <1, passing on to the limit
-
that for every
m-
(A-13)
(m~oo),
we conclude
d~O,
(A.14)
where the equality sign holds when G =G •
l 2
A
Therefore, by the weak convergence
-1
of {!:n[Fl(F l (t);nl,n2)-t],O~t~1} to W, and by (A·ll) and (A·14), we obtain
that for every a>O,
.e
(A· IS)
which completes the proof.
-21REFERENCES
Abernathy, J.R., Greenberg, B.G., Horvitz, D.G. "Estimates of induced abortion
in Urban North Carolina," Demography I, (1970) 19-29.
Billingsley, P. (1968). Convergence of Probability Measures. New York, John
Wiley.
Birnbaum, Z.W. (1952). Numerical tabulation of the distribution of Kolmogorov's statistic for finite sample sizes. Jour. Amer. Statist. Assoc.
!!2, 425Birnbaum, Z.W., McCarty, R.C. (1958). A distribution-free opper bound for
Pr{X<Y} based on independent samples of X and Y. Ann. Math. Statist.
~, 558-583.
Fraser, D.A.S. (1953).
Math • .§., 42-
Completeness of the order statistics.
Canadian Jour.
Greenberg, B.G., Kuebler, R.R., Jr., Abernathy, J.R., Horvitz, D.G. "Application of the randomized response technique in obtaining quantitative
data". Jour. Amer. Statist. Assoc. 66, (1971) 243-250.
.e
Halmos, P.R. (1946).
.!I, 34-43 .
The theory of unbiased estimation.
Ann. Math. Statist .
Hoeffding, W. (1948). A class of statistics with asymptotically normal
distribution. Ann. Math. Statist. 19, 293-325.
Mises, R. von (1967). On the asymptotic distribution of differentiable
statistical functions. Ann. Math. Statist. 18, 309-348.
Owen, D.B. (1962).
Wesley.
Handbook of Statistical Tables. Reading, Mass; Addison-
Puri, M.L., Sen, P.K. (1971).
New York, John Wiley.
Nonparametric Methods in Multivariate Analysis.
Sen, P.K. (1960). On some convergence properties of U-statistics.
Statist. Assoc. Bull. 10, 1-18.
Calcutta
Sen, P.K. (1968). Asymptotic normality of sample quantiles for m-dependent
processes. Ann. Math. Stat. 39, 1724-1730.
Sen, P.K. Bhattacharyya, B.B., Suh, M.W. (1973). Limiting behavior of the
extremum of certain sample functions. Ann. Statist. 1,
Sen, P.K., Ghosh, M. (1971). On bounded length sequential confidence intervals
based on one-sample rank order process. Ann. Math. Statist. 42, 189-203.