Statistica Sinica: Supplement
POPULATION EMPIRICAL LIKELIHOOD FOR NONPARAMETRIC
INFERENCE IN SURVEY SAMPLING
Sixia Chen and Jae Kwang Kim
Iowa State University
Supplementary Material
In this supplement material, we provided proofs for Theorems in the paper.
S1
Proof of Theorem 1
We first prove the consistency of (θ̂, λ̂). Let
Q̂1 (θ, λ) =
N
1 X Ii πi−1 fN Ui (θ)
,
nB i=1
1 + λ0 Ψi
Q̂2 (θ, λ) =
N
Ψi
1 X
,
nB i=1 1 + λ0 Ψi
(S1.1)
0
where fN = nB /N, Ψi = fN (Ii πi−1 − 1), Ii πi−1 (h0i − h̄0N ) . Let λ̂ = ρδ, where ||δ|| = 1,
so according to (S1.1), we have
0
=
||n−1
B
= n−1
B
N
X
Ψi
i=1
1 + λ̂0 Ψi
N
X
i=1
= n−1
B
N
X
N
X
|| ≥ n−1
B
i=1
δ 0 Ψi 1 + ρδ 0 Ψi
δ 0 Ψi (1 + ρδ 0 Ψi − ρδ 0 Ψi ) 1 + ρδ 0 Ψi
δ 0 Ψi − n−1
B
i=1
N
X
ρδ 0 Ψi Ψ0i δ 1 + ρδ 0 Ψi
i=1
N
N
X
−1 X
ρδ 0 Ψi ΨTi δ −1
0
≥ |nB
δ Ψi | − |nB
|.
1 + ρδ 0 Ψi
i=1
i=1
Hence,
|
N
1 X 0
δ Ψi | =
nB i=1
≥
|
N
1 X ρδ 0 Ψi Ψ0i δ
|
nB i=1 1 + ρδ 0 Ψi
N
0 1 X
|ρ|
δ
Ψi Ψ0i δ ,
nB i=1
1 + |ρ|u∗
(S1.2)
S2
SIXIA CHEN AND JAE KWANG KIM
where u∗ = maxi∈A ||Ψi ||.
PN
0
Under assumption (C3), we have n−1
B
i=1 Ψi Ψi = ΣΨ + op (1), and ΣΨ is a positive
definite matrix. Let λp be the smallest eigenvalue of ΣΨ , then λp > 0. So, the following
holds
N
0 −1 X
δ n
Ψi Ψ0i δ ≥ λp + op (1).
(S1.3)
B
i=1
In addition, according to Assumption (C3),
N
1 X 0
−1/2
δ Ψi = Op (nB ).
nB i=1
(S1.4)
By (S1.2), (S1.3), (S1.4) and assumptions (C5), (C6),
−1/2
λp |ρ| = Op (nB
−1/2
Thus, we have |ρ| = Op (nB
−1/2
), which means ||λ̂|| = Op (nB
−1/2
Because maxi∈A |λ̂0 Ψi | = Op (nB
apply Taylor expansion and get
0
) + op (|ρ|).
).
)op (nB ) = op (1) and assumption (C4), we can
=
N
1 X fN Ii πi−1 Ui (θ̂)
nB i=1
1 + λ̂0 Ψi
=
N
1 X
fN Ii πi−1 Ui (θ̂) −
nB i=1
+
Op (n−1
B ).
(
N
1 X
fN Ii πi−1 Ui (θ̂)Ψ0i
nB i=1
)
λ̂
(S1.5)
By assumption (C4), it can be shown that
n−1
B
N
X
fN Ii πi−1 Ui (θ̂)Ψ0i = Op (1),
(S1.6)
i=1
and
sup ||
θ∈Θ
N
N
1 X
1 X
fN Ii πi−1 Ui (θ) −
Ui (θ)|| →p 0,
nB i=1
N i=1
(S1.7)
POPULATION EMPIRICAL LIKELIHOOD
S3
so according to (S1.5), (S1.6) and (S1.7),
0
=
N
1 X
fN Ii πi−1 Ui (θ̂)
p lim nB i=1
=
N
N
N
1 X
1 X
1 X
p lim fN Ii πi−1 Ui (θ̂) −
Ui (θ̂) +
Ui (θ̂)
nB i=1
N i=1
N i=1
≥
N
N
N
1 X
1 X
1 X
p lim |
fN Ii πi−1 Ui (θ̂) −
Ui (θ̂)| − |
Ui (θ̂)|
nB i=1
N i=1
N i=1
=
N
1 X
Ui (θ̂).
N i=1
(S1.8)
By (S1.8), assumptions (C1) and (C2), we have θ̂ →p θ0 . Hence,
(θ̂P OEL , λ̂) →p (θ0 , 0).
(S1.9)
According to (S1.9), assumptions (C2) and (C4), we can apply the standard arguments using Taylor expansion to get
0 = Q̂1 (θ̂, λ̂) = Q̂1 (θ0 , 0) +
∂ Q̂1 (θ0 , 0)
∂ Q̂1 (θ0 , 0)
(θ̂ − θ0 ) +
(λ̂ − 0) + op (δn ),
∂θ0
∂λ0
0 = Q̂2 (θ̂, λ̂) = Q̂2 (θ0 , 0) +
∂ Q̂2 (θ0 , 0)
∂ Q̂2 (θ0 , 0)
(θ̂ − θ0 ) +
(λ̂ − 0) + op (δn ),
∂θ0
∂λ0
and
with δn = ||θ̂ − θ0 || + ||λ̂||. Let
∂ Q̂1 (θ0 , 0)/∂λ ∂ Q̂1 (θ0 , 0)/∂θ
Sn =
.
∂ Q̂2 (θ0 , 0)/∂λ ∂ Q̂2 (θ0 , 0)/∂θ
Under the existence of moments, we can obtain
∗
∗
S11 S12
Sn →p
,
∗
S21
0
and
∗
||S11
− S11 || = op (1),
∗
||S12
− S12 || = op (1),
∗
||S21
− S21 || = op (1),
(S1.10)
where
S11 = −
N
N
1
1 X
1
1 X
fN ( − 1)Ui ,
fN Ui (hi − h̄N )0 ,
N i=1
πi
N i=1
πi
S12 = N −1
N
X
∂Ui (θ0 )
i=1
,
∂θ
(S1.11)
S4
SIXIA CHEN AND JAE KWANG KIM
and
S21 = −
N
!
PN
−1
−1
−1
0
f
(π
−
1)
N
f
(π
−
1)(h
−
h̄
)
N
N
N
i
i
i
i=1
i=1
P
.
N
−1
N −1 i=1 fN πi−1 (hi − h̄N )⊗2
i=1 fN (πi − 1)(hi − h̄N )
(S1.12)
PN
N −1
PN
−1
According to assumption (C3),
Q̂1 (θ0 , 0) = N −1
N
X
−1/2
Ii πi−1 Ui (θ0 ) = Op (nB
),
Q̂2 (θ0 , 0) = N −1
i=1
N
X
−1/2
Ψi (θ0 , 0) = Op (nB
i=1
−1/2
so we have δn = Op (nB
algebra,
). Also, according to (S1.10), (S1.11) and (S1.12), after some
−1/2
−1
λ̂ = −S21
Q̂2 (θ0 , 0) + op (nB
),
(S1.13)
and
θ̂ − θ0
o
n
−1/2
−1
−1
Q̂2 (θ0 , 0) + op (nB )
Q̂1 (θ0 , 0) − S11 S21
−S12
o
n
−1/2
= −τ Q̂1 (θ0 , 0) − B ∗ Q̂2 (θ0 , 0) + op (nB ),
=
(S1.14)
−1
where τ = S12 , B ∗ = Ω1 Ω−1
S11 and Ω2 = −(N αN )−1 S21 . Hence,
2 , Ω1 = −(N αN )
(3.7) in Theorem 1 is proved. Result (3.10) can be obtained by (S1.14) and assumptions
(C3), (C4).
S2
Proof of Theorem 2
Maximizing (3.1) subject to
N
X
ωi = 1,
i=1
N
X
ωi fN (
i=1
Ii
− 1) = 0,
πi
N
X
ωi
i=1
Ii
fN (hi − h̄N ) = 0,
πi
leads, after some algebra, to
l(θ̂) =
N
X
log(ωi (θ̂)) = −N log(N ) −
i=1
N
X
log(1 + λ̂0 Ψi1 ),
i=1
where Ψi1 = (fN (Ii πi−1 − 1), Ii πi−1 fN (hi − h̄N )0 )0 . Similarly, consider maximizing (3.1)
subject to
N
X
i=1
ωi = 1,
N
X
i=1
ωi fN (
Ii
− 1) = 0,
πi
N
X
i=1
ωi
Ii
fN (hi − h̄N ) = 0,
πi
),
POPULATION EMPIRICAL LIKELIHOOD
S5
and
N
X
ωi fN ri = 0,
i=1
−1
with ri = Ii πi−1 Ui (θ0 )−B1∗ (Ii πi−1 −1)−B2∗ Ii πi−1 (hi − h̄N ) and B ∗ = (B1∗ , B2∗ ) = S11 S21
,
where S11 , S21 are defined in (S1.11) and (S1.12) of the proof of Theorem 1. After some
algebra,
N
N
X
X
l(θ0 ) =
log(ωi (θ0 )) = −N log(N ) −
log(1 + λ00 Ψi2 ),
i=1
where Ψi2 =
(fN (Ii πi−1
−
i=1
1), Ii πi−1 fN (hi
− h̄N )
0
, fN ri0 )0 .
So, we can write
N
N
X
X
Rn (θ0 ) = 2
log(1 + λ00 Ψi2 ) −
log(1 + λ̂0 Ψi1 ) ,
i=1
(S2.1)
i=1
and λ0 is the solution of Q̂3 (θ0 , λ) = 0 with
Q̂3 (θ0 , λ) =
N
Ψi2 (θ0 )
1 X
.
nB i=1 1 + λ0 Ψi2 (θ0 )
By the same argument for (S1.9), we have λ0 →p 0. We can apply a Taylor expansion
to get
∂ Q̂3 (θ0 , 0)
0 = Q̂3 (θ0 , λ0 ) = Q̂3 (θ0 , 0) +
λ0 + op (||λ0 ||).
∂λ
PN
−1/2
According to assumption (C3), Q̂3 (θ0 , 0) = n−1
), hence
B
i=1 Ψi2 (θ0 , 0) = Op (nB
−1/2
||λ0 || = Op (nB ), so
−1/2
λ0 = −S −1 Q̂3 (θ0 , 0) + op (nB ),
(S2.2)
with
S21
0
S=
0
Sr
,
(S2.3)
where
Sr
= fN {−N Vpoi (r̄N ) − N −1
N
X
Ui⊗2 − B2∗ N −1
i=1
+ N −1
N
X
i=1
0
Ui (hi − h̄N ) B2∗
0
N
X
(hi − h̄N )⊗2 B2∗
0
i=1
N
X
0
+ B2∗ N −1
(hi − h̄N )Ui },
(S2.4)
i=1
PN
and r̄N = N −1 i=1 ri , S21 is defined in (S1.12) in the proof of Theorem 1 and Vpoi is
the variance under Poisson sampling.
According to assumption (C6), nB /N = o(1) and (S2.4), it can be shown that
||Sr + fN N Vpoi (r̄N )|| = o(1).
(S2.5)
S6
SIXIA CHEN AND JAE KWANG KIM
Similarly, by a Taylor expansion with respect to λ0 = 0,
2
N
X
log(1 + λ00 Ψi2 ) = 2
i=1
N
X
λ00 Ψi2 −
i=1
N
X
λ00 Ψi2 Ψ0i2 λ0 + op (1).
(S2.6)
i=1
According to (S2.2), we have
∂ Q̂3 (θ0 , 0)/∂λ = −n−1
B
N
X
Ψi2 (θ0 )Ψ0i2 (θ0 ) →p S.
(S2.7)
i=1
By plugging (S2.2) into (S2.6) and according to (S2.7), we have
2
N
X
log(1 + λ00 Ψi2 (θ0 )) = −nB Q̂03 (θ0 , 0)S −1 Q̂3 (θ0 , 0) + op (1).
(S2.8)
i=1
Similarly, according to (S1.13) and by using a Taylor expansion around λ̂ = 0,
2
N
X
log(1 + λ̂0 Ψi1 (θ̂))
−1
= −nB Q̂02 (θ0 , 0)S21
Q̂2 (θ0 , 0) + op (1).
(S2.9)
i=1
By assumption (C3) and (C4), we can apply the central limit theorem to get
−1/2
Vpoi
(r̄N )r̄N →d N (0, I).
(S2.10)
Therefore, plugging (S2.8) and (S2.9) into (S2.1) and by (S2.3), we have
Rn (θ0 )
=
−1
Q̂2 (θ0 , 0) + op (1)
−nB Q̂03 (θ0 , 0)S −1 Q̂3 (θ0 , 0) + nB Q̂02 (θ0 , 0)S21
=
−1
r̄N (−n−1
(r̄N )0 + op (1).
B Sr )
(S2.11)
According to (S2.5), (S2.11) and (S2.10),
Rn (θ0 ) = r̄N {Vpoi (r̄N )}
where r̄N = N −1
S3
PN
i=1 ri ,
−1
(r̄N )0 + op (1) →d χ2p ,
and p is the dimension of θ0 .
Proof of Theorem 3
Similar as the proof of Theorem 1, θ̂ can be obtained by solving Q̂1 (θ, λ) = 0 and
Q̂2 (θ, λ) = 0, where Q̂1 (θ, λ) and Q̂2 (θ, λ) are defined in (S1.1) with πi , Ψi replaced by
0
−1 0
∗
0
∗
0 0
pi , Ψ∗i , and Ψ∗i = fN (Ii p−1
i − 1)zi , fN Ii pi (hi − h̄N ) , with zi = (1, zi ) . Without loss
P
N
2
of generality, we assume z̄N = 0 and nB N −2 i=1 (1 − pi )p−1
i zi = 1. Hence, according
to assumption (C9) and (C10) in Section 4,
πi
=
P r(i ∈ s|Q̂p,n ≤ γ 2 ) =
P r(Q̂p,n ≤ γ 2 |i ∈ s)P r(i ∈ s)
= pi 1 + Cγ ηi + op (n−1
B ) ,
P r(Q̂p,n ≤ γ 2 )
(S3.1)
POPULATION EMPIRICAL LIKELIHOOD
S7
and
πij
P r(Q̂p,n ≤ γ 2 |i, j ∈ s)P r(i, j ∈ s)
=
P r(i, j ∈ s|Q̂p,n ≤ γ 2 ) =
=
pij 1 + Cγ (ηi + ηj ) + op (n−1
B ) ,
P r(Q̂p,n ≤ γ 2 )
(S3.2)
2
with Cγ = g1N (γ 2 )G−1
N (γ ).
According to (S3.1), (S3.2) and by using a similar argument as the proof of Theorem
1, it can be shown that (θ̂, λ̂) →p (θ0 , 0). After some algebra,
n
o
−1/2
−1
−1
θ̂ − θ0 = −S12
Q̂1 (θ0 , 0) − S11 S21
Q̂2 (θ0 , 0) + op (nB ),
(S3.3)
where
S11 = −(N −1
N
X
N
X
0
−1
∗
−1
fN (πi p−2
i − πi pi )Ui zi , N
i=1
0
fN πi p−2
i Ui (hi − h̄N ) ),
i=1
S12 =
N
1 X πi ∂Ui (θ0 )
N i=1 pi ∂θ
and
S21 = −N
−1
PN
∗ ∗0
f (π p−2 − 2πi p−1
i + 1)zi zi
PNi=1 N i 2i
−1
∗0
i=1 fN (πi pi − πi pi )(hi − h̄N )zi
PN
∗
0
fN (πi p2i − πi p−1
i )zi (hi − h̄N )
i=1P
N
−2
⊗2
i=1 fN πi pi (hi − h̄N )
By (S3.1) and (S3.2),
||
N
N
1 X Ii
1 X Ii
−1/2
Ui (θ0 ) −
Ui (θ0 )|| = op (nB ).
N i=1 pi
N i=1 πi
Hence,
N
N
1 X Ii
1 X Ii
−1/2
Ui (θ0 ) =
Ui (θ0 ) + op (nB ).
N i=1 pi
N i=1 πi
(S3.4)
Similarly, it can be shown that
N
N
1 X Ii
1 X Ii
−1/2
( − 1)zi∗ =
( − 1)zi∗ + op (nB ),
N i=1 pi
N i=1 πi
(S3.5)
N
N
1 X Ii
1 X Ii
−1/2
(hi − h̄N ) =
(hi − h̄N ) + op (nB ),
N i=1 pi
N i=1 πi
(S3.6)
∗
||S12 − S12
|| = op (1),
∗
||S11 − S11
|| = op (1),
∗
||S21 − S21
|| = op (1),
!
.
S8
SIXIA CHEN AND JAE KWANG KIM
where
∗
S11
= −(N −1
N
X
0
fN (πi−1 − 1)Ui zi∗ , N −1
i=1
N
X
fN πi−1 Ui (hi − h̄N )0 ),
i=1
∗
S12
=
1
N
N
X
i=1
∂Ui (θ0 )
∂θ
and
∗
S21
=−
PN
0
N −1 i=1 fN (πi−1 − 1)zi∗ zi∗
P
0
N
N −1 i=1 fN (πi−1 − 1)(hi − h̄N )zi∗
!
PN
N −1 i=1 fN (πi−1 − 1)zi∗ (hi − h̄N )0
PN
.
N −1 i=1 fN πi−1 (hi − h̄N )⊗2
Hence according to previous derivations and (S3.3),
)
(
N
N
1 X Ii
1 X Ii
−1/2
θ̂ − θ0 = −τ
Ui (θ0 ) − B(
ηi − η̄N ) + op (nB )
N i=1 πi
N i=1 πi
(S3.7)
∗
−1 ∗
∗
with τ = S12
, η = (zi∗ , (h−h̄N )0 )0 , B = Ω1 Ω−1
S11 , Ω2 = −(N fN )−1 S21
.
2 , Ω1 = −(N fN )
Thus, (4.9) in Theorem 3 is proved.
PN
Let ei = Ui − Bηi and ēˆp = N −1 i=1 Ii p−1
i ei . Next we want to prove
||Vrej (ēˆp ) − Vpoi (ēˆp )|| = op (n−1
B ),
(S3.8)
where Vrej and Vpoi denote the variances under rejective Poisson sampling and Poisson
sampling, respectively. According to (S3.1) and (S3.2),
Vrej (ēˆp )
=
N N
1 X X πij − πi πj 0
ei ej
N 2 i=1 j=1
pi pj
=
N
1 X πij − πi πj 0
1 X πi − πi2 ⊗2
e
+
ei ej
i
N 2 i=1 p2i
N2
pi pj
i6=j
N
X
1 − pi ⊗2
1
ei + 2
pi
N
N
X
=
1
N2
+
nB
1 X −1 −1
pi pj op ( 2 )ei e0j + op (n−1
B )
N2
N
i=1
−2 2 ⊗2
(1 − pi )p−2
zi ei
i nB N
i=1
i6=j
=
N
1 X 1 − pi ⊗2
−1
ˆ
e + op (n−1
B ) = Vpoi (ēp ) + op (nB ).
N 2 i=1 pi i
So, (S3.8) is proved. Together with (S3.4), (S3.5) and (S3.6),
||Vrej (ēˆHT ) − Vpoi (ēˆp )|| = op (n−1
B ),
where ēˆHT = N −1
(S3.9)
PN
−1
i=1 Ii πi ei .
Hence, result (4.12) in Theorem 3 can be obtained by (S3.7), (S3.9) and assumptions
(C3), (C4).
POPULATION EMPIRICAL LIKELIHOOD
S4
S9
Proof of Theorem 4
By using the argument similar to the proof of Theorem 2, it can be shown that
−1
Rn (θ0 ) = r̄N {Vpoi (r̄N )}
(r̄N )0 + op (1),
(S4.1)
−1
where r̄N = Q̂1 (θ0 , 0) − S11 S21
Q̂2 (θ0 , 0), and Q̂1 (θ0 , 0), Q̂2 (θ0 , 0), S11 , S21 are defined in
PN
(S3.3) of the proof for Theorem 3. r̄N = N −1 i=1 ri , and p is the dimension of θ0 .
According to (S3.8) in the proof of Theorem 3 and (S4.1), we have Rn (θ0 ) →d χ2p .