Likelihood Ratio Test
Likelihood Ratio Test
I
Recall: N responses x1 , x2 , . . . , xN are assumed to be independent,
and the αth response is related to q covariate values, represented by
the (q × 1) vector zα , through
xα ∼ Np (βzα , Σ) ,
α = 1, 2, . . . , N.
where β is a (p × q) matrix of regression coefficients.
I
Partition β into (p × q1 ) and (p × q2 ) blocks:
β = (β 1 β 2 )
I
We want to test the null hypothesis
H0 : β 1 = β ∗1
for some given matrix β ∗1 .
NC STATE UNIVERSITY
1 / 12
Statistics 784
Multivariate Analysis
Likelihood Ratio Test
I
Write Σ̂Ω for the unrestricted mle of Σ:
Σ̂Ω =
N
0
1 X
xα − β̂zα xα − β̂zα .
N
α=1
I
The corresponding maximized value of the likelihood is
− 1 N 1
1
2
e − 2 pN
(2π)− 2 pN det Σ̂Ω
NC STATE UNIVERSITY
2 / 12
Statistics 784
Multivariate Analysis
Likelihood Ratio Test
I
To maximize the likelihood under H0 , write
yα = xα − β ∗1 z(1)
α
where zα is similarly partitioned:
(1)
zα =
I
zα
(2)
zα
!
(2)
Then under H0 , yα ∼ Np β 2 zα , Σ , so β 2 is estimated by
β̂ 2,ω =
N
X
0
yα z(2)
α
!
A−1
2,2
α=1
= (C2 − β ∗1 A1,2 ) A−1
2,2
with the corresponding partitioning of A and C.
NC STATE UNIVERSITY
3 / 12
Statistics 784
Multivariate Analysis
Likelihood Ratio Test
I
Then Σ is estimated by Σ̂ω , where
N Σ̂ω =
=
=
N X
α=1
N
X
yα − β̂ 2,ω z(2)
α
yα − β̂ 2,ω z(2)
α
0
0
yα yα0 − β̂ 2,ω A2,2 β̂ 2,ω
α=1
N X
xα − β ∗1 z(1)
α
xα − β ∗1 z(1)
α
0
0
− β̂ 2,ω A2,2 β̂ 2,ω
α=1
I
The corresponding maximized value of the likelihood is
− 1 N 1
1
2
(2π)− 2 pN det Σ̂ω
e − 2 pN
NC STATE UNIVERSITY
4 / 12
Statistics 784
Multivariate Analysis
Likelihood Ratio Test
I
The generalized likelihood ratio criterion is therefore
1N
2
det Σ̂Ω
λ=
1N
2
det Σ̂ω
and the test procedure is to reject H0 if λ < λ0 for appropriate λ0 .
I
The two estimates of Σ are related by
0
N Σ̂ω = N Σ̂Ω + β̂ 1,Ω − β ∗1 A1,1·2 β̂ 1,Ω − β ∗1
where
A1,1·2 = A1,1 − A1,2 A−1
2,2 A2,1 .
NC STATE UNIVERSITY
5 / 12
Statistics 784
Multivariate Analysis
Likelihood Ratio Test
I
The first term, N Σ̂Ω , is the matrix of sums of squares and products
of residuals, or the Error SSCP.
I
The second term,
0
β̂ 1,Ω − β ∗1 A1,1·2 β̂ 1,Ω − β ∗1 ,
is the SSCP matrix associated with the Hypothesis.
I
If q1 = 1, the second term has rank 1, and λ is a monotone function
of the T 2 -like statistic
0 −1 ∗
∗
A1,1·2 β̂ 1,Ω − β 1 Σ̂Ω β̂ 1,Ω − β 1 .
NC STATE UNIVERSITY
6 / 12
Statistics 784
Multivariate Analysis
Likelihood Ratio Test
Invariance
I
If D is (p × p) nonsingular, then
I
I
the null hypothesis H0 : β 1 = 0, and
the likelihood ratio criterion λ for testing it
are both invariant under the transformation x∗α = Dxα .
I
To be precise,
x∗α ∼ Np (β ∗ zα , Σ∗ ) ,
where
β ∗ = Dβ = (Dβ 1 Dβ 2 )
and
Σ∗ = DΣD0
NC STATE UNIVERSITY
7 / 12
Statistics 784
Multivariate Analysis
Likelihood Ratio Test
I
So β ∗1 = Dβ 1 = 0 iff β 1 = 0, showing invariance of H0 .
I
Also (after some algebra)
∗
Σ̂Ω = DΣ̂Ω D0
and
∗
Σ̂ω = DΣ̂ω D0
whence λ∗ = λ.
I
But, when q1 > 1, λ is not the only invariant test statistic (more later
. . . ).
NC STATE UNIVERSITY
8 / 12
Statistics 784
Multivariate Analysis
Likelihood Ratio Test
Distribution of the Likelihood Ratio
I
Write
U = λ2/N
det Σ̂Ω
=
det Σ̂ω
det N Σ̂Ω
=
0 ∗
∗
det N Σ̂Ω + β̂ 1,Ω − β 1 A1,1·2 β̂ 1,Ω − β 1
I
We know that
N Σ̂Ω ∼ Wp (Σ, N − q).
I
Using an appropriate orthogonal transformation, we can also show
that under H0
0
β̂ 1,Ω − β ∗1 A1,1·2 β̂ 1,Ω − β ∗1 ∼ Wp (Σ, q1 ).
NC STATE UNIVERSITY
9 / 12
Statistics 784
Multivariate Analysis
Likelihood Ratio Test
I
Also these two Wishart-distributed matrices are independent.
I
So the criterion U has the distribution of
Up,m,n =
det(E)
1
=
det(E + H)
det(I + E−1 H)
where:
I
I
I
I
E ∼ Wp (Σ, n) , n = N − q;
H ∼ Wp (Σ, m) , m = q1 ;
E and H are independent.
This distribution does not depend on Σ, so we can take Σ = Ip .
NC STATE UNIVERSITY
10 / 12
Statistics 784
Multivariate Analysis
Likelihood Ratio Test
I
Basic result: Up,m,n has the distribution of
p
Y
Vi ,
i=1
where:
I
I
I
Vi has the beta density β 12 (n + 1 − i), 21 m ;
V1 , V2 , . . . , Vp are independent.
Also: Up,m,n has the same distribution as Um,p,n+m−p .
NC STATE UNIVERSITY
11 / 12
Statistics 784
Multivariate Analysis
Likelihood Ratio Test
Special Cases
I
p = 1 (and hence also m = 1):
1 − U1,mn,
n
×
= Fm,n .
U1,m,n
m
I
p = 2 (and hence also m = 2):
p
1 − U2,m,n n − 1
p
×
= F2m,2(n−1) .
m
U2,m,n
I
Bartlett’s correction: for large N, approximately,
"
#
N − q − 12 (p − q1 + 1)
−2 ×
× log λ ∼ χ2pq1 .
N
NC STATE UNIVERSITY
12 / 12
Statistics 784
Multivariate Analysis