Solutions
Stat755
Homework 7
9.1 Show that the covariance matrix ⎡1.0 .63 .45⎤
ρ = ⎢⎢.63 1.0 .35⎥⎥ ⎢⎣.45 .35 1.0 ⎥⎦
For the p = 3 standardized random variables Z1 , Z 2 and Z3 can be generated by the
m = 1 factor model Z1 = .9 F1 + ε1
Z 2 = .7 F2 + ε 2 Z 3 = .5 F3 + ε 3
0⎤
⎡.19 0
⎢
where var ( F1 ) = 1, cov ( ε, F1 ) = 0, and Ψ = cov ( ε ) = 0 .51 0 ⎥ . That is, write ρ in ⎢
⎥
⎢⎣ 0
0 .75⎥⎦
the form ρ = LLT + Ψ. ⎡.9 ⎤
Sln: Since Z is the standardized r.v.s, and L = ⎢.7 ⎥ , we have the following: ⎢ ⎥
⎢⎣.5 ⎥⎦
0 ⎤ ⎡.81 .63 .45⎤ ⎡1.0 .63 .45⎤
⎡.9 ⎤
⎡.19 0
⎢
⎥
⎢
LL + Ψ = ⎢.7 ⎥ [.9 .7 .5] + ⎢ 0 .51 0 ⎥⎥ = ⎢⎢.63 .49 .35⎥⎥ + ⎢⎢.63 1.0 .35⎥⎥ = ρ. ⎢⎣.5 ⎥⎦
⎢⎣ 0
0 .75⎥⎦ ⎢⎣.45 .35 .25⎥⎦ ⎢⎣.45 .35 1.0 ⎥⎦
T
9.2 Use the information in Exercise 9.1 (a) Calculate communalities hi2 , i = 1, 2,3, and interpret these quantities. (b) Calculate Corr ( Z i , F1 ) for i = 1, 2,3. Which variable might carry the greatest weight in “naming” the common factor? Why? Sln: (a) Since m = 1, then, by Equ. (9‐6), the communalities are calculated as the following: 2
h12 = A11
= .92 = .81, h22 = A 221 = .7 2 = .49, and h32 = A 231 = .52 = .25. 1 Solutions
Stat755
Homework 7
(b) By Equ. (9‐5), Corr ( Z i , F1 ) = A i1. Therefore, Corr ( Z, F1 ) = L = [.9 .7 .5] . Because T
the first variable Z1 has the largest correlation with common factor, Z1 will carry greatest weight in term of F1. 9.3 The eigenvalues and eigenvectors of the correlation matrix ρ in Exercise 9.1 are λ1 = 1.96,
e1T = [.625 .593 .507 ]
λ2 = .68,
e2T = [ − .219 −.491 .843] λ3 = .36,
e3T = [.749 −.638 −.177 ]
(a) Assuming an m = 1 factor model, calculate the loading matrix L and matrix of specific variances Ψ using the principal component solution method. Compare the results with those in Exercise 9.1. (b) What proportion of the total population variance is explained by the first common factor? Sln: (a) By principal component solution methods (9‐15, 9‐16 and 9‐17), we have: i = ⎡ λ e ⎤ = [.875 .830 .710]T . L
⎣ 1 1⎦
m
m
j =1
j =1
Ψ1 = ρ11 − ∑ A 12 j = 1.0 − .8752 = .234; Ψ 2 = ρ 22 − ∑ A 22 j = 1.0 − .8302 = .311 and
m
Ψ 3 = ρ33 − ∑ A 23 j = 1.0 − .7102 = .496. Thus, the matrix of specific variances is: j =1
0 ⎤
⎡.234 0
⎢
i = 0 .311 0 ⎥ . Ψ
⎢
⎥
⎢⎣ 0
0 .496 ⎥⎦
Both estimation is different from the results of Exercise 9.1. (b) By Equ. (9‐20), the proportion of the total estimated population variance due to ˆ
the first common factor is: λp1 = 1.96
for a factor analysis of matrix ρ. 3 = 65.3%,
9.4 Given matrix ρ and Ψ in Exercise 9.1 and an m = 1 factor model, calculate the reduced correlation matrix ρ = ρ − Ψ and the principal factor solution for the loading matrix L. Is the result consistent with the information in Exercise 9.1? Should it be? Sln: The reduce correlation matrix is calculated as: 2 Solutions
Stat755
Homework 7
0 ⎤ ⎡.81 .63 .45⎤
⎡1.0 .63 .45⎤ ⎡.19 0
⎢
⎥
⎢
ρ = ρ − Ψ = ⎢.63 1.0 .35⎥ − ⎢ 0 .51 0 ⎥⎥ = ⎢⎢.63 .49 .35⎥⎥ . ⎢⎣.45 .35 1.0 ⎥⎦ ⎢⎣ 0
0 .75⎥⎦ ⎢⎣.45 .35 .25⎥⎦
(b) The first eigen pair for reduced correlation matrix is:
T
λ1 = 1.55; e1 = [.723 .562 .402] . Therefore, by the Principal Component method, for m = 1, L = λ1 e1 = 1.55 [.723 .562 .402] = [.9 .7 .5] . The result should be T
T
consistent with the information in Exercise 9.1, because ρ = ρ − Ψ = LLT . When m = 1 , L is fully determined. In other words, orthogonal rotation doesn’t exist in \1. 9.6 Verify the following matrix identities. (a) ( I + LT Ψ −1L ) LT Ψ −1L = I − ( I + LT Ψ −1L ) Hint: pre‐multiply both sides by
−1
−1
(I + L Ψ L). −1
T
(b) ( LLT + Ψ ) = Ψ −1 − Ψ −1L ( I + LT Ψ −1L ) LT Ψ −1 Hint: post‐multiply both sides by
−1
( LL
T
−1
+ Ψ ) and use (a). (c) LT ( LLT + Ψ ) = ( I + LT Ψ −1L ) LT Ψ −1 Hint: Post‐multiply the result in (b) by L, −1
−1
use (a), and take the transpose, noting that ( LLT + Ψ ) , Ψ −1 and ( I + LT Ψ −1L ) are −1
−1
symmetric matrices. Sln: (a) I + L Ψ L )( I + L Ψ L ) L Ψ
(
T
−1
T
−1
−1
T
−1
L = ( I + LT Ψ −1L ) ⎡⎢I − ( I + LT Ψ −1L ) ⎤⎥
⎣
⎦
−1
I
⇔
I ( LT Ψ −1L ) = ( I + LT Ψ −1L ) I − ( I + LT Ψ −1L )( I + LT Ψ −1L ) −1
I
−1
−1
L Ψ L = I+L Ψ L−I
⇔
L−1Ψ −1L = LT Ψ −1L
3 −1
⇔
T
Solutions
Stat755
Homework 7
(b) ( LL
T
+ Ψ)
−1
( LL
T
−1
+ Ψ ) = ⎡⎢ Ψ −1 − Ψ −1L ( I + LT Ψ −1L ) LT Ψ −1 ⎤⎥ ( LLT + Ψ )
⎣
⎦
I = Ψ −1 ( LLT + Ψ ) − Ψ −1L ( I + LT Ψ −1L ) LT Ψ −1L LT
−1
⇔
(
I − I + LT Ψ −1L
)
−1
by ( a )
− Ψ −1L ( I + LT Ψ −1L ) LT Ψ −1Ψ
−1
−1
I = Ψ −1LLT + I − Ψ −1L ⎡⎢I − ( I + LT Ψ −1L ) ⎤⎥ LT
⎣
⎦
⇔
− Ψ −1L ( I + LT Ψ −1L ) LT
−1
⇔
I = Ψ −1LLT + I − Ψ −1LLT
⇔
I=I
(c) ( LL
( LL
T
⇔
T
+ Ψ ) = Ψ −1 − Ψ −1L ( I + LT Ψ −1L ) LT Ψ −1
−1
−1
+ Ψ ) L = Ψ −1L − Ψ −1L ( I + LT Ψ −1L ) LT Ψ −1L
−1
−1
by ( a )
⇔
( LL
−1
−1
+ Ψ ) L = Ψ −1L − Ψ −1L ⎡⎢I − ( I + LT Ψ −1L ) ⎤⎥
⎣
⎦
⇔
( LL
+ Ψ ) L = Ψ −1L ( I + LT Ψ −1L )
T
T
−1
−1
−1
−1
T
⇔ LT ⎡⎢( LLT + Ψ ) ⎤⎥ = ⎡⎢( I + LT Ψ −1L ) ⎤⎥ LT ⎡⎣ Ψ −1 ⎤⎦
⎣
⎦
⎣
⎦
T
T
⇔
LT ( LLT + Ψ ) = ( I + LT Ψ −1L ) LT Ψ −1
−1
−1
9.7 (The factor model parameterization need not be unique) Let the factor model 2
with p = 2 and m = 1 prevail. Show that σ 11 = A11
+ ψ 1 , σ 12 = σ 21 = A11A 21 , σ 22 = A 221 + ψ 2 .
and, for given σ11, σ22 and σ12, there is an infinity of choices for L and Ψ. Sln: By (9‐1), Let factor model given as: X 1 − μ1 = A11 F1 + ε1
X 2 − μ2 = A 21 F1 + ε 2
0⎤
⎡ψ
where, cov ( F1 ) = 1, E ( F1 ) = 0, E ( ε ) = 0, cov ( ε ) = Ψ = ⎢ 1
⎥ . And F1 and ε are ⎣ 0 ψ2⎦
independent. 4 Solutions
Stat755
Homework 7
Therefore, we have: σ 11 = var ( X 1 )
2
= E ⎡( X 1 − μ1 ) ⎤
⎣
⎦
2
= E ⎡( A11 F1 + ε1 ) ⎤
⎣
⎦
= A E ⎡⎣ F ⎤⎦ + 2A11 E [ε1 ] + E ⎡⎣ε ⎤⎦
2
var ( F1 ) + var ( ε1 )
= A11
2
11
2
1
2
1
2
= A11
+ψ 1
and σ 21 = σ 12 = cov ( X 1 , X 2 )
= E ⎡⎣( X 1 − μ1 )( X 2 − μ 2 ) ⎤⎦
= E ⎡⎣( A11 F1 + ε1 )( A 21 F1 + ε 2 ) ⎤⎦
= A11A 21 E ⎡⎣ F12 ⎤⎦ + A11 E [ F1ε 2 ] + A 21 E [ F1ε1 ] + E [ε1ε 2 ]
= A11A 21 var ( F1 ) + A11 cov ( F1ε 2 ) + A 21 cov ( F1ε1 ) + cov ( ε1ε 2 )
= A11A 21
Similarly, we can show σ 22 = A 221 + ψ 2 . Another way to solve this is from Equation Σ = LLT + Ψ. When m = 1, we have 2
+ψ 1 A11A 21 ⎤
⎡σ 11 σ 12 ⎤ ⎡A11
=
⎢
⎥. ⎢σ
⎥
2
⎣ 21 σ 22 ⎦ ⎣ A11A 21 A 21 + ψ 2 ⎦
To show there is an infinity of choices for L and Ψ, we want to find a subinterval for
A11 , A 21 , ψ 1 andψ 2 , where there exists a pair‐wise one‐to‐one correspondence among them. Sinceψ 1 andψ 2 are the covariance term of Ψ, we haveψ 1 ,ψ 2 ≥ 0. Therefore,
2
ψ 1 = σ 11 − A11
⇒ A11 ≤ σ 11 , andψ 2 = σ 22 − A 221 ⇒ A 21 ≤ σ 22 . ⎡ σ
Since A11A 21 = σ 12 , we can solve A11 ∈ ⎢ 12 ,
⎣⎢ σ 22
⎤
⎡ σ 12
σ 11 ⎥ and A 21 ∈ ⎢
,
⎤
σ 22 ⎥ . These ⎣⎢ σ 11
⎦⎥
2
2
⎡
⎡
σ ⎤
σ ⎤
intervals must imply thatψ 1 ∈ ⎢0, σ 11 − 12 ⎥ andψ 2 ∈ ⎢0, σ 22 − 12 ⎥ . Hence, we’ve σ 22 ⎦
σ 11 ⎦
⎣
⎣
5 ⎦⎥
Solutions
Stat755
Homework 7
found an one‐to‐one correspondence between A 11 and A 21 orψ 1 orψ 2 . Since the choice of any variable is infinite, The solutions for A 11 , A 21 ,ψ 1 andψ 2 are infinite. 9.8 (Unique but improper solution: Heywood case.) Consider an m = 1 factor model for the population with covariance matrix ⎡ 1 .4 .9 ⎤
Σ = ⎢⎢.4 1 .7 ⎥⎥ ⎢⎣.9 .7 1 ⎥⎦
Show that there is a unique choice of L and with Σ = LLT + Ψ, but thatψ 3 < 0, so the choice is not admissible. Sln: Σ = LLT + Ψ, and m = 1 implies 2
1 = A11
+ψ 1
.4 = A11A 21
1 = A +ψ 2
2
21
.9 = A11A 31
.7 = A 21A 31 1 = A 231 + ψ 3
The pair equations .9 = A11A 31 and .7 = A 21A 31 implies A 21 =
equation in the equation .4 = A11A 21 yields A11 = ±.717. Therefore, we have A 21 =
.7
A11. Substitute this .9
.7
.9
A11 = ±.558 and A 31 =
= ±1.255. .9
A11
But 1 = A 231 + ψ 3 impliesψ 3 = 1 − A 231 = 1 − 1.575 = −.575 which is inadmissible for covariance matrix. 6