Factor Analysis and Structural equation
modelling
Herman Adèr
Previously:
Department Clinical Epidemiology and Biostatistics,
VU University medical center, Amsterdam
Stavanger July 4–13, 2006
Herman Adèr
1
Overview
1
E ←→ C
Exploratory ←→ Confirmatory methods
2
EFA
Exploratory Factor Analysis
3
CFA
Confirmatory Factor Analysis
4
SEM
Structural Equation Modelling
Herman Adèr
2
Overview
1
E ←→ C
Exploratory ←→ Confirmatory methods
2
EFA
Exploratory Factor Analysis
3
CFA
Confirmatory Factor Analysis
4
SEM
Structural Equation Modelling
Herman Adèr
2
Overview
1
E ←→ C
Exploratory ←→ Confirmatory methods
2
EFA
Exploratory Factor Analysis
3
CFA
Confirmatory Factor Analysis
4
SEM
Structural Equation Modelling
Herman Adèr
2
Overview
1
E ←→ C
Exploratory ←→ Confirmatory methods
2
EFA
Exploratory Factor Analysis
3
CFA
Confirmatory Factor Analysis
4
SEM
Structural Equation Modelling
Herman Adèr
2
Overview
1
E ←→ C
Exploratory ←→ Confirmatory methods
2
EFA
Exploratory Factor Analysis
3
CFA
Confirmatory Factor Analysis
4
SEM
Structural Equation Modelling
Herman Adèr
2
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Part VI
Factor analysis and Structural equation modelling
Herman Adèr
Factor Analysis and SEM
3
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Detective ←→ Judge
Herman Adèr
Factor Analysis and SEM
5
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Characterization of Tukey (1977)
Exploratory data analysis is detective in character.
Confirmatory data analysis is judicial or quasi-judicial in
character. . . Unless the detective finds the clues, judge or jury
has nothing to consider. Unless exploratory data analysis
uncovers indications, usually quantitative ones, there is likely to
be nothing for confirmatory data analysis to consider.
On the other hand:
Exploratory data analysis can never be the whole story, but
nothing else can serve as the foundation stone – the first step.
Herman Adèr
Factor Analysis and SEM
7
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Characterization of Tukey (1977)
Exploratory data analysis is detective in character.
Confirmatory data analysis is judicial or quasi-judicial in
character. . . Unless the detective finds the clues, judge or jury
has nothing to consider. Unless exploratory data analysis
uncovers indications, usually quantitative ones, there is likely to
be nothing for confirmatory data analysis to consider.
On the other hand:
Exploratory data analysis can never be the whole story, but
nothing else can serve as the foundation stone – the first step.
Herman Adèr
Factor Analysis and SEM
7
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
F = M ∪ U ∪ E,
of which M indicate the meaningful factors, U so-called
‘unique’ factors (factors on which only one item loads) and E
error factors.
Questions we try to settle using EFA
1
How many meaningful dimensions are present?
2
What is the structure of those dimensions?
Herman Adèr
Factor Analysis and SEM
8
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
F = M ∪ U ∪ E,
of which M indicate the meaningful factors, U so-called
‘unique’ factors (factors on which only one item loads) and E
error factors.
Questions we try to settle using EFA
1
How many meaningful dimensions are present?
2
What is the structure of those dimensions?
Herman Adèr
Factor Analysis and SEM
8
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Data set:
D=
n
Subject
Items
o
Exploratory Factor Analysis
unrotated ?
factor structure
Three steps
determine
meaningful factors
rotate k meaningful factors
1
Determine the meaningful factors
2
Rotate
3
Interpret the factor structure
?
factor structure
determine factor meaning
?
final factor solution
Herman Adèr
Factor Analysis and SEM
9
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
determine meaningful factors:
scree plot
eigenvalues larger than 1
GoF change
significant
between solutions doubtful total variance explained more than 50%
select factors with
intelligible
loading patterns
and name them communalities ≈ reliabilities Herman Adèr
Factor Analysis and SEM
10
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
CES-D: A multidimensional scale to assess Depression
FACTOR
/VARIABLES cesd1 cesd2 cesd3 cesd4 cesd5
cesd6 cesd7 cesd8 cesd9 cesd10
cesd11 cesd12 cesd13 cesd14 cesd15
cesd16 cesd17 cesd18 cesd19 cesd20
/MISSING LISTWISE
/ANALYSIS cesd1 cesd2 cesd3 cesd4 cesd5
cesd6 cesd7 cesd8 cesd9 cesd10
cesd11 cesd12 cesd13 cesd14 cesd15
cesd16 cesd17 cesd18 cesd19 cesd20
/PRINT all
/FORMAT BLANK(.30)
/PLOT EIGEN
/CRITERIA MINEIGEN(0) ITERATE(100) factors(3)
/EXTRACTION pc
/ROTATION norotate.
Herman Adèr
Factor Analysis and SEM
12
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
CES-D: A multidimensional scale to assess Depression
FACTOR
/VARIABLES cesd1 cesd2 cesd3 cesd4 cesd5
cesd6 cesd7 cesd8 cesd9 cesd10
cesd11 cesd12 cesd13 cesd14 cesd15
cesd16 cesd17 cesd18 cesd19 cesd20
/MISSING LISTWISE
/ANALYSIS cesd1 cesd2 cesd3 cesd4 cesd5
cesd6 cesd7 cesd8 cesd9 cesd10
cesd11 cesd12 cesd13 cesd14 cesd15
cesd16 cesd17 cesd18 cesd19 cesd20
/PRINT all
/FORMAT BLANK(.30)
/PLOT EIGEN
/CRITERIA MINEIGEN(0) ITERATE(100) factors(3)
/EXTRACTION pc
/ROTATION norotate.
Herman Adèr
Factor Analysis and SEM
12
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Scree Plot
10
8
6
Eigenvalue
4
2
0
1
3
2
5
4
7
6
9
8
11
10
13
12
15
14
17
16
19
18
20
Component Number
Herman Adèr
Factor Analysis and SEM
13
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Component
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Total
8.516
1.775
1.229
1.040
.944
.767
.738
.684
.606
.577
.508
.442
.426
.339
.318
.286
.274
.202
.178
.149
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Initial Eigenvalues
% of Variance
Cumulative %
42.578
42.578
8.876
51.454
6.147
57.600
5.201
62.802
4.719
67.520
3.836
71.356
3.689
75.045
3.419
78.465
3.030
81.495
2.886
84.381
2.542
86.923
2.211
89.134
2.129
91.263
1.697
92.960
1.592
94.552
1.429
95.981
1.371
97.352
1.010
98.362
.891
99.253
.747
100.000
Extraction Method: Principal Component Analysis.
Herman Adèr
Factor Analysis and SEM
14
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Interpretation of the unrotated factor structure
1
CESD1
CESD2
CESD3
CESD4
CESD5
CESD6
CESD7
CESD8
CESD9
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
2
.525
3
4
.372
.757
.802
-.573
.687
.689
.726
-.635
.721
.672
.623
-.690
.599
.710
.432
-.758
.657
.763
.482
.784
5
-.373
Component
6
-.318
.389
.538
.336
.349
.531
-.353
.398
.415
.489
-.513
.490
.383
-.627
-.447
.500
.349
-.376
.359
Extraction Method: Principal Component Analysis.
Herman Adèr
Factor Analysis and SEM
15
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Communalities: four factors and all factors
Communalities
CESD1
CESD2
CESD3
CESD4
CESD5
CESD6
CESD7
CESD8
CESD9
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Initial
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
Communalities
Extraction
.418
.751
.798
.683
.492
.526
.533
.763
.695
.543
.597
.734
.655
.546
.442
.721
.699
.709
.631
.624
CESD1
CESD2
CESD3
CESD4
CESD5
CESD6
CESD7
CESD8
CESD9
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Extraction Method: Principal Component Analysis.
Herman Adèr
Initial
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
Extraction
.966
.970
.868
.817
.919
.912
.830
.834
.812
.883
.896
.841
.861
.829
.961
.829
.828
.793
.963
.773
Extraction Method: Principal Component Analysis.
Factor Analysis and SEM
16
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Communalities: two and three factors
Communalities
CESD1
CESD2
CESD3
CESD4
CESD5
CESD6
CESD7
CESD8
CESD9
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Initial
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
Communalities
Extraction
.414
7.631E-02
.795
.617
.481
.489
.531
.686
.544
.539
.399
.716
.389
.540
.427
.721
.434
.587
.291
.615
CESD1
CESD2
CESD3
CESD4
CESD5
CESD6
CESD7
CESD8
CESD9
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Extraction Method: Principal Component Analysis.
Herman Adèr
Initial
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
Extraction
.418
.650
.796
.639
.491
.517
.532
.720
.669
.541
.571
.733
.392
.546
.437
.721
.449
.587
.490
.621
Extraction Method: Principal Component Analysis.
Factor Analysis and SEM
17
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Goodness-of-fit criterium
# Facts
1
2
3
4
5
GoF χ2
444.399
278.177
218.422
170.845
126.361
Df
170
151
133
116
100
χ2 change
Df
χ2α=.05
166.222
59.755
47.577
44.484
19
18
17
16
30.144
28.869
27.587
26.296
As one can see by comparing the 0.05 threshold χ2 in the last column
Herman Adèr
Factor Analysis and SEM
18
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Item
CESD01
CESD02
CESD03
CESD04
CESD05
CESD06
CESD07
CESD08
CESD09
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Unrotated
Factor
I
II
III
Rotated
Factor
I
II
III
+
+
+
+
+
−
+
+
+
−
+
+
+
−
+
+
+
−
+
+
+
+
+
+
+
+
+
+
+
+
−
+
+
+
+
+
+
+
Herman Adèr
−
+
+
+
−
+
+
−
+
+
+
+
+
+
+
−
+
+
+
Factor Analysis and SEM
19
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Interpretation of table I
Item
CESD01
CESD02
CESD03
CESD04
CESD05
CESD06
CESD07
CESD08
CESD09
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Unrotated
Factor
I
II III
+ +
+
+ +
− +
+
+
+
− +
+
−
+
+
+
− +
+
+
+ +
− +
+
+
+
+
Rotated
Factor
I
II III
+
+
+
−
+ +
+ +
+ +
−
+ +
+
+
+
−
+
+
+
−
+ +
+ +
+
+ +
Herman Adèr
All items with high positive loadings on
the unrotated Factor I still load on this
factor after rotation, but some of them
(5, 6, 7, 9, 17, 18 and 20) also load on
factor II. This makes that Factor I is
now separated into two sublists, of
which the above could be called the
pure depression scale while (1, 3, 10,
11, 13, 14, 15, 19) includes a sublist
indicating less severe symptoms of
depression.
Factor Analysis and SEM
20
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Interpretation of table II
Item
CESD01
CESD02
CESD03
CESD04
CESD05
CESD06
CESD07
CESD08
CESD09
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Unrotated
Factor
I
II III
+ +
+
+ +
− +
+
+
+
− +
+
−
+
+
+
− +
+
+
+ +
− +
+
+
+
+
Rotated
Factor
I
II III
+
+
+
−
+ +
+ +
+ +
−
+ +
+
+
+
−
+
+
+
−
+ +
+ +
+
+ +
Herman Adèr
Factor II now also contains all the
items that loaded negatively on Factor
I before (4, 8, 12 and 16). We called
this a factor that measures:
general attitude towards life.
Factor Analysis and SEM
21
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Interpretation of table III
Item
CESD01
CESD02
CESD03
CESD04
CESD05
CESD06
CESD07
CESD08
CESD09
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Unrotated
Factor
I
II III
+ +
+
+ +
− +
+
+
+
− +
+
−
+
+
+
− +
+
+
+ +
− +
+
+
+
+
Rotated
Factor
I
II III
+
+
+
−
+ +
+ +
+ +
−
+ +
+
+
+
−
+
+
+
−
+ +
+ +
+
+ +
Herman Adèr
After rotation, Factor III now contains
only item 2 (‘I did not feel like eating:
my appetite was poor’) and item 11
(‘My sleep was restless’), suggesting
that it is a factor that has to do with
physical aspects of depression.
Factor Analysis and SEM
22
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example of EQS input
Example of EQS input
Conclusions of EFA and CFA combined
Example of EQS input
/TITLE Cesd;
/SPECIFICATIONS
VARIABLES=6; CASES= 200;
DATAFILE=’cesd.ess’; MATRIX=RAW;
ME = ML;
/EQUATIONS
V1 = 2 * F1 + E1;
V2 = 2 * F1 + E2;
V3 = 2 * F2 + E3;
V4 = 2 * F2 + E4;
V5 = 2 * F3 + E5;
V6 = 2 * F3 + E6;
F1 = 2 * F2 + 2*F3+ D1;
/VARIANCES
E1 to E10 = 0.2*;
D1, D2, D3 = 0.2*;
/COVARIANCES
F2,F3 = .5*;
/END
Herman Adèr
Factor Analysis and SEM
23
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example of EQS input
Example of EQS input
Conclusions of EFA and CFA combined
CES-D: Confirmatory factor analysis
Conclusions of the CFA
The varimax solution is not fully confirmed: item 5, 6, 7, 18
and 20 have negligible coefficients on the second factor.
The assumption of a orthogonal factor structure is
unfounded: The correlations between the factors are
(F1, F2) = 0.241; (F1, F3) = 0.382 and (F2, F3) = −0.025.
Herman Adèr
Factor Analysis and SEM
24
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example of EQS input
Example of EQS input
Conclusions of EFA and CFA combined
CES-D: Confirmatory factor analysis
Conclusions of the CFA
The varimax solution is not fully confirmed: item 5, 6, 7, 18
and 20 have negligible coefficients on the second factor.
The assumption of a orthogonal factor structure is
unfounded: The correlations between the factors are
(F1, F2) = 0.241; (F1, F3) = 0.382 and (F2, F3) = −0.025.
Herman Adèr
Factor Analysis and SEM
24
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example of EQS input
Example of EQS input
Conclusions of EFA and CFA combined
Conclusions of EFA and CFA combined
1
The first factor is a general depression factor (item 1, 3, 5,
6, 7, 10, 13, 14, 15, 18, 19 and 20)
2
The second factor contains the positively formulated items
(4, 8, 9, 12, 16, 17). It represents a general attitude
towards life.
3
The third factor contains items that have to do with
physical aspects of depression.
Herman Adèr
Factor Analysis and SEM
25
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example: Post-traumatic stress disorder
Example of a postulated structure
Draw pictures!
Herman Adèr
Factor Analysis and SEM
26
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example: Post-traumatic stress disorder
Example of a postulated structure
Draw pictures!
Herman Adèr
Factor Analysis and SEM
26
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example: Post-traumatic stress disorder
Example of a postulated structure
Draw pictures!
Herman Adèr
Factor Analysis and SEM
26
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Postulated structure
Structure found
Herman Adèr
Factor Analysis and SEM
27
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Recommended literature
Factor analysis: Friendly (1995)
(http://www.psych.yorku.ca/lab/psy6140/fa/factorbi.htm)
Principal component analysis: Jackson (1991)
Structural equation modelling: Bollen (1989), Kaplan (2000)
Herman Adèr
Factor Analysis and SEM
28
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Summary
Herman Adèr
Factor Analysis and SEM
29
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Summary
In factor analysis the contrast between exploratory and
confirmatory approaches is quite clear: the available
techniques are different although related.
Exploratory factor analysis can well be performed with
principal component analysis, possibly combined with
Maximum likelihood factor analysis. PCA is first used to
determine the meaningful factors, MLFA for the final factor
structure.
Structural Equation Modelling makes it possible to analyze
a research problem that has been represented as a
diagram.
Herman Adèr
Factor Analysis and SEM
30
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Bollen, K. A. (1989). Structural equations with latent variables. New
York: John Wiley and Sons.
Friendly, M. (1995). Annotated Factor Analysis Bibliography.
Jackson, J. E. (1991). A user’s guide to principal components. New
York: Wiley.
Kaplan, D. (2000). Structural Equation Modeling. Foundations and
Extensions. Thousand Oaks London New Delhi: Sage
Publications.
Tukey, J. W. (1977). Exploratory data analysis. Reading, MA:
Addison Wesley.
Herman Adèr
Factor Analysis and SEM
31