JSS
Journal of Statistical Software
May 2012, Volume 48, Issue 4.
http://www.jstatsoft.org/
qgraph: Network Visualizations of Relationships in
Psychometric Data
Sacha Epskamp
Angélique O. J. Cramer
Lourens J. Waldorp
University of Amsterdam
University of Amsterdam
University of Amsterdam
Verena D. Schmittmann
Denny Borsboom
University of Amsterdam
University of Amsterdam
Abstract
We present the qgraph package for R, which provides an interface to visualize data
through network modeling techniques. For instance, a correlation matrix can be represented as a network in which each variable is a node and each correlation an edge; by
varying the width of the edges according to the magnitude of the correlation, the structure
of the correlation matrix can be visualized. A wide variety of matrices that are used in
statistics can be represented in this fashion, for example matrices that contain (implied)
covariances, factor loadings, regression parameters and p values. qgraph can also be used
as a psychometric tool, as it performs exploratory and confirmatory factor analysis, using
sem and lavaan; the output of these packages is automatically visualized in qgraph, which
may aid the interpretation of results. In this article, we introduce qgraph by applying the
package functions to data from the NEO-PI-R, a widely used personality questionnaire.
Keywords: R, networks, correlations, data visualization, factor analysis, graph theory.
1. Introduction
The human visual system is capable of processing highly dimensional information naturally.
For instance, we can immediately spot suggestive patterns in a scatterplot, while these same
patterns are invisible when the data is numerically represented in a matrix.
We present qgraph, a package for R (R Development Core Team 2012), available from the Comprehensive R Archive Network at http://CRAN.R-project.org/package=qgraph. qgraph
accommodates capacities for spotting patterns by visualizing data in a novel way: through
networks. Networks consist of nodes (also called ‘vertices’) that are connected by edges
2
qgraph: Network Visualizations of Relationships in Psychometric Data
(Harary 1969). Each edge has a certain weight, indicating the strength of the relevant connection, and in addition edges may or may not be directed. In most applications of network
modeling, nodes represent entities (e.g., people in social networks, or genes in gene networks).
However, in statistical analysis it is natural to represent variables as nodes. This representation has a longstanding tradition in econometrics and psychometrics (e.g., see Bollen and
Lennox 1991; Edwards and Bagozzi 2000), and was a driving force behind the development
of graphical models for causal analysis (Spirtes et al. 2000; Pearl 2000). By representing relationships between variables (e.g., correlations) as weighted edges important structures can be
detected that are hard to extract by other means. In general, qgraph enables the researcher
to represent complex statistical patterns in clear pictures, without the need for data reduction
methods.
qgraph was developed in the context of network approaches to psychometrics (Cramer et al.
2010; Borsboom 2008; Schmittmann et al. 2012), in which theoretical constructs in psychology
are hypothesized to be networks of causally coupled variables. In particular, qgraph automates
the production of graphs such as those proposed in Cramer et al. (2010). However, the
techniques in the package have also proved useful as a more general tool for visualizing data,
and include methods to visualize output from several psychometric packages like sem (Fox
2006; Fox et al. 2012) and lavaan (Rosseel 2012).
A number of R packages can be used for the visualization and analysis of networks, e.g.,
network (Butts et al. 2012; Butts 2008a), statnet (Handcock et al. 2008), igraph (Csardi and
Nepusz 2006). In visualizing graphs qgraph distinguishes itself by being specifically aimed at
the visualization of statistical information. This usually leads to a special type of graph: a
non-sparse weighted graph. Such graphs typically contain many edges (e.g., a fully connected
network with 50 nodes has 2450 edges) thereby making it hard to interpret the graph; as
well as inflating the file size of vector type image files (e.g., PDF, SVG, EPS). qgraph is
specifically aimed at presenting such graphs in a meaningful way (e.g., by using automatic
scaling of color and width, cutoff scores and ordered plotting of edges) and to minimize the
file size of the output in vector type image files (e.g., by minimizing the amount of polygons
needed). Furthermore, qgraph is designed to be usable by researchers new to R, while at the
same time offering more advanced customization options for experienced R users.
qgraph is not designed for numerical analysis of graphs (Boccaletti et al. 2006), but can be
used to compute the node centrality measures of weighted graphs proposed by Opsahl et al.
(2010). Other R packages as well as external software can be used for more detailed analyses.
qgraph facilitates these methods by using commonly used methods as input. In particular,
the input is the same as used in the igraph package for R, which can be used for many different
analyses. In this article we introduce qgraph using an included dataset on personality traits.
We describe how to visualize correlational structures, factor loadings and structural equation
models and how these visualizations should be interpreted. Finally we will show how qgraph
can be used as a simple unified interface to perform several exploratory and confirmatory
factor analysis routines available in R.
2. Creating graphs
Throughout this article we will be working with a dataset concerning the five factor model of
personality (Benet-Martinez and John 1998; Digman 1989; Goldberg 1990, 1998; McCrae and
Journal of Statistical Software
3
Costa 1997). This is a model in which correlations between responses to personality items
(i.e., questions of the type ‘do you like parties?’, ‘do you enjoy working hard?’) are explained
by individual differences in five personality traits: neuroticism, extraversion, agreeableness,
openness to experience and conscientiousness. These traits are also known as the ‘Big Five’.
We use an existing dataset in which the Dutch translation of a commonly used personality
test, the NEO-PI-R (Costa and McCrae 1992; Hoekstra et al. 2003), was administered to
500 first year psychology students (Dolan et al. 2009). The NEO-PI-R consists of 240 items
designed to measure the five personality factors with items that cover six facets per factor1 .
The scores of each subject on each item are included in qgraph, as well as information on the
factor each item is designed to measure (this information is in the column names). All graphs
in this paper were made using R version 2.14.1 (2011-12-22) and qgraph version 1.0.0.
First, we load qgraph and the NEO-PI-R dataset:
R> library("qgraph")
R> data("big5")
2.1. Input modes
The main function of qgraph is called qgraph(), and its first argument is used as input
for making the graph. This is the only mandatory argument and can either be a weights
matrix, an edge-list or an object of class "qgraph", "loadings" and "factanal" (stats; R
Development Core Team 2012), "principal" (psych; Revelle 2012), "sem" and "semmod"
(sem; Fox et al. 2012), "lavaan" (lavaan; Rosseel 2012), "graphNEL" (Rgraphviz; Gentry
et al. 2012) or "pcAlgo" (pcalg; Kalisch et al. 2012). In this article we focus mainly on
weights matrices, information on other input modes can be found in the documentation.
A weights matrix codes the connectivity structure between nodes in a network in matrix form.
For a graph with n nodes its weights matrix A is a square n by n matrix in which element aij
represents the strength of the connection, or weight, from node i to node j. Any value can
be used as weight as long as (a) the value zero represents the absence of a connection, and
(b) the strength of connections is symmetric around zero (so that equal positive and negative
values are comparable in strength). By default, if A is symmetric an undirected graph is
plotted and otherwise a directed graph is plotted. In the special case where all edge weights
are either 0 or 1 the weights matrix is interpreted as an adjacency matrix and an unweighted
graph is made.
For example, consider the following weights

0
0
0
matrix:

1 2
0 3
0 0
This matrix represents a graph with 3 nodes with weighted edges from node 1 to nodes 2 and
3, and from node 2 to node 3. The resulting graph is presented in Figure 1.
Many statistics follow these rules and can be used as edge weights (e.g., correlations, covariances, regression parameters, factor loadings, log odds). Weights matrices themselves also
1
A facet is a subdomain of the personality factor; e.g., the factor neuroticism has depression and anxiety
among its subdomains.
4
qgraph: Network Visualizations of Relationships in Psychometric Data
1
3
2
Figure 1: A directed graph based on a 3 by 3 weights matrix with three edges of different
strengths.
11
16
●●●●●●
●
21
●
26
●●
●
31
●
36
●●
●
41
●
●
46
●
●
51
●
●
56
●
●
61
●
●
66
●
●
71
●
●
76
●
●
81
●
●
86
●●
●
91
●
96
●●
●
●●●●●●●●
226
231 236
1
6
221
216
211
206
201
196
191
186
181
176
171
166
161
20
●●●●●●●
25
●
30
●●
●
35
●
●
40
●
●
45
●
●
50
●
●
55
●
●
60
●
●
65
●
●
70
●
●
75
●
●
80
●
●
85
●
●
90
●●
●
95
●
●●
●
●●●●●●●●
230
5 10 15
235 240
225
220
215
210
156
151
146
101
141
136
131 126
111
121 116
106
205
187
182
180
177
175
172
170
162
160
157
155
152
100
147
105
137
4 9 14
19
●●●●●●
●
24
●
29
●●
●
34
●
●
39
●
●
44
●
●
49
●
●
54
●
●
59
●
●
64
●
●
69
●
●
74
●
●
79
●
●
84
●
●
89
●●
●
94
●
99
●●
●
●●●●●●●●
234 239
224
219
214
209
204
199
194
189
184
179
174
169
164
159
154
149
104
144
139
134 129
114
124 119
●
109
●
●
102
142
110
229
●
167
165
135 130
115
125 120
207
197
190
140
232 237
192
185
145
227
222
217
202
200
195
150
2 7 12
●●
●●●
17
22
●●●
●
27
●
●
32
●
●
37
●
●
42
●
●
47
●
●
52
●
●
57
●
●
62
●
●
67
●
●
72
●
●
77
●
●
82
●
●
87
●●
●
92
●
97
●●
●
●●●●●●●●
212
132 127
112
122 117
107
●
Neuroticism
Extraversion
Openness
Agreeableness
Conscientiousness
3 8 13
18
●●●●●●
●
23
●
28
●●
●
33
●
●
38
●
●
43
●
●
48
●
●
53
●
●
58
●
●
63
●
●
68
●
●
73
●
●
78
●
●
83
●
●
88
●●
●
93
●
98
●●
●
●●●●●●●●
228
233 238
223
218
213
208
203
198
193
188
183
178
173
168
163
158
153
148
143
103
138
133 128
113
123 118
108
Figure 2: A visualization of the correlation matrix of the NEO-PI-R dataset. Each node
represents an item and each edge represents a correlation between two items. Green edges
indicate positive correlations, red edges indicate negative correlations, and the width and color
of the edges correspond to the absolute value of the correlations: the higher the correlation,
the thicker and more saturated is the edge.
Journal of Statistical Software
5
occur naturally (e.g., as a correlation matrix) or can easily be computed. Taking a correlation matrix as the argument of the function qgraph() is a good start to get acquainted with
the package.
With the NEO-PI-R dataset, the correlation matrix can be plotted with:
R> qgraph(cor(big5))
This returns the most basic graph, in which the nodes are placed in a circle. The edges between
nodes are colored according to the sign of the correlation (green for positive correlations, and
red for negative correlations), and the thickness of the edges represents the absolute magnitude
of the correlation (i.e., thicker edges represent higher correlations).
Visualizations that aid data interpretation (e.g., are items that supposedly measure the same
construct closely connected?) can be obtained either by using the groups argument, which
groups nodes according to a criterion (e.g., being in the same psychometric subtest) or by
using a layout that is sensitive to the correlation structure. First, the groups argument can be
used to specify which nodes belong together (e.g., are designed to measure the same factor).
Nodes belonging together have the same color, and are placed together in smaller circles. The
groups argument can be used in two ways. First, it can be a list in which each element is
a vector containing the numbers of nodes belonging together. Secondly, it can be a factor in
which the levels belong together. The names of the elements in the list or the levels in the
factor are used in a legend of requested.
For the Big 5 dataset, the grouping of the variables according to the NEO-PI-R manual is
included in the package. The result of using the groups argument is a network representation
that readily facilitates interpretation in terms of the five personality factors:
R> data("big5groups")
R> Q <- qgraph(cor(big5), groups = big5groups)
Note that we saved the graph in the object Q, to avoid specifying these arguments again in
future runs. It is easy to subsequently add other arguments: for instance, we may further
optimize the representation by using the minimum argument to omit correlations we are not
interested in (e.g., very weak correlations), borders to omit borders around the nodes, and
vsize to make the nodes smaller:
R> Q <- qgraph(Q, minimum = 0.25, borders = FALSE, vsize = 2)
The resulting graph is represented in Figure 2.
2.2. Layout modes
Instead of predefined circles (as was used in Figure 2), an alternative way of facilitating
interpretations of correlation matrices is to let the placement of the nodes be a function
of the pattern of correlations. Placement of the nodes can be controlled with the layout
argument. If layout is assigned "circular", then the nodes are placed clockwise in a circle,
or in smaller circles if the groups argument is specified (as in Figure 2). If the nodes are placed
such that the length of the edges depends on the strength of the edge weights (i.e., shorter
edges for stronger weights), then a picture can be generated that shows how variables cluster.
6
qgraph: Network Visualizations of Relationships in Psychometric Data
This is a powerful exploratory tool, that may be used as a visual analogue to factor analysis.
To make the length of edges directly correspond to the edge weights an high dimensional
space would be needed, but a good alternative is the use of force-embedded algorithms (Di
Battista et al. 1994) that iteratively compute the layout in two-dimensional space.
A modified version of a force-embedded algorithm that was proposed by Fruchterman and
Reingold (1991) is included in qgraph. This is a C function that was ported from the sna
package (Butts 2010, 2008b). A modification of this algorithm for weighted graphs was taken
from igraph (Csardi and Nepusz 2006). This algorithm uses an iterative process to compute
a layout in which the length of edges depends on the absolute weight of the edges. To use
the Fruchterman-Reingold algorithm in qgraph() the layout argument needs to be set to
"spring". We can do this for the NEO-PI-R dataset, using the graph object Q that we defined
earlier, and omitting the legend:
R> qgraph(Q, layout = "spring", legend = FALSE)
Figure 3 shows the correlation matrix of the Big Five dataset with the nodes placed according to the Fruchterman-Reingold algorithm. This allows us inspect the clustering of the
variables. The figure shows interesting structures that are far harder to detect with conventional analyses. For instance, neuroticism items (i.e., red nodes) cluster to a greater extent
when compared to other traits; especially openness is less strongly organized than the other
factors. In addition, agreeableness and extraversion items are literally intertwined, which
offers a suggestive way of thinking about the well known correlation between these traits.
The placement of the nodes can also be specified manually by assigning the layout argument
a matrix containing the coordinates of each node. For a graph of n nodes this would be a
n by 2 matrix in which the first column contains the x coordinates and the second column
contains the y coordinates. These coordinates can be on any scale and will be rescaled to
fit the graph by default. For example, the following matrix describes the coordinates of the
graph in Figure 1:


2 2
3 1
1 1
This method of specifying the layout of a graph is identical to the one used in the igraph
(Csardi and Nepusz 2006) package, and thus any layout obtained through igraph can be
used2 .
One might be interested in creating not one graph but an animation of several graphs that
are similar to each other. For example, to illustrate the growth of a network over time or
to investigate the change in correlational structure in repeated measures. For such similar
but not equal graphs the Fruchterman-Reingold algorithm might return completely different
layouts which will make the animation unnecessary hard to interpret. This problem can be
solved by limiting the amount of space a node may move in each iteration. The function
qgraph.animate() automates this process and can be used for various types of animations.
2
To do this, first create an "igraph" object by calling graph.adjacency() on the weights matrix with the
arguments weighted = TRUE. Then, use one of the layout functions (e.g., layout.spring()) on the "igraph"
object. This returns the matrix with coordinates wich can be used in qgraph().
Journal of Statistical Software
●
18
●
148
78
●
●
190
●
●
●
105
172
153
7
●
●
143
238
●
173
75
●
33
● ● ●
●
21
63
3
●
●
●
90
●
●
●
60
53
●
●
●
70
83
●
●
●
●
●
●
40
81 ●
●
●
82
● ●
●
●
93
23
●
●
●
38
●
●
●
● ●
●
●
46
●
●
22
●
● ● ●
●
98
5 ●
●
●
73
●
●
●
50
●
●
●
55
●
58
●
●
●
●
65
●
● ●
69
15
51
●
●
●
●
8
49
●
80
●
●
●
●
●
95
28
●
●
●
48
● ● 10
●
●
●
35
●
●
30
●
24
● ●
●
13
●
● ●
●
85
68
●
●
36
●
●
25
●
57
●
●
●
●
●
45
●
●
54
●
● ● ●
●
20
● ●
●
43
47
●
●
●
●
●
●
14
39
76
●
●
●
●
●
●
77
9
56
●
●
72
●
●
●
●● ●
84
● ● 66 ●
● 86 ● ● ●
●
●
●
●
●
44
●
● ●
● ●
● ●
●
26 ● ●
●
27
●
●
59
●
● ●
●
●
●
31
●
41
●
●
12
●
●
●
●
●
● ● ● ● ● ●
●
74
89
●
●
92
●
●
●
99
●
91
29
●
●● ●
●
●
62
● ●
6
1
71
●
●
●
4
19
● ●
●
16
61
11 ●
● ●
●
● ●
● ●
96
37
●
●
●
●
42
●
● 87
97
17
●
●
●
●
64
34
7
●
94
● ●
●●
32
●
●
●
●
79
67
●
●
●
●
2
●
120
111
100
218
197
228
213
113
88
●
183
165
240
239
108
112
220
230
201
171
233
180
123
168
200
150
203
125
235
128
195
141
110
202
158
210
188
215
223
145
167
118
138
225
170
232
229
198
101
207
115
208
160
135
140
205
163
199
132
175
185
157
103
142
133
159
119
206
231
192
102
130
176
116
104
227
155
178
117
189
137
236
219
164
181
187
107
146
161
191
106
204
182
234
126
151
149
217
216
226
209
224
222
211
196
129
193
144
221
114
134
177
136
194
152
156
124
122
127
174
212
121
147
184
169
179
237
214
154
186
131
162
109
166
52
●
●
139
Figure 3: A graph of the correlation matrix of the NEO-PI-R dataset in which the nodes are
placed by the Fruchterman-Reingold algorithm. The specification of the nodes and edges are
identical to Figure 2.
2.3. Output modes
To save the graphs, any output device in R can be used to obtain high resolution, publicationready image files. Some devices can be called directly by qgraph() through the filetype
argument, which must be assigned a string indicating what device should be used. Currently
filetype can be "R" or "x11"3 to open a new plot in R, raster types "tiff", "png" and
"jpg", vector types "eps", "pdf" and "svg" and "tex". A PDF file is advised, and this can
thus be created with qgraph(..., filetype = "pdf").
3
RStudio users are advised to use filetype = "x11" to plot in R.
8
qgraph: Network Visualizations of Relationships in Psychometric Data
Often, the number of nodes makes it potentially hard to track which variables are represented
by which nodes. To address this problem, one can define mouseover tooltips for each node,
so that the name of the corresponding variable (e.g., the item content in the Big Five graph)
is shown when one mouses over the relevant node. In qgraph, mouseover tooltips can be
placed on the nodes in two ways. The "svg" filetype creates a SVG image using the
RSVGTipsDevice package (Plate and Luciani 2011)4 . This filetype can be opened using most
browsers (best viewed in Firefox) and can be used to include mouseover tooltips on the node
labels. The tooltips argument can be given a vector containing the tooltip for each node.
Another option for mouseover tooltips is to use the "tex" filetype. This uses the tikzDevice
package (Sharpsteen and Bracken 2012) to create a .tex file that contains the graph5 , which
can then be built using pdfLATEX. The tooltips argument can also be used here to create
mouseover tool tips in a PDF file6 .
2.4. Standard visual parameters
In weighted graphs green edges indicate positive weights and red edges indicate negative
weights7 . The color saturation and the width of the edges corresponds to the absolute weight
and scale relative to the strongest weight in the graph (i.e., the edge with the highest absolute
weight will have full color saturation and be the widest). It is possible to control this behavior
by using the maximum argument: when maximum is set to a value above any absolute weight
in the graph then the color and width will scale to the value of maximum instead8 . Edges with
an absolute value under the minimum argument are omitted, which is useful to keep filesizes
from inflating in very large graphs.
In larger graphs the above edge settings can become hard to interpret. With the cut argument
a cutoff value can be set which splits scaling of color and width. This makes the graphs much
easier to interpret as you can see important edges and general trends in the same picture.
Edges with absolute weights under the cutoff score will have the smallest width and become
more colorful as they approach the cutoff score, and edges with absolute weights over the
cutoff score will be full red or green and become wider the stronger they are.
In addition to these standard arguments there are several arguments that can be used to
graphically enhance the graphs to, for example, change the size and shape of nodes, add a
background or venn diagram like overlay and visualize test scores of a subject on the graph.
The documentation of the qgraph() function has detailed instructions and examples on how
these can be used.
3. Visualizing statistics as graphs
3.1. Correlation matrices
In addition to the representations in Figures 2 and 3, qgraph offers various other possibilities
for visualizing association structures. If a correlation matrix is used as input, the graph
4
RSVGTipsDevice is only available for 32bit versions of R.
Note that this will load the tikzdevice package which upon loading checks for a LATEX compiler. If this is
not available the package might fail to load.
6
We would like to thank Charlie Sharpsteen for supplying the tikz codes for these tooltips.
7
The edge colors can currently not be changed except to grayscale colors using gray = TRUE.
8
This must be done to compare different graphs; a good value for correlations is 1.
5
Journal of Statistical Software
9
●
●
219
103
●
88
●
59
●
80
●
218
74
●
17
●
86
●
146
28
●
●
102
118
135
●
41
●
●
●
211
11
●
●
●
51
●
●
25
●
●
●
●
26
●
69
●
181
●
35
●
167
193
●
105
8
●
50
●
●
●
●
23
●
192
●
●
171
●
●
139
●
232
164
72
●
182
87
●
199
●
●
●
79
●
68
●
●
239
29
●
42
●
144
32
●
36
●
62
●
●
●
●
147
111
●
180
●
113
●
●
205
●
155
●
●
131
169
●
84
●
141
●
228
●
141
162
●
●
●
●
96
●
209
178
228
118
44
●
●
●
●
204
165
148
●
●
●
●
192
238
140
66
●
155
75
●
●
●
35
●
185
234
52
●
●
198
72
●
●
205
●
●
166
149
108
139
98
●
●
68
●
●
82
●
28
●
58
●
77
●
51
●
130
49
●
●
134
30
●
29
107
●
●
45
●
46
●
●
198
●
●
111
73
●
●
127
●
132
●
226
●
●
236
240
●
●
90
●
101
64
●
●
15
●
●
119
22
●
102
●
175
109
13
●
157
153
●
104
202
●
●
●
207
122
●
●
●
●
49
●
237
153
223
43
●
207
●
●
202
●
214
187
164
227
163
114
85
●
178
222
147
184
183
●
14
●
60
●
●
●
●
133
194
217
●
127
●
112
208
42
●
89
●
237
152
137
145
130
●
●
126
193
●
●
206
52
●
168
●
117
104
65
●
134
21
●
●
171
154
182
●
190
21
●
177
81
●
186
●
188
223
156
183
142
●
●
●
●
●
177
224
179
●
99
●
23
●
219
●
174
185
200
83
●
128
213
229
●
4
2
●
67
● ●
●
27
●
19
●
59
● ●
●
94
● ●
7
●
92
●
●
● ●
97
●
34
●
●
87
● ●
●
●
● 74
●
●
●
●
●
37
43
●
●
● ●
32
57
●
●
●
● ● ●
99
●
●
62
● ● ●
124
70
●
●
●
●
●
113
8
●
●
63
●
●
123
201
●
169
4
●
106
126
225
137
●
●
38
●
93
●
48
●
●
●
54
●
159
208
129
119
97
●
●
●
233
●
●
●
●
39
●
9
●
14
●
84
●
●
●
136
18
●
142
●
184
227
●
●
13
212
157
●
●
●
78
●
79
●
158
233
98
●
173
174
●
●
●
234
209
5
●
●
40
●
●
162
163
●
●
53
55
●
●
●
151
188
●
115
179
●
114
●
221
19
●
●
161
●
172
152
138
24
●
109
199
●
●
●
●
170
154
195
214
●
58
●
47
●
●
●
212
156
●
●
3
●
53
●
●
143
88
●
189
71
●
33
●
●
●
10
●
●
67
●
194
229
●
●
235
203
180
150
●
189
150
●
●
●
●
12
●
16
●
196
107
117
110
37
●
22
●
217
90
●
●
●
●
●
7
94
●
●
●
120
54
●
76
●
●
●
83
110
●
138
33
203
173
●
170
239
●
●
160
240
144
3
●
48
24
●
●
143
93
●
27
●
140
128
160
●
18
●
●
129
100
●
124
148
187
210
166
39
●
●
215
175
135
196
●
●
159
210
●
●
●
●
197
222
101
●
●
168
149
224
158
●
●
63
●
●
125
9
●
●
●
195
100
146
6
●
120
115
211
121
77
●
82
●
145
220
186
213
●
230
151
231
181
221
200
235
125
176
206
56
●
106
●
57
●
30
● ● ●
●
●
60
47
●
●
●
20
●
●
●
50
25
● ● ●
●
●
●
80
●
65
●
●
75
55
●
●
●
5
●
85
●
● ●
10
●
●
●
70
15
●
●
40
●
●
236
121
161
216
73
●
92
●
204
201
●
●
31
●
●
112
●
●
●
●
191
20
●
230
●
66
●
●
●
89
●
56
45
1
●
●
96
●
172
91
●
91
●
136
220
95
●
216
11
●
191
123
46
●
●
238
●
44
●
●
●
●
●
226
38
●
● 1
●
●
16
●
●
●
●
26
●
17
●
●
31
●
●
●
●
●
71
●
●
●
●
●
76
●
●
36
●
41
61
●
●
●
103
165
●
86
●
116
●
61
●
●
6
131
78
●
64
●
190
●
34
●
●
122
116
●
176
2
●
●
215
●
133
●
●
132
69
●
●
●
105
231
12
●
●
167
108
●
197
●
●
225
95
●
218
●
81
●
232
Figure 4: Additional visualizations based on the correlations of the NEO-PI-R dataset. Panel
(a) shows a concentration graph with partial correlations and panel (b) shows a graph in
which connections are based on an exploratory factor analysis.
argument of qgraph() can be used to indicate what type of graph should be made. By
default this is "association" in which correlations are used as edge weights (as in Figures 2
and 3).
Another option is to assign "concentration" to graph, which will create a graph in which
each edge represents the partial correlation between two nodes: partialling out all other variables. For normally distributed continuous variables, the partial correlation can be obtained
from the inverse of the correlation (or covariance) matrix. If P is the inverse of the correlation
matrix, then the partial correlation ωij of variables i and j is given by:
−pij
ωij = √
pii pjj
Strong edges in a resulting concentration graph indicate correlations between variables that
cannot be explained by other variables in the network, and are therefore indicative of causal
relationships (e.g., a real relationship between smoking and lung cancer that cannot be explained by other factors, for example gender), provided that all relevant variables are included
in the network.
The left panel of Figure 4 shows a concentration graph for the NEO-PI-R dataset. This graph
shows quite a few partial correlations above 0.3. Within the factor model, these typically
indicate violations of local independence and/or simple structure.
A third option for the graph argument is "factorial", which creates an unweighted graph
based on an exploratory factor analysis on the correlation (or covariance) matrix with promax
rotation (using factanal() from stats). By default the number of factors that are extracted
in the analysis is equal to the number of eigenvalues over 1 or the number of groups in the
groups argument (if supplied). Variables are connected with an edge if they both load higher
on the same factor than the cutoff score that is set with the cut argument. As such, the
10
qgraph: Network Visualizations of Relationships in Psychometric Data
"factorial" option can be used to investigate the structure as detected with exploratory
factor analysis.
The right panel of Figure 4 shows the factorial graph of the NEO-PI-R dataset. This graph
shows five clusters, as expected, but also displays some overlap between the extraversion and
agreeableness items.
qgraph has two functions that are designed to make all of the above graphs in a single run. The
first option is to use the qgraph.panel() function to create a four-panel plot that contains
the association graph with circular and spring layouts, a concentration graph with the spring
layout, and a factorial graph with the spring layout. We can apply this function to the Big
Five data to obtain all graphs at once:
R> qgraph.panel(cor(big5), groups = big5groups, minimum = 0.25,
+
borders = FALSE, vsize = 1, cut = 0.3)
A second option to represent multiple graphs at once is to use the qgraph.svg() function
to produce an interactive graph. This function uses RSVGTipsDevice (only available for
32bit versions of R; Plate and Luciani 2011) to create a host of SVG files for all three types
of graphs, using circular and spring layouts and different cutoff scores. These files contain
hyperlinks to each other (which can also be used to show the current graph in the layout of
another graph) and can contain mouseover tool tips as well. This can be a useful interface
to quickly explore the data. A function that does the same in tex format will be included in
a later version of qgraph, which can then be used to create a multi-page PDF file containing
the same graphs as qgraph.panel().
Matrices that are similar to correlation matrices, like covariance matrices and lag-1 correlations in time series, can also be represented in qgraph. If the matrix is not symmetric (as is
for instance the case for lag-1 correlations) then a directed graph is produced. If the matrix
has values on the diagonal (e.g., a covariance matrix) these will be omitted by default. To
show the diagonal values the diag argument can be used. This can be set to TRUE to include
edges from and to the same node, or "col" to color the nodes according to the strength of
diagonal entries. Note that it is advisable to only use standardized statistics (e.g., correlations
instead of covariances) because otherwise the graphs can become hard to interpret.
3.2. Significance
Often a researcher is interested in the significance of a statistic (p value) rather than the
value of the statistic itself. Due to the strict cutoff nature of significance levels, the usual
representation then might not be adequate because small differences (e.g., the difference
between edges based on t statistics of 1.9 and 2) are hard to see.
In qgraph statistical significance can be visualized with a weights matrix that contains p values
and assigning "sig" to the mode argument. Because these values are structurally different
from the edge weights we have used so far, they are first transformed according to the following
function:
0.4
wi = 0.7(1 − pi )log0.95 0.7
where wi is the edge weight of edge i and pi the corresponding significance level. The resulting graph shows different levels of significance in different shades of blue, and omits any
Journal of Statistical Software
11
insignificant value. The levels of significance can be specified with the alpha argument. For
a black and white representations, the gray argument can be set to TRUE.
For correlation matrices the fdrtool package (Strimmer 2012) can be used to compute p values
of a given set of correlations. Using a correlation matrix as input the graph argument should
be set to "sig", in which case the p values are computed and a graph is created as if mode
= "sig" was used. For the Big 5 data, a significance graph can be produced through the
following code:
R> qgraph(cor(big5), groups = big5groups, vsize = 2, graph = "sig",
+
alpha = c(1e-04, 0.001, 0.01))
3.3. Factor loadings
A factor-loadings matrix contains the loadings of each item on a set of factors obtained through
factor analysis. Typical ways of visualizing such a matrix is to boldface factor loadings that
exceed, or omit factor loadings below, a given cutoff score. With such a method smaller,
but interesting, loadings might easily be overlooked. In qgraph, factor-loading matrices can
be visualized in a similar way as correlation matrices: by using the factor loadings as edge
weights in a network. The function for this is qgraph.loadings() which uses the factorloadings matrix to create a weights matrix and a proper layout and sends that information
to qgraph().
There are two functions in qgraph that perform an exploratory analysis based on a supplied
correlation (or covariance) matrix and send the results to qgraph.loadings(). The first
is qgraph.efa() which performs an exploratory factor analysis (EFA; Stevens 1996) using
factanal() (stats; R Development Core Team 2012). This function requires three arguments
plus any additional argument that will be sent to qgraph.loadings() and qgraph(). The
first argument must be a correlation or covariance matrix, the second the number of factors
to be extracted and the third the desired rotation method.
To perform an EFA on the Big 5 dataset we can use the following code:
R> qgraph.efa(big5, 5, groups = big5groups, rotation = "promax",
+
minimum = 0.2, cut = 0.4, vsize = c(1, 15), borders = FALSE,
+
asize = 0.07, esize = 4, vTrans = 200)
Note that we supplied the groups list and that we specified a promax rotation allowing the
factors to be correlated.
The resulting graph is shown in the left panel of Figure 5. The factors are placed in an
inner circle with the variables in an outer circle around the factors9 . The factors are placed
clockwise in the same order as the columns in the loadings matrix, and the variables are
placed near the factor they load the highest on. Because an EFA is a reflective measurement
model, the arrows point towards the variables and the graph has residuals (Bollen and Lennox
1991; Edwards and Bagozzi 2000).
The left panel of Figure 5 shows that the Big 5 dataset roughly conforms to the 5 factor model.
That is, most variables in each group of items tend to load on the same factor. However, we
9
For a more traditional layout we could set layout = "tree".
12
qgraph: Network Visualizations of Relationships in Psychometric Data
188
193
198
203
208
213
218
223
228
233
238
1
6
11 16 21
26 31
36 41
46
51
183
56
178
61
173
66
168
178
183
188
193
198
203
208
213
218
223
228
233
238
1
6
11 16 21
26 31
36 41
46 51
71
76
163
81
158
86
153
56
91
96
148
61
173
66
168
143
71
163
101
138
76
158
81
106
133
86
153
111
128
91
148
116
123
96
143
121
118
101
138
133
126
113
106
111
128
131
108
116
123
136
103
141
121
118
98
126
113
146
93
131
108
136
103
151
88
141
98
156
83
146
93
88
161
78
151
166
73
156
83
78
171
68
161
166
73
176
63
171
68
181
58
176
63
186
53
191
181
58
48
186
53
206
33
Neuroticism
216
Openness
23
216
Openness
18
211
28
211
28
23
206
33
201
38
201
38
196
43
196
43
191
48
18
Neuroticism
221
226
221
13
231
226
13
8
231
8
2
7
239
2
7
239
22
224
22
224
27
219
27
219
17
229
17
229
12
234
12
234
236
3
236
3
32
214
32
214
37
209
37
209
204
42
199
47
204
42
199
47
194
52
194
189
57
189
184
62
184
179
67
179
174
72
174
169
77
82
164
52
57
62
67
72
169
77
164
82
87
159
Agreeableness
154
87
159
Extraversion
92
97
149
102
144
139
107
134
112
129
117
124
122
119
127
Agreeableness
154
Extraversion
92
97
149
102
144
139
107
134
112
129
117
124
122
132
114
119
109
127
137
132
114
104
142
109
99
137
147
94
152
89
157
Conscientiousness
84
162
79
167
74
172
69
177
64
104
142
99
147
94
152
89
157
Conscientiousness
84
162
79
167
182
59
187
54
192
49
197
44
202
39
207
34
212
29
74
172
69
177
64
182
59
187
54
192
49
197
217
24
222
19
227
14
232
9
237
5
4
10
240
15
235
20
230
25
225
220
215
210
205
200
195
190
185
180
175
170
165
160
155
150
145
140
135
130
125
120
115
110
105
100
75
85 80
95 90
70
65
60
55
50
45
40
35
30
44
202
39
207
34
212
29
217
24
222
19
227
14
232
9
237
5
4
10
240
15
235
20
230
25
225
30
220
35
215
210
205
200
195
190
185
180
175
170
165
160
155
150
145
140
135
130
125
120
115
110
105
100
85 80
95 90
75
70
65
60
55
50
45
40
Figure 5: Visualization of an exploratory factor analysis (a) and a principal component analysis (b) in the NEO-PI-R dataset.
also see many crossloadings, indicating departures from simple structure. Neuroticism seems
to be a strong factor, and most crossloadings are between extraversion and agreeableness.
The second function that performs an analysis and sends the results to qgraph.loadings()
is qgraph.pca(). This function performs a principal component analysis (PCA; Jolliffe 2002)
using princomp() of the psych package (Revelle 2012). A PCA differs from an EFA in that it
uses a formative measurement model (i.e., it does not assume a common cause structure). It
is used in the same way as qgraph.efa(); we can perform a PCA on the big 5 dataset using
the same code as with the EFA:
R> qgraph.pca(big5, 5, groups = big5groups, rotation = "promax",
+
minimum = 0.2, cut = 0.4, vsize = c(1, 15), borders = FALSE,
+
asize = 0.07, esize = 4, vTrans = 200)
The right panel of Figure 5 shows the results. Notice that the arrows now point towards the
factors, and that there are no residuals, as is the case in a formative measurement model10 .
Note that the correlations between items, which are not modeled in a formative model, are
omitted from the graphical representation to avoid clutter.
3.4. Confirmatory factor analysis
Confirmatory factor models and regression models involving latent variables can be tested
using structural equation modeling (SEM; Bollen 1989; Pearl 2000). SEM can be executed
in R with three packages: sem (Fox 2006), OpenMx (Boker et al. 2011) and lavaan (Rosseel
2012). qgraph currently supports sem and lavaan, with support for OpenMx expected in
10
In qgraph.loadings there are no arrows by default, but these can be set by setting the model argument
to "reflective" or "formative".
Journal of Statistical Software
13
a future version. The output of sem (a "sem" object) can be sent to (1) qgraph() for a
representation of the standardized parameter estimates, (2) qgraph.semModel() for a path
diagram of the specified model, and (3) qgraph.sem() for a 12 page PDF containing fit indices
and several graphical representations of the model, including path diagrams and comparisons
of implied and observed correlations. Similarly, the output of lavaan (a "lavaan" object) can
be sent to qgraph() or qgraph.lavaan().
SEM is often used to perform a confirmatory factory analysis (CFA; Stevens 1996) in which
variables each load on only one of several correlated factors. Often this model is identified
by either fixing the first factor loading of each factor to 1, or by fixing the variance of each
factor to 1. Because this model is so common, it should not be necessary to fully specify this
model for each and every run. However, this is currently still the case. Especially using sem
the model specification can be quite long.
The qgraph.cfa() function can be used to generate a CFA model for either the sem package
or the lavaan package and return the output of these packages for further inspection. This
function uses the groups argument as a measurement model and performs a CFA accordingly.
The results can be sent to another function, and are also returned. This is either a "sem"
or "lavaan" object which can be sent to qgraph(), qgraph.sem(), qgraph.lavaan() or any
function that can handle the object. We can perform the CFA on our dataset using lavaan
with the following code:
R> fit <- qgraph.cfa(cov(big5), N = nrow(big5), groups = big5groups,
+
pkg = "lavaan", opts = list(se = "none"), fun = print)
lavaan (0.4-11) converged normally after 128 iterations
Number of observations
Estimator
Minimum Function Chi-square
Degrees of freedom
P-value
500
ML
60838.192
28430
0.000
Note that we did not estimate standard errors to save some computing time. We can send
the results of this to qgraph.lavaan to get an output document.
Figure 6 shows part of this output: a visualization of the standardized parameter estimates.
We see that the first loading of each factor is fixed to 1 (dashed lines) and that the factors are correlated (bidirectional arrows between the factors). This is the default setup of
qgraph.cfa() for both sem and lavaan11 . From the output above, we see that this model
does not fit very well, and inspection of another part of the output document shows why this
is so: Figure 7 shows a comparison of the correlations that are implied by the CFA model
and the observed correlations, which indicates the source of the misfit. The model fails to
explain the high correlations between items that load on different factors; this is especially
true for extraversion and agreeableness items. The overlap between these items was already
11
Using lavaan allows to easily change some options by passing arguments to cfa() using the opts argument.
For example, we could fix the variance of the factors to 1 by specifying qgraph.cfa(..., opts = list(std.lv
= TRUE)).
14
qgraph: Network Visualizations of Relationships in Psychometric Data
C190
C195
C200
C205
C210
C235 C240
C225 C230
C215 C220
N1 N6
N11
N16
N21
N26
N31
N36
N41
N46
N51
N56
N61
C185
N66
C180
C175
N71
N76
C170
N81
C165
N86
C160
N91
C155
N96
C150
N101
C145
N106
C140
C135
N111
N116
C130
C125
N121
N126
C120
N131
C115
N136
C110
C105
N141
N146
C100
N151
C95
N156
C90
N161
C85
N166
C80
C75
N171
N176
C70
N181
C65
N186
C60
N191
C55
N196
C50
N201
C45
N206
C40
C35
N211
C30
C
C25
C20
N216
N
N221
N226
N231
C15
N236
C10
C5
E2
E7
A239
A234
E12
A229
E17
A224
E22
A219
E27
E32
A214
A209
E37
A204
E42
A199
E47
A194
E52
A189
E57
A184
E62
A179
E67
A174
E72
A169
E77
A164
E82
A159
A
A154
A149
E87
E
E92
E97
E102
A144
A139
E107
A134
E112
A129
E117
A124
E122
A119
E127
E132
A114
A109
E137
A104
E142
A99
E147
A94
E152
O
A89
A84
A79
E157
E162
E167
A74
E172
A69
E177
A64
E182
A59
E187
A54
E192
A49
E197
E202
A44
A39
E207
A34
E212
A29
E217
A24
E222
A19
E227
E232
A14
A9
E237
A4
O3
O8
O238
O233
O13
O228
O18
O223
O23
O218
O28
O213
O33
O208
O38
O203
O43
O198
O48
O193
O53
O188
O58
O183
O63
O178
O173
O168
O163
O158
O153 O148
O143 O138
O103
O133 O128 O123 O118 O113 O108
O98
O93
O88
O83
O78
O73
O68
Figure 6: Standardized parameter estimations of a confirmatory factor analysis performed on
the NEO-PI-R dataset.
evident in the previous figures, and this result shows that this overlap cannot be explained
by correlations among the latent factors in the current model.
4. Concluding comments
The network approach offers novel opportunities for the visualization and analysis of vast
datasets in virtually all realms of science. The qgraph package exploits these opportunities
by representing the results of well-known statistical models graphically, and by applying
network analysis techniques to high-dimensional variable spaces. In doing so, qgraph enables
researchers to approach their data from a new perspective.
qgraph is optimized to accommodate both unexperienced and experienced R users: The former
can get a long way by simply applying the command qgraph.panel() to a correlation matrix
of their data, while the latter may utilize the more complex functions in qgraph to represent
Journal of Statistical Software
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●●●●
45
46
47 48
1
2
3
4
8
13
14
35
193
240
194
198
199
28
23
27 26
25 24
22
93
204
229
205
228
220
218
216
217
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●●●●
192
145
146
147
190
148
189
149
188
150
187
151
186
152
185
153
184
154
183
155
182
156
181
157
180
158
179
159
178
160
161
177
176
162
175
163
164
174
173
165
166
172
171
170
169
168
60
85
61
62
63
64
65
66
67
68
77
215
191
86
78
214
219
59
79
212
213
221
58
80
211
222
57
81
210
223
56
82
209
224
55
83
208
225
54
84
206
226
53
87
207
227
52
88
203
230
49 50
95 96
51
89
202
231
94
92
91
90
201
232
167
10
11
12
13
14
15
16
17
32
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●●●●
21
200
233
9
33
20
29
197
8
34
19
30
196
7
35
18
31
195
6
36
17
32
239
238
237
236
5
37
16
33
4
38
15
34
3
39
12
36
2
40
11
37
1
41
10
38
47 48
42
9
39
46
44
43
7
40
235
45
6
41
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●●●●
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●●●●
5
44
43
42
234
15
76
75 74
71
73 72
70
69
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●●●●
239
143
144
142
97 98
99
233
232
231
230
204
229
205
228
132
110
131
191
122
121
120
150
151
152
153
184
154
155
182
156
181
157
180
158
179
159
178
160
161
177
176
162
175
163
164
174
173
117
123
149
183
118
124
148
185
116
125
147
190
114
126
165
166
172
119
171
170
169
168
66
67
68
77
189
115
127
146
61
62
65
78
188
113
128
145
60
63
64
79
187
112
129
192
59
80
215
186
111
130
216
217
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●●●●
107
109
218
58
81
214
219
57
82
212
220
56
83
213
221
55
84
211
222
54
85
210
223
53
86
209
224
52
87
208
225
49 50
95 96
51
88
206
226
94
92
91
90
89
207
227
106
133
93
21
203
105
108
22
202
104
134
23
27 26
25 24
201
103
136
28
200
102
135
20
29
199
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●●●●
19
30
198
101
137
18
31
195
197
100
141
140
139
194
196
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●●●●
138
193
240
238
237
236
235
234
167
76
75 74
71
73 72
70
69
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●●●●
143
144
142
97 98
99
100
101
141
140
102
139
103
138
104
137
105
136
106
135
107
134
108
133
109
132
110
131
111
130
112
129
113
128
114
115
127
126
116
125
117
118
124
123
122
121
120
119
Figure 7: The observed correlations in the NEO-PI-R dataset (left) and the correlations that
are implied by the model of Figure 6 (right).
the results of time series modeling. Overall, however, the package is quite accessible and
works with carefully chosen defaults, so that it almost always produces reasonable graphs.
Hopefully, this will allow the network approach to become a valuable tool in data visualization
and analysis.
Since qgraph is developed in a psychometric context, its applications are most suitable for
this particular field. In this article we have seen that qgraph can be used to explore several
thousands of correlations with only a few lines of code. This resulted in figures that not only
showed the structure of these correlations but also suggested where exactly the five factor
model did not fit the data. Another example is the manual of a test battery for intelligence
(IST; Liepmann et al. 2010) in which such graphs were used to argue for the validity of
the test. Instead of examining full datasets qgraph can also be used to check for statistical
assumptions. For example, these methods can be used to examine multicollinearity in a set
of predictors or the local independence of the items of a subtest.
Clearly, we are only beginning to scratch the surface of what is possible in the use of networks
for analyzing data, and the coming years will see considerable developments in this area.
Especially in psychometrics, there are ample possibilities for using network concepts (such
as centrality, clustering, and path lengths) to gain insight in the functioning of items in
psychological tests.
References
Benet-Martinez V, John OP (1998). “Los Cinco Grandes across Cultures and Ethnic Groups:
Multitrait Multimethod Analyses of the Big Five in Spanish and English.” Journal of
Personality and Social Psychology, 75, 729–750.
16
qgraph: Network Visualizations of Relationships in Psychometric Data
Boccaletti S, Latora V, Moreno Y, Chavez M, Hwang DU (2006). “Complex Networks: Structure and Dynamics.” Physics Reports, 424(4-5), 175–308.
Boker S, Neale M, Maes H, Wilde M, Spiegel M, Brick T, Spies J, Estabrook R, Kenny S,
Bates T, Mehta P, Fox J (2011). “OpenMx: An Open Source Extended Structural Equation
Modeling Framework.” Psychometrika, 76(2), 306–317.
Bollen K, Lennox R (1991). “Conventional Wisdom on Measurement: A Structural Equation
Perspective.” Psychological Bulletin, 110(2), 305–314.
Bollen KA (1989). Structural Equations with Latent Variables. John Wiley & Sons, New
York.
Borsboom D (2008). “Psychometric Perspectives on Diagnostic Systems.” Journal of Clinical
Psychology, 64(9), 1089–1108.
Butts CT (2008a). “network: A Package for Managing Relational Data in R.” Journal of
Statistical Software, 24(2), 1–36. URL http://www.jstatsoft.org/v24/i02/.
Butts CT (2008b). “Social Network Analysis with sna.” Journal of Statistical Software, 24(6),
1–51. URL http://www.jstatsoft.org/v24/i06/.
Butts CT (2010). sna: Tools for Social Network Analysis. R package version 2.2-0, URL
http://CRAN.R-project.org/package=sna.
Butts CT, Handcock MS, Hunter DR (2012). network: Classes for Relational Data. R package version 1.7-1, URL http://CRAN.R-project.org/package=network.
Costa PT, McCrae RR (1992). Neo Personality Inventory Revised (NEO PI-R). Psychological
Assessment Resources.
Cramer AOJ, Waldorp LJ, van der Maas HLJ, Borsboom D (2010). “Comorbidity: A Network
Perspective.” Behavioral and Brain Sciences, 33(2-3), 137–150.
Csardi G, Nepusz T (2006). “The igraph Software Package for Complex Network Research.”
InterJournal, Complex Systems, 1695. URL http://igraph.sf.net/.
Di Battista G, Eades P, Tamassia R, Tollis IG (1994). “Algorithms for Drawing Graphs:
An Annotated Bibliography.” Computational Geometry – Theory and Application, 4(5),
235–282.
Digman JM (1989). “Five Robust Trait Dimensions: Development, Stability, and Utility.”
Journal of Personality, 57(2), 195–214.
Dolan CV, Oort FJ, Stoel RD, Wicherts JM (2009). “Testing Measurement Invariance in
the Target Rotated Multigroup Exploratory Factor Model.” Structural Equation Modeling,
16(2), 20.
Edwards JR, Bagozzi RP (2000). “On the Nature and Direction of Relationships between
Constructs and Measures.” Psychological Methods, 5(2), 155–174.
Fox J (2006). “Structural Equation Modeling with the sem Package in R.” Structural Equation
Modeling, 13(3), 465–486.
Journal of Statistical Software
17
Fox J, Nie Z, Byrnes J (2012). sem: Structural Equation Models. R package version 3.0-0,
URL http://CRAN.R-project.org/package=sem.
Fruchterman TMJ, Reingold EM (1991). “Graph Drawing by Force-Directed Placement.”
Software: Practice and Experience, 21(11), 1129–1164.
Gentry J, Long L, Gentleman R, Falcon S, Hahne F, Sarkar D, Hansen K (2012). Rgraphviz:
Provides Plotting Capabilities for R Graph Objects. R package version 1.34.0, URL http:
//www.bioconductor.org/packages/release/bioc/html/Rgraphviz.html.
Goldberg LR (1990). “An Alternative “Description of Personality”: The Big-Five Factor
Structure.” Journal of Personality and Social Psychology, 59(6), 1216–1229.
Goldberg LR (1998). “The Structure of Phenotypic Personality Traits.” Personality: Critical
Concepts in Psychology, 48, 351.
Handcock MS, Hunter DR, Butts CT, Goodreau SM, Morris M (2008). “statnet: Software
Tools for the Representation, Visualization, Analysis and Simulation of Network Data.”
Journal of Statistical Software, 24(1), 1–11. URL http://www.jstatsoft.org/v24/i01/.
Harary F (1969). Graph Theory. Addison-Wesley, Reading.
Hoekstra HA, de Fruyt F, Ormel J (2003). “Neo-Persoonlijkheidsvragenlijsten: NEO-PI-R,
NEO-FFI [Neo Personality Questionnaires: NEO-PI-R, NEO-FFI].”
Jolliffe I (2002). Principal Component Analysis. John Wiley & Sons.
Kalisch M, Mächler M, Colombo D, Maathuis MH, Bühlmann P (2012). “Causal Inference
Using Graphical Models with the R Package pcalg.” Journal of Statistical Software, 47(11),
1–26. URL http://www.jstatsoft.org/v47/i11/.
Liepmann D, Beauducel A, Brocke B, Amthauer R (2010). Intelligentie Structuur Test.
Hogrefe Uitgevers B.V. Dutch Translation by Vorst HCM.
McCrae RR, Costa PT (1997). “Personality Trait Structure as a Human Universal.” American
Psychologist, 52(5), 509–516.
Opsahl T, Agneessens F, Skvoretz J (2010). “Node Centrality in Weighted Networks: Generalizing Degree and Shortest Paths.” Social Networks, 32(3), 245–251.
Pearl J (2000). Causality: Models, Reasoning, and Inference. Cambridge University Press.
Plate T, Luciani TJ (2011). RSVGTipsDevice: An R SVG Graphics Device with Dynamic Tips and Hyperlinks. R package version 1.0-4, URL http://CRAN.R-project.org/
package=RSVGTipsDevice.
R Development Core Team (2012). R: A Language and Environment for Statistical Computing.
R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http:
//www.R-project.org/.
Revelle W (2012). psych: Procedures for Psychological, Psychometric, and Personality
Research. Northwestern University, Evanston, Illinois. R package version 1.2.1, URL
http://personality-project.org/r/psych.manual.pdf.
18
qgraph: Network Visualizations of Relationships in Psychometric Data
Rosseel Y (2012). “lavaan: An R Package for Structural Equation Modeling.” Journal of
Statistical Software, 48(2), 1–36. URL http://www.jstatsoft.org/v48/i02/.
Schmittmann VD, Cramer AOJ, Waldorp LJ, Epskamp S, Kievit RA, Borsboom D (2012).
“Deconstructing the Construct: A Network Perspective on Psychological Phenomena.” New
Ideas in Psychology. doi:10.1016/j.newideapsych.2011.02.007.
Sharpsteen C, Bracken C (2012). tikzDevice: A Device for R Graphics Output in
PGF/TikZ Format. R package version 0.6.2, URL http://CRAN.R-project.org/package=
tikzDevice.
Spirtes P, Glymour CN, Scheines R (2000). Causation, Prediction, and Search. MIT Press.
Stevens J (1996). Applied Multivariate Statistics for the Social Sciences. Lawrence Erlbaum
Associates.
Strimmer K (2012). fdrtool: Estimation and Control of (Local) False Discovery Rates. R
package version 1.2.8, URL http://CRAN.R-project.org/package=fdrtool.
Affiliation:
Sacha Epskamp
University of Amsterdam
Department of Psychological Methods
Weesperplein 4
1018 XA Amsterdam, The Netherlands
E-mail: [email protected]
URL: http://www.sachaepskamp.com/
Journal of Statistical Software
published by the American Statistical Association
Volume 48, Issue 4
May 2012
http://www.jstatsoft.org/
http://www.amstat.org/
Submitted: 2011-06-16
Accepted: 2012-01-18