Layout Effects: Comparison of
Sociogram Drawing Conventions
Technical Report Number 575
October 2005
Weidong Huang, Seok-Hee Hong, Peter Eades
ISBN 1 86487 771 5
School of Information Technologies
University of Sydney NSW 2006
Layout Effects: Comparison of Sociogram Drawing Conventions
Weidong Huang, Seok-Hee Hong, Peter Eades
School of Information Technologies, University of Sydney, Australia
IMAGEN Program, National ICT Australia
{whua5569, shhong, peter @it.usyd.edu.au}
Abstract: This report describes a within-subjects experiment in which we compare the relative effectiveness of
five sociogram drawing conventions in communicating underlying network substance, based on user task
performance and usability preference, in order to examine effects of different spatial layout formats on human
sociogram perception. We also explore the impact of edge crossings, a widely accepted readability aesthetic.
Subjective data were gathered based on the methodology of Purchase et al. [2002]. Objective data were collected
through an online system.
We found that both edge crossings and drawing conventions pose significant affects on user preference and task
performance of finding groups, but either has little impact on the perception of actor status. On the other hand, the
node positioning and angular resolution might be more important in perceiving actor status. In visualising social
networks, it is important to note that the techniques that are highly preferred by users do not necessarily lead to
best task performance.
Introduction
A social network [Wikipedia 2005] is a collection of actors (such as people, organizations or other
social entities) and relationships among the actors, indicating the way in which they are connected
socially (such as friendship, trade or information exchange). Social network analysis is the mapping
and measuring of these relationships. This has long been an important technique as well as a popular
topic in many academic fields such as Sociology, Social Psychology, and more recently Information
Visualization.
Social networks can be modelled as graphs, and visualized as node-edge diagrams where nodes
represent actors, and edges represent relationships between them. With advances in display media, the
use of node-edge diagrams or sociograms (see Figure 1 for an example) has been increasingly
important and popular in social network analysis. Sociograms serve as simple visual illustrations in
helping people to explore and understand network structure, and to communicate specific information
about network characteristics to others.
Figure 1: advice network formed by an auditing team. Courtesy of Krackhardt [1996]. Ellipses represent
managers; diamonds represent staff auditors and boxes represent secretaries. A line from Donna to Nancy
indicates that Donna seeks advice from Nancy.
When a social network is mapped to a sociogram, what matters is relationship patterns, not the physical
positioning of nodes [Scott 2000, p. 64]. Many network visualization programs such as Pajek [Batagelj
et al. 1998], KrackPlot [Krackhardt et al. 1994], or MultiNet [Richards 1999] typically use for example,
variants of the spring embedder [Eades 1984], or multidimensional scaling [Kruskal et al. 1978] to
determine node positions. However, previous studies [McGrath et al. 1996, 1997] revealed that the
1
spatial layout of nodes in a sociogram does affect human in perceiving social network characteristics,
such as the existence of subgroups and the social positions of actors, although these characteristics are
purely determined by the intrinsic network structure. In other words, different drawings of the identical
underlying network are likely to convey different, even conflicting information to humans.
Unfortunately, due to diversity and difference of network structure characteristics, probably there is no
single best arrangement of nodes to show all the network information at once [McGrath et al. 1997].
Consequently, sociograms have been used to highlight one or two aspects of network structure, or to
show a general pattern of connections without any specific highlights. Many techniques have been
proposed, such as hierarchical approach [Brandes et al. 2001], which is aimed to convey information
about actor status (more details later in section 2).
One of the major concerns in network visualization is effectiveness. There are two issues involved here:
one is readability. Readability can be affected by not only intrinsic network characteristics [Ghoniem et
al. 2004], but also layout. Among many factors, edge crossings has been widely accepted a major
aesthetic affecting the ease of reading. Moreno [1953, p. 141] suggested edge crossing should be
avoided in constructing “good” sociograms. Purchase [1997], in her pioneering work of a user study
which compared the relative effects of five aesthetic criteria (bends, crosses, angels, orthogonality and
symmetry) on user task performance, also concluded that edge crossings was the most important
aesthetic, which has greatest impact on human graph understanding. Subsequently the aesthetic of edge
crossings was validated on UML diagrams [Purchase et al. 2002]. The remark of Purchase et al. [2002]
that there is no guarantee that results of domain-independent experiments could automatically apply to
domain-specific diagrams motivates this research. We are not aware of any previous user studies in
examining the impact of edge crossings on the perception of social networks when domain-specific
tasks are performed.
The other is communication. Good readability does not necessarily lead to effective underlying
semantics communication. However, the ultimate purpose of visualization is to communicate the
network substance, such as actor importance and group existence. Therefore, a visual representation of
the underlying network should communicate exactly the same information which is intended to convey,
both qualitatively and quantitatively [Brandes 1999]. The arbitrary visual presentation of network data
may end up with a visual ‘mystery’ [Tufte 1983] allowing many different interpretations, or even
worse, communicating false information [Brandes 1999]. This is because when interpreting sociograms,
viewers bring a rich of their own prior knowledge about graphical representations and interpretation
conventions into this process [McGrath et al. 2004]. This sufficiently demonstrates that social network
interpretation is much more sensitive to the spatial arrangement of elements of a graphical display than
others such as understanding UML diagrams. As a result, extra cautions and efforts should be taken to
make sure that any misleading and false interpretations are always avoided. In other words, the
question is how confident we are about what viewers see is exactly what is intended to convey, when
we propose a new or adopt an existing sociogram constructing technique. What is more, since the
visualization process is expensive, it would be always desirable if the visualization technique used can
convey other network information as much as possible except what is highlighted, and prevent
conveying any misleading or false information at the same time. Surprisingly, although considerable
amount of fancy programs and techniques have been proposed in literature [Freeman 2000], their
success and usefulness in communicating the substantive content of the underlying network is typically
judged based on personal opinions and common senses. Very little empirical evidence is available to
support their effectiveness in communicating network structure to humans [McGrath et al. 2003,
Brandes 1999].
To address the above questions, we conducted a user study. In this study, we compared the
communication effectiveness of five sociogram drawing conventions, and investigated the impact of
edge crossings under each convention, based on user preference and task performance. We expected to
1) understand how a particular visualization format or drawing property can affect human sociogram
perception, and propose a set of preliminary recommendations for sociogram design. 2) for our longtime goal, gain some new insights on how people interact with and perceive visual diagrams.
The rest of paper is organized as follows. First in section 2, we briefly have a look at the five social
network visualization conventions, and related work on graph perception. Next the experiment is
presented in section 3, followed by our experiment results in section 4 and discussion in section 5. And
finally in section 6 we summarize our findings and envision some directions for future work.
2
Background and related work
1. Sociogram constructing conventions
Many different and effective techniques have been proposed in literature. They are aimed to highlight
one or more aspects of the network structure, and confirm to some aesthetics to improve the readability.
Of our particular interest are the following five conventions:
Circular layout: For representing social network data visually, the circular layout, in which all nodes
are put on a circle, is used and accepted as a common technique which is intended to make the
relationship patterns among actors salient [Scott 2000, p. 146].
Radial layout: This technique first was proposed by Northway [1940] as a target sociogram (see Figure
2). Nodes are laid inside of concentric circles whose radii increase while the status levels of nodes
decrease, with the most central nodes in the centre of the diagram. The ordering of nodes in circles is
manually determined so that lines connecting them are short. Brandes et al. [2003] suggested a similar
approach which places nodes on the circumference of circles in a way that their distances from the
centre exactly reflect their centrality levels, and nodes on circles are arranged by so-called three-phase
energy-based layout model to make the final layout readable (e. g., Figure 3).
Figure 2: target sociogram [Northway 1940], where
node status is defined by how often the node is
chosen.
Figure 3: radial visualization of betweenness
centrality. Courtesy of Brandes et al. [2003].
Hierarchical layout: To aid in exploring network structure and communicating information about actor
status, Brandes et al. [2001] proposed a novel approach, a prescriptive layout model which directly
maps actors’ status scores to the nodes’ vertical coordinates, and the horizontal positioning of nodes is
“algorithmically” computed such that the ease of overall readability in the final visual network
representation is achieved [e.g., Figure 4]
Group layout: This layout (for a review, see [Freeman 1999]) is used to present information about
groupings. It highlights the group existence by separating different groups and placing nodes in the
same group close to one another.
Free layout: This technique does not have any purpose of highlighting a particular network feature.
Instead sociograms are drawn following general aesthetic principles so that the layout is readable.
Many automatic graph drawing methods can be used for this purpose [Kaufmann et al. 2001].
2. Sociogram perception studies
There are several important studies reporting the spatial layout impact on human understanding of
social networks at different structure levels. For example, at individual level, in their work
investigating how different drawings of the same network can affect the perception of prominence or
bridging of a particular node, McGrath et al. [1997] found that both the network structure and spatial
arrangements influence users’ perceptions. Further at group level, a user study conducted again by
McGrath et al. [1996] suggested the perception of network groupings can be significantly affected by
visual clusters appearing in a sociogram. They proposed a “First principle” in designing sociograms,
which is “adjacent nodes must be placed near to each other if possible, and Euclidean distance should
3
be correlated with path distance”. More recently at overall network level, in investigating the impact of
motion and 2 different layout styles (centrality and hierarchy) on human perception of network changes,
McGrath et al. [2004] found that 1) the difference of layout formats did not have significant effect on
user performance. 2) the introduction of motion to reflect the graph structure changes with hierarchical
layout significantly increased the accuracy of network change perception. 3) subjects’ previous
knowledge and experience with a particular layout format could affect their overall network
interpretations.
3. Readability studies
In a big project investigating the effects of aesthetics and algorithms on graph perception, Purchase et
al. conducted a series of empirical studies both in abstract graph and application domains. Among
them, Purchase et al. [1995] first validated three graph drawing aesthetics (crossings, bends and
symmetry) using paper-based experiments, and revealed that these layout aesthetics do affect human
graph understanding. Then in an online study, Purchase [1997] considered the five aesthetic criteria,
and demonstrated that minimising crossings overwhelmingly aid in understanding graph structure; edge
crossings was “by far the most important aesthetic”. Also they extended their approaches in testing the
user priority of aesthetics on UML diagrams [Purchase et al. 2002]. In [Purchase et al. 2002] it is
pointed out that there is no guarantee that the results of domain-independent experiments would
automatically apply to the domain-specific diagrams. Further, in an experiment which examined
several aesthetics within the same set of computer-generated diagrams, Ware and Purchase [2003]
suggested and demonstrated that good path-continuity has positive impact and detailed that what
matters is the crossings on the shortest paths for shortest-path searching tasks. Another notable
contribution in this study is the introduction of new evaluation methodology which enables us to do
cognitive cost assessment and is applicable for similar perception problems.
Instead of aesthetics affect, Ghoniem et al. [2004] compared readability effects of two visualization
techniques (node-link and matrix-based representations of abstract graphs) and graph intrinsic
properties (size and density). They found that readability effects of the two techniques vary with size
and density of graphs for different types of tasks. In particular, node-link diagrams are more suitable
for small graphs while matrices achieve better readability for large or dense graphs. And more
significantly, they proposed a generic task list and methods for the evaluations of the similar type, and
recommendations in choosing effective graph representations.
Experiment
The aims of this experiment are twofold: 1) comparison of the communication effectiveness of five
network visualization conventions: radial, circle, group, and free layout styles; and 2) for each
convention, the impact of edge crossings.
The measurements are task performance, usability preference, questionnaires and interviews. By
making use of a combination of these methods, we expect to obtain thorough understanding of human
sociogram perception. We believe that the subjective data abstracted from questionnaires and
interviews can help in gaining in-depth insights which otherwise is unlikely by just looking into static
and silent objective data.
1. Subjects
Twenty-three subjects were recruited from a student population in computer science on a completely
voluntary basis. All the subjects were postgraduates and computer literate. All had node-edge diagram
experience such as UML or ER, associated with their study units; six of them were graph drawing
research students. All had neither academic nor working experience related to social networks. They
were reimbursed $20 each for their time upon the completion of their tasks.
2. Design
Networks: For this experiment, two networks are used. One is a real world advice network formed by a
small auditing team, which was described by Krackhardt [1996]. The network is modelled as a directed
graph with 14 actors and 23 edges as shown in Figure 4. A line from actor A to actor B indicates that A
seeks advice from B. The network has three groups in a sense that we discuss later in this section.
4
Figure 4: Krackhardt’s advice network used in the
study (hierarchical layout).
Nancy
1.00
Bob
0.00
Donna
0.66
Carol
0.00
Manuel
0.57
Harold
0.00
Stuart
0.19
Wynn
0.00
Charles
0.17
Susan
0.00
Kathy
0.08
Fred
0.02
Tanya
0.08
Sharon
0.02
Table 1: actors’ Katz status scores
The Katz status scores [Katz 1953] of the actors are shown in Table 1. Due to its suitability in
measuring actor status in the underlying context of this advice network, Katz status score was used as
the index of importance. We expected subjects to perceive the importance in accordance with, and we
measured their perceptions against actors’ Katz status scores.
The second one is a fictionalised network which was produced from the first one by eliminating all the
directions. This gives an undirected graph with 14 actors and 23 edges, which we call a collaboration
network. A line between A and B means that A and B collaborate with each other.
Sociograms: We used a total of 12 drawings: 1) for each of the 5 conventions, two drawings of the
advice network – one with minimum crossings and one with many crossings (see Table 2); 2) for the
collaboration network, a free layout drawing with minimum crossings and a free layout drawing with
many crossings. For simplicity, we call the two drawings for each convention a pair of non-crossing
drawing and crossing drawing.
All nodes were labelled with different names; in every drawing, each node was mapped to a new name.
By providing a context and background for each network, and names for actors, subjects were expected
to perform tasks from the real world social network perspective [McGrath 1997]. However, subjects
were not made aware that the drawings had the same graph structure.
Tasks: Social network analysis focuses on patterns of relations among actors. There are many different
techniques having been developed with each measuring a particular network structure feature
[Wasserman & Faust, 1994]. However, for this experiment, as an initial step, we only considered two
common social network measures which are frequently highlighted in sociograms: one is prominence
or importance of actors; the other is the presence of social groups, in which the connections among
actors are relatively dense. The whole session included 5 tasks:
1) Online tasks: (a) find the 3 most powerful (popular) persons and rate them according to their
importance levels; and (b) determine how many groups are in the network and separate the 4
highlighted persons according to their group membership, given the condition that one person
should not belong to more than one group, and one group should not include only one person. In
formal tests, the same 4 nodes across all drawings were highlighted in red rectangles.
2) Usability acceptance rating: with one page showing all 6 crossing drawings, and the other page
showing all 6 non-crossing drawings, subjects were required to rate their usability based on a scale
from -3 (completely unacceptable) to +3 (completely acceptable) for importance tasks and group
tasks, respectively.
3) Crossing preference rating: each crossing (A) and non-crossing (B) pair was shown one by one, 6
pairs in total. Subjects needed to indicate their preferences for importance tasks and group tasks,
respectively, based on a scale from -2 (strongly A) to +2 (strongly B), where, for example,
“Strongly A” means A is strongly preferred over B.
5
Non-crossing drawing
Crossing drawing
Radial
Hierarchical
Circular
Group
Free
Table 2: 10 sociograms for the advice network used in this study
6
4) Overall usability ranking: with all 10 advice network drawings being shown in one page, subjects
needed to choose 3 drawings that they least preferred and 3 drawings that they most preferred for
their overall usability, using a scale from -3 to -1 and from 1 to 3, respectively.
5) Questionnaires: There were 2 questionnaires with each having a different focus, and to be
presented to subjects before and after they were debriefed about edge crossings and drawing
conventions, respectively. The first questionnaire asked subjects information about their study
background, experience with node-link diagrams and social networks, how they interpret
sociograms, and any network structure and sociogram features that they think may influence
(positive and negative) their graph perceptions. The second questionnaire asked about their
thoughts about conventions and edge crossings.
For the above tasks (2)-(4), subjects were also required to write down a short explanation for each
answer.
Online system: Sociograms were displayed by a custom-built online experimental system. The system
was designed so that:
− A question is shown first, a button on the screen is pressed, then the corresponding drawing is
shown; after answering, the button is pressed and the next question is shown, and so on.
− Each subject’s response time for each drawing is logged. This starts once a drawing is completely
displayed and ends once the button is clicked.
The study employed a within-subjects design. For online tasks, 10 of the 12 drawings were randomly
chosen and shown to comply with the time schedule and to reduce fatigue. The order of group and
importance questions for each drawing was also random. Subjects were told they could have breaks
during the question viewing periods if they wished. There was no time limit on task completion,
although they were recommended to answer each question in one minute. During the preparation time,
subjects were instructed to answer each question in the context of the underlying network and as
quickly as possible without compromising accuracy.
4. Procedure
Four of the recruited subjects who did not have any social network background participated with a trial
study to check our methodology. They showed that they quickly understood the questions and felt
comfortable with the experiment. They related the visual network representations with their daily social
relationship experiences when performing tasks. Their performance and comments demonstrated that
the tasks used were challenging but not overwhelmingly difficult. However, they had quite different
criteria in dividing networks into groups. To make the experiment controllable, we added for the group
question a condition which has already been reflected in the previous statements about online tasks in
this section. This is trivial since our aim is to evaluate the drawing conventions not the question itself.
The formal experiment took place in a computer laboratory of the School of Information Technologies
on the Campertown campus of the University of Sydney. All PCs in the lab had the same specifications.
Before starting the experiment, subjects were asked to read the information sheet, sign the consent form,
read through and understand the tutorial material, ask questions and practice with the online system.
The drawings used for practice were randomly generated normal node-link diagrams, which were quite
different from the ones used in formal tests, since the practice was only for familiarisation with the
procedure and the system, not for them to get experienced with sociogram reading.
Once ready to start, subjects indicated to the experimenter, and started running the online system
performing tasks formally. After the online reading tasks followed by a short break which was to
refresh subjects' memory, they proceeded with the rating tasks (2)-(4), and the first questionnaire. Then,
after being given a debriefing document explaining the nature of the study, edge crossings and drawing
conventions, subjects were asked to do the rating tasks (2) and (3) again, and finally finished with the
second questionnaire. Subjects were also encouraged to verbalize any thoughts and feelings about the
experiment. The whole session took about 60 minutes.
7
Subjects’ online responses, rating scale data were analysed using statistical analysis methods. All
significant (p<0.05) differences found will be discussed in the next section. Qualitative data from
questionnaires and interviews were analysed to explain and reinforce our observations.
Results
Twenty-seven subjects participated with the formal tests. However, the data produced by three of the
subjects were discarded since they did not follow the instructions thoroughly enough. Since the
collaboration network sociograms did not create distinct difference with their counterparts, these data
have been omitted in our analysis. For simplicity and convenience, we use C, R, G, H, and F to
represent circular, radial, group, hierarchical and free drawing conventions, respectively, and use P for
non-crossing drawings, and C for crossing drawings. Therefore CP denotes circular non-crossing
drawing; CC denotes circular crossing drawing; HC denotes hierarchical crossing drawing, and so on.
1. User preference data
1.1 Usability acceptance
Subjects’ usability rating scale data were analysed using ANOVA (with Fisher’s PLSD, for pairwise
comparisons). To examine edge crossings impact, we compared each non-crossing and crossing pair of
drawings for each convention, and all non-crossing drawings with all crossing drawings (as indicated in
Overall column in Table 3). To examine convention effect, we performed statistical tests among all
non-crossing, all crossing and all 10 non-crossing and crossing drawings, respectively (as indicated in
the last column in Table 3). And if there was a significant difference among all non-crossing drawings,
then the further pairwise comparisons would be conducted (We did not consider all crossing drawings
for pairwise comparisons due to the negative effect of edge crossings demonstrated throughout the
analysis). The same statistical procedure was followed for response time analysis.
2.5
4
P
2
1.5
2
1
1
0.5
C
0
0
-0.5
P
3
C
-1
C
F
G
H
R
-1
F
G
H
R
-3
Figure 5: mean usability scores for importance
Importance
P
C
P value
C
-2
C
0.125
-0.500
0.247
F
1.750
0.292
0.000
Figure 6: mean usability scores for group
G
1.708
1.417
0.359
H
2.208
1.750
0.230
R
1.292
0.417
0.008
Overall
1.417
0.608
0.000
P value
0.000
0.000
0.000
Table 3: mean scores for importance and p values
Importance
CP
FP
GP
HP
RP
CC
FC
GC
HC
RC
Pre
0.125
1.750
1.708
2.208
1.292
-0.500
0.292
1.417
1.750
0.417
Post
Paired P
value
0.042
1.375
1.333
2.750
1.875
-0.500
0.542
1.083
2.292
1.625
0.770
0.083
0.131
0.039
0.001
1.000
0.266
0.119
0.045
0.000
Table 4: mean scores of pre- and post-debriefing for importance and p values
For importance tasks, as seen from Figure 5 and Table 3, the non-crossing drawing was generally rated
higher than the crossing one for each convention. Only CC was considered unacceptable. Among all
non-crossing drawings, HP was rated highest, followed by FP, GP, RP, and finally CP, which was
considered having little usefulness. There was a significant crossings effect between all non-crossing
and all crossing drawings (p=0.000). Also there was a significant convention effect among all drawings
(p=0.000). The pairwise comparisons revealed that the user acceptance for CP was significantly
different for GP, HP and FP, respectively; the user acceptance for RP was significantly different for CP
and HP, respectively. Furthermore, as indicated in Table 4, paired t tests showed that after debriefing,
drawings of both H and R conventions were rated significantly higher at the levels of 0.05 and 0.001,
8
respectively. In particular, the mean scores of the pair of H convention were higher than all others,
showing that node positions were perceived more important than edge crossings for importance tasks.
Group
P
C
P value
C
-0.583
-1.833
0.024
F
0.750
-1.125
0.000
G
2.833
2.833
1.000
H
0.750
-0.667
0.003
R
1.958
0.375
0.000
Overall
1.142
-0.083
0.000
P value
0.000
0.000
0.000
Table 5: mean scores for group usability and p values
Group
CP
FP
GP
HP
RP
CC
FC
GC
HC
RC
Pre
-0.583
0.750
2.833
0.750
1.958
-1.833
-1.125
2.833
-0.667
0.375
Post
Paired P
value
-0.500
0.667
2.833
1.333
1.500
-1.458
-0.542
2.875
-0.083
0.792
0.753
0.780
1.000
0.075
0.018
0.026
0.016
0.575
0.016
0.135
Table 6: mean scores of pre- and post-debriefing for group and p values
For group tasks, it can be seen from Figure 6 and Table 5 that the 2 drawings of C convention were
rated unacceptable, while the 2 drawings of G convention were rated acceptable (highest). For other
conventions, only non-crossing drawings were considered acceptable. Among all, 2 group drawings
(GP and GC) were rated much higher than others; in fact the others were perceived as having little
usefulness. Analysis found a significant crossings affect for each pair of drawings for all conventions
(p=0.000) except G. Also, conventions produced a significant difference among all drawings (p=0.000).
Pairwise comparisons showed that all pairs were significantly different except between HP and FP.
Table 6 shows that in subjects’ post-debriefing ratings, there was a significant change at the level of
0.05 for RP, CC, FC and HC.
1.2 Crossing preference
Crossing
Non-Crossing
Crossing
Non-Crossing
R
R
H
H
G
G
F
F
C
-2
-1.5
-1
-0.5
C
0
0.5
1
1.5
2
Figure 7: mean preference scores for importance
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Figure 8: mean preference scores for group
Subjects’ mean preference scores were summarised in Figures 7-8. Generally, they preferred noncrossing drawings more. For both importance and group tasks, subjects most strongly preferred the
non-crossing drawing for H convention, while for G convention, the non-crossing drawing was
preferred only slightly more than the crossing drawing. The paired t tests revealed that there were no
significant changes between the pre- and post-debriefing ratings, although the post-debriefing
preferences were generally weaker than pre-debriefing for both importance and group tasks.
Particularly after debriefing, subjects preferred the crossing group drawing slightly more than its noncrossing counterpart instead.
The 1–sample t tests against the hypothesized mean (=0) revealed that for all conventions except G,
subjects’ preference for non-crossing drawings over crossing drawings was statistically significant
(P<=0.01), as shown in Table 7.
Importance
C
F
G
H
R
Mean
0.96
1.38
0.67
1.58
1.38
Pre
P value
0.010
0.000
0.001
0.000
0.000
Mean
0.71
1.04
0.33
1.29
0.96
Post
P value
0.005
0.000
0.073
0.000
0.001
Group
C
F
G
H
R
Mean
0.88
1.33
0.04
1.63
1.46
Table 7: mean crossing preference scores of pre- and post-debriefings
9
Pre
P value
0.001
0.000
0.770
0.000
0.000
Mean
0.92
1.08
-0.17
1.38
0.96
Post
P value
0.000
0.000
0.491
0.000
0.000
1.3 Overall usability ranking:
50
30
10
-10
-30
-50
CC
FC
HC
RC
CP
FP
RP
HP
GP
GC
-45
-45
-20
-14
-5
3
7
22
47
50
Figure 9: weighted values for overall usability
Subjects’ ranking values for each drawing were summed as a weighted value, and weighted values for
all drawings are shown in Figure 9. It can be seen that generally, the crossing drawings were least
preferred except GC, which was ranked the highest for its overall usability, followed by GP, then HP.
Both CC and FC had the lowest weighted value, indicating that they were considered having little
overall practical utility.
2. User performance data
2.1. Response time
Subjects’ response time data were illustrated in Figures 10-11 and summarised in Tables 8-9. The nonparametric method of Kruskal-Wallis was used for response time analysis.
70
120
P
60
P
100
C
50
C
80
40
60
30
20
40
10
20
0
0
C
F
G
H
R
C
Figure 10: median time (sec.) for importance
Importance
P
C
P value
C
50.683
54.989
0.588
F
57.168
57.643
0.989
G
44.694
50.983
0.109
F
G
H
R
Figure 11: median time (sec.) for group
H
50.749
45.546
0.63
R
52.046
61.634
0.211
Overall
49.266
53.558
0.495
P value
0.185
0.327
0.262
H
57.740
63.401
0.53
R
54.684
74.497
0.133
Overall
55.540
73.110
0.021
P value
0.001
0.001
0.000
Table 8: median time (sec.) for importance and p values
Group
P
C
P value
C
55.029
95.777
0.012
F
71.573
76.219
0.603
G
26.598
29.017
0.729
Table 9: median time (sec.) for group and p values
For importance tasks, subjects spent shorter time with the non-crossing drawing than the crossing
drawing for each convention in general. Among all non-crossing drawings, the shortest time was spent
with GP, followed by CP, HP, RP, and finally FP. However, statistical tests revealed that these
differences of response time were not statistically significant in terms of the effect of either edge
crossings or drawing conventions.
For group tasks, following a similar pattern, again shorter time was spent with the non-crossing
drawing than the crossing one for each convention. Among all non-crossing drawings, the shortest time
10
was spent with GP, followed by RP, CP, HP, and finally FP. Analysis showed that there was a
significant crossings effect between the non-crossing and crossing pair of C convention (p=0.012), and
between all non-crossing and all crossing drawings (p=0.021), and a significant convention difference
among all drawings (p=0.000). Pairwise comparisons showed that subjects spent significant shorter
time with GP than all others at the 0.01 level.
2.2 Identifying most important people
700
600
500
400
300
200
100
0
CC
CP
FC
FP
GC
GP
HC
HP
RC
RP
542.9 499.8 615.7 515 394.3 482.3 440 466.8 516.7 522
Figure 12: weighted values for identifying most important people
Correctness
(%)
1st
(×5)
2nd
(×2)
3rd
(×1)
Weighted
Value
CC
CP
FC
FP
GC
GP
HC
HP
RC
RP
100
14.3
14.3
94.7
10.5
5.3
100
52.6
10.5
90
25
15
72.2
11.1
11.1
94.1
5.9
0
85
5
5
77.8
27.8
22.2
88.9
33.3
5.6
94.4
22.2
5.6
542.9
499.8
615.7
515
394.3
482.3
440
466.8
516.7
522
Table 10: correctness percentages and weighted values.
Figure 12 and Table 10 show the correctness percentages and weighted values for all drawings. The
weighted value is to measure a drawing’s overall effectiveness of conveying information about
importance, and calculated in the following way: First we gave an index of 5 to the most important
person, 2 to the second and 1 to the third; then the productions of indices and corresponding
correctness percentages were summed as a weighted value for each drawing. It can be clearly seen that
for identifying the most important person, both CC and FC had the highest correctness percentage
(100%), followed by CP (94.7%), then RP (94.4%). GC had the lowest percentage; only 72.2% of
subjects found the most important person correctly. Similarly, for overall effectiveness, FC had the
highest weighted value (615.7), followed by CC (542.9), then RP (522), and again GC had the lowest
weighted value (394.3).
2.3 Reported group number and member group assignment
P
C
C
3.42
3.23
F
3.40
2.84
G
3.06
3.22
H
3.61
3.45
R
3.17
3.44
P value
0.087
0.124
Table 11: mean number of groups reported and p values
Table 11 shows the means of subjects’ reported group number and p values. An analysis of variance of
the reported group number among all drawings showed there was a significant difference at the level of
0.066.
11
1 2
3 4
1
5
2 3
4 5
GC
FP
FC
CP
CC
80
60
40
20
0
RP
RC
HP
HC
GP
80
60
40
20
0
1
2 3
1 2
4 5
3
1 2
4 5
3 4
5
Figure 13: distribution of reported group number
Figure 13 illustrates the distribution of reported group number for each drawing. As can be seen, GP
yielded the highest correctness rate; 82.4% of subjects responded nodes ‘correctly’ (3 groups as
expected), followed by RP (72.2%) and GC (66.7%). HP came as the lowest; only 22.2% of subjects
reported the group number was 3, suggesting its inability in communicating groupings.
Also, at dyad level, the perception of whether two nodes belong to the same group can be affected not
only by their structural relationship, but also spatial proximity [Ware 1999 p. 203, McGrath et al. 1996].
The member group assignment task was to investigate edge crossings and convention impact on
perception of nodes’ co-memberships. Subjects’ responses in dividing 4 given people into groups were
summarised and illustrated in Figure 14.
90
80
70
60
50
40
30
20
10
0
CC
CP
FC
FP
GC
GP
HC
HP
RC
RP
9.5
47.4
47.4
10
72.2
76.5
5
12.5
11.1
22.2
Figure 14: group assignment correctness percentage
As can be seen, a relatively larger proportion of subjects performed this task correctly on the noncrossing drawing than on the crossing drawing for each convention except F. Among all non-crossing
drawings, GP had the highest correctness percentage (76.5%), followed by CP (47.4%), then RP
(22.2%), HP (12.5%), and finally FP (10%). In the case of HC, only 5% of subjects responded to this
task correctly.
Qualitative data and discussions
1. Edge crossings effect
Our quantitative results showed that in general, edge crossings did make significant differences in user
preference, usability acceptance, and group task performance. Nearly 100% of subjects indicated that
drawings with fewer crossings are desirable and easy to read. They commented crossing drawings
“hard to read”, “confusing”, etc. This is consistent with previous research results. However, no evident
results were found to show that edge crossings have significant effect on importance task performance.
We guess that in perceiving who is important, subjects’ main focuses were on identifying outgoing and
incoming arrows, and this can be done by only looking around the nodes, does not involve links
searching much. However, user comments vary. For example: “In finding important people, maybe
location is more important than crossing”; “Crossings matter when counting the number of groups”;
“Crossings matter when seeing which nodes are important in a directed graph, and edges with fewer
crossings can reveal relationships easily”.
12
2. Convention effects
The quantitative analysis revealed that conventions had significant impact on usability acceptance, and
group task performance. Again, no apparent evidence was found that there was a significant convention
impact on user importance task performance.
Circular layout: Circular layout had moderate task performance, but was least accepted in usability.
Indeed, it treats all nodes equally by placing them in a circle without any particular priority, thus
making network structure features hard to discern. User comments include “I hate meaningless circular
layout”; “nodes in the same circle look like they are equally important”; “it is frustrating following
long edges from one side to the other side to find groups”.
Radial layout: Radial layout had moderate user ratings and task performance. Although there is visible
big circle around the diagram, very few subjects realised nodes were actually arranged radially. Since
people’s reading habit is normally left-to-right and top-to-bottom, adding a big circle around the
diagram does not necessarily make viewers read radially. Providing clear visual hints such as adding
circles at each level could be an option [Brandes et al. 2003]. But this also increases visual complexity,
and likely confuses novice viewers. Some subjects commented like “what is the outer circle for? I do
not think it was necessary”; “it is better to have circles visible if radial layout is to be used”; “it is good
to put the most important node in the centre, but it is crowded with others”.
Group layout: Group layout was rated high for group and overall usability, and moderate for
importance usability. It also had very high user performance for group tasks, and moderate
performance for importance tasks. User comments include “groups are already spatially separated”;
“very clear overall structure”; “good for grouping”; “relationships are clear”; “it would be better if
putting the group having most important people on the top”.
One finding worthy mention here is related to crossings impact on group layout. Although edge
crossings surely make perceiving groupings harder in general, some subjects commented that as shown
in Figure 15, when sociograms are drawn with group convention, crossing edges within groups gives a
stronger sense that nodes are linked closely. Besides, since links within a group are naturally dense, by
crossing edges, it is easier to identify relationship patterns among nodes. For example, “Crossings
probably do not matter when nodes belonging to different clusters have already been separated”;
“Crossings are helpful when there is a pattern”. This indicates that edge crossings have little negative
consequence when group layout is used for communicating groupings. However, although the
perceived usability (acceptance and response time) was high for both the non-crossing and crossing
drawings of group layout, subjects did not achieve satisfied response accuracy for importance tasks,
while very high for group tasks.
Figure 15: it is argued that the right side picture gives a stronger sense of
group, and is easier to find relationship patterns.
Free layout: Free layout had moderate user ratings for importance tasks, and low ratings and
performance for group tasks. Some subjects commented that nodes seemed “not organised”,
“unordered, making grouping difficult”; some edges were “unnecessary long”; “each time I look the
diagram, it seems there are different ways of grouping”; “it is easy to think Nancy is the most important
because she is topright corner, but clearly Tony is more important but he is easy to miss because he is
at the bottom”; “it is easy to identify important nodes”.
Hierarchical layout: This layout was most preferred for importance tasks, but had low ratings and
performance for group tasks. Comments include “I prefer tree-like layout”; “it is good to arrange
13
arrows pointing the same direction”; “it is easy to see most important people on the top”; “long edges
make some nodes in the same group separate”.
Quite surprisingly, subjects were overwhelmingly in favor of hierarchical convention for importance
tasks; they spent relatively short time with HP, but obtained the lowest correctness rate among all
minimum-crossing drawings. On the other hand, FC obtained the highest correctness rate, but relatively
long time was spent with it. We realized that some subjects had complained in questionnaires and
verbally that in some drawings, edges were incident to nodes too closely to clearly identify arrow
directions. Visual inspection revealed that indeed, free convention drawings had very good angular
resolution, while hierarchical convention made angular resolution relatively low, where edges had to be
crowded in one side of nodes. In addition, subjects spent longer time with FC, which might actually
allow them to have better chance to understand network structure better.
3. Summary and recommendations
From subjects’ responses to the questionnaires, it is clear that there are two distinct groups of subjects
who have different approaches in perceiving sociograms. When asked “what factors did you consider
in finding important people and groups”, almost everyone responded that only actual relationships
among people should be considered, However, in responding to another question about sociogram
perception, about 85% of subjects indicated that their final answers were determined not only by
relations but also spatial layouts. For example: “50% relations, 50% layout”, “a little bit layout”, “only
layout of drawing”, “try my best to find the answer according to relations; when too complex, I will
rely on the layout”, etc. On the other hand, 15% of subjects indicated that their answers were only
determined by relations. They responded like “although a bad layout may affect me to understand
network correctly, the final answer should be similar; it only takes a longer time”; “better graph layout
makes faster to understand”, etc.
Interestingly, these two groups of subjects also have slightly different focuses in response to the
question “what features of visual representations help or hinder your understanding”. The first group of
subjects seemed to purely concern about the spatial arrangement. For example, “placing important
people on the top or in the centre helps”, “some important people are crowded with other”, “groups
should be clearly separated”, “low degree nodes on the top should be avoided”, etc. The second group
focused more on the visual reflection of intrinsic structure. For example, “when relationship
dependency makes a cycle, it is difficult to judge who is more important”; “clustering groups
‘correctly’ helps”; “nodes should not be evenly distributed in a diagram”; “whether a node is important
or not depends on links it has, not on its position in the diagram”, etc. Unfortunately, we had
insufficient data to identify the factors contributing the existence of the two groups of subjects.
In terms of sociogram perception, subjects have a strong preference of placing nodes on the top or in
the centre to highlight importance, and clustering nodes in the same group and separating groups to
highlight groups. They have tendency to believe that nodes in the centre or on the top are more
important, and nodes in close proximity belong to the same group. In addition, with regard to graph
reading behaviour, subjects tend to consider those nodes in their central attentions high in importance.
Due to the close relation between visual attention and eye movements [Ware 1999, p.154], 3
hypotheses can be derived based on subjects’ comments which need confirmation with further
specifically designed experiments: 1) When looking at a diagram, the first couple of fixations are on
top and centre areas, and overall more time is spent on these areas than other peripheral areas. 2) When
looking at a diagram, overall more time is spent on sparse areas than dense areas where nodes and links
are crowded. 3) However, their normal reading behaviours (eye movements) can be changed and
guided by introducing visual hints into diagrams. For example, circles in radial layout.
To be able to propose a set of aesthetics for design reference, we summarised and analysed subjects’
responses, and finally made a list of guidelines as follows:
1)
2)
3)
4)
5)
6)
Reduce crossing number
Highlight important nodes, by using colour, size, etc.
Separate important nodes with others
Do not crowd nodes together
Put the most important nodes on the top or in the centre
Do not distribute nodes randomly
14
7)
8)
9)
10)
11)
12)
13)
14)
15)
Do not cross edges haphazardly
Do not treat nodes equally such as putting nodes in a circle or distribute nodes evenly
Keep edges shorter when underlying relationships are closer
Arrange arrows to point the same direction
Increase angular resolution
Provide visual hints when necessary
Cluster nodes in the same group
Cross edges when links are dense in a group
Separate groups spatially or by adding boundaries
The above guidelines are rough and not a priority list. Most items on the list are consistent with general
rules which are widely adopted in practice. However, it is very important to be aware that visualization
techniques that are highly preferred by users do not necessarily always produce best task performance,
as demonstrated in this study. Further, some tradeoffs should be balanced when a technique is to be
used for visualization, since it is very unlikely we can maximise all aesthetics at the same time. This
list can surely be refined and extended as our understanding about human sociogram perception is
getting better. In terms of network visualization conventions, what we can recommend based on the
results from this study is that hierarchical layout is more suitable for highlighting important actors, and
group layout is more suitable for conveying groupings and overall information.
The issue of confounders has always been a major concern for this kind of experiments. The design of
explanations required in user rating tasks was to enable us to identify possible confounders. However,
the following facts resulted from careful inspection have made us consider this issue trivial, to a large
extent: 1) even before debriefing, an overwhelming majority of subjects clearly indicated their
preference and acceptance in terms relevant to edge crossings and conventions. For example:
“overlapping edges”, “structure is clearer”, “circular layout”, “edges cross randomly”, “even
distribution of nodes”, etc. 2) what seems to be confounders is either closely related to a specific
convention or common for all conventions. For example: hierarchical layout automatically leads to
important nodes on the top, and arrows pointing upward. 3) The significant difference among the
quantitative data was consistent before and after subjects were debriefed about edge crossings and
conventions.
Conclusions and future work
This study compared relative effectiveness of five visualization conventions: radial, circular, group,
hierarchical and free drawing, in conveying internal structure information at the individual and group
levels, and investigated the impact of edge crossings on human perception.
We found strong evidence that edge crossings contribute to the significant difference in user preference,
usability acceptance, and group task performance. Edge crossings not only affect the ease of reading,
but also affect the understanding of network structures.
With respect to drawing conventions, for importance tasks, hierarchical layout was strongly preferred,
while for group tasks, group layout was strongly preferred. Users achieved the highest response
accuracy with group layout for group tasks. However, the highest response accuracy did not come with
hierarchical layout for importance tasks. For overall usability, group layout was the one for which the
usability was rated high and user performance was well as well.
However, no obvious evidence was found that either edge crossings or conventions pose significant
impact on user importance task performance. Users generally performed better when they took longer
time. We conjecture that only those tasks which are closely related to edges and involve edge searching
can be significantly affected, such as finding groupings. On the other hand, for communicating
information about actor status, the node positioning and angular resolution in a sociogram may be more
important, compared to reducing crossings number and drawing conventions.
In a broader theme, this study, together with previous research, has demonstrated that how sensitive the
human sociogram perception is to spatial layouts, and how important it is to have visualization
techniques evaluated for their actual effectiveness in communication from human understanding point
of view.
15
The findings from this study should be interpreted within the limitations of the given experimental
settings. One limit which is related to general human graphical perception experiments is that people
are likely to behave differently from the way they do in their daily lives, when explicitly asked to
perform online tasks in an experiment [Cleveland 1985]. In this study we had only investigated the
relative effectiveness of five “explanatory visualization” [Brandes et al. 2001] conventions and
crossings impact under each convention, in communicating actor status and subgroup information to
novice audience. Their usability in assisting investigators to explore and understand social networks
remains untouched and is beyond the scope of this study. For a comprehensive overview in this field,
see [Card et al. 1999]. In addition, individual differences, time limits in performing tasks and
characteristics of different networks concerned may make a difference in experimental results, and
have not been included in this investigation. In particular, a user’s understanding can be affected by his
personal social experience, their familiarity with the visual presentations and the network contexts, the
attributes of internal relationship structure, the way the network is presented. Although the
generalization and extension of the findings requires further finer designed experiments, this study, as a
preliminary approach, at least provides experimental basis for further studies and empirical supporting
evidence for what is widely accepted as general practices, if not more.
Additional studies are needed to empirically identify and investigate the impact of other possible
variables of human sociogram perception in a more controlled manner. Another direction of research
that we are going to pursue is to try to understand how people interact and interpret graphs.
Acknowledgements
The authors are grateful to Mr Kelvin Cheng, and Mr Le Song for their helpful comments on
questionnaire design, and to students who willingly took part in the experiment. Ethical clearance for
this study was granted by the University of Sydney, December 2004.
References
[1] Batagelj, V. and Mrvar, A. (1998) Pajek: Program for large network analysis. Connections, 21(2):47-57.
[2] Card, S., Mackinlay, J., and Shneiderman, B. (1999) Readings in information visualization: using vision to
think. Morgan Kaufmann Publishers, San Francisco, CA.
[3] Cleveland, W. S., and McGill, R. (1984) Graphical perception: Theory, experiments and application to the
development of graphical methods, JASA 39, pp. 531-554.
[4] De Nooy, W., Batagelj, V. and Mrvar, A. (2000) Visual analysis and prespecified blockmodels. Sunbelt XX,
Vancouver.
[5] Eades, P. (1984) A heuristic for graph drawing. Congressus Numerantium, 42:149-160.
[6] Frank, K. A. (1996) Mapping interactions within and between cohesive subgroups. Social networks, 18(2):
93-119.
[7] Freeman, L. (2000) Visualizing Social Networks. Journal of Social Structure, vol. 1, number 1.
[8] Freeman, L. (1999) Visualizing Social Groups. American Statistical Association, 1999 Proceedings of the
Section on Statistical Graphics, 47-54.
[9] Ghoniem, M., Fekete, J. and Castagliola, P. (2004) A Comparison of the Readability of Graphs Using NodeLink and Matrix-Based Representations. Proceedings of the 10th IEEE Symposium on Information
Visualization (InfoVis'04), Austin, TX, Oct 2004. IEEE Press. pp. 17-24.
[10] Katz, L. (1953) A new status index derived from sociometric analysis. Psychometrika, 18:39-43.
[11] Kaufmann M. and Wagner D. (Eds.) (2001) Drawing Graphs: Methods and Models, Lecture Notes in
Computer Science 2025, Springer Verlag.
[12] Krackhard, D., Blythe, J. and McGrath, C. (1994) KrackPlot 3.0: An improved network drawing program.
Connections, 17(2):53-55.
[13] Krackhardt, David. (1996) Social Networks and Liability of Newness for Managers. In C. L. Cooper and D.
M. Rousseau (eds.), John Wiley & Sons, Ltd. New York, NY. Trends in Organizational Behavior, Volume 3,
pp. 159-173.
[14] Krempel, L. (2000) Exploring and communicating block models visually. Sunbelt XX, Vancouver.
[15] Kruskal, J. B. and Wish, M. (1978) Multidimensional scaling, Beverly Hills, CA: Sage.
[16] McGrath, Cathleen, Jim Blythe and David Krackhardt. (1997) The effect of spatial arrangement on judgments
and errors in interpreting graphs. Social Networks, 19(3):223-242.
16
[17] McGrath, Cathleen, Jim Blythe and David Krackhardt. (1996) Seeing Groups in Graph Layout. Connections,
19(2): 22-29.
[18] McGrath, Cathleen and Jim Blythe (2004) Do You See What I Want You to See? The Effects of Motion and
Spatial Layout on Viewers' Perceptions of Graph Structure. Journal of Social Structure, vol 5, number 2.
[19] McGrath, Cathleen, David Krackhardt, and Jim Blythe (2003) Visualizing Complexity in Networks: Seeing
Both the Forest and the Trees. Connections, 25(1): 37-47.
[20] Moreno, J. L. (1953) Who shall survive: Foundations of Sociometry, Group Psychotherapy, and Sociodrama.
Beacon House Inc.
[21] Northway, M. L. (1940) A method for depicting social relationships obtained by sociometric testing.
Sociometry, vol. 3, no. 2, pp. 144-150.
[22] Purchase, H. C., Cohen R., and James M. (1995): Validating graph drawing aesthetics. Proceedings of the
Graph Drawing Symposium, Passau, Germany, 435-446, Springer-Verlag.
[23] Purchase, H. C., Jo-Anne Allder, David A. Carrington. (2002) Graph Layout Aesthetics in UML Diagrams:
User Preferences. J. Graph Algorithms Appl. 6(3): 255-279.
[24] Purchase, H. C. (1997) Which aesthetic has the greatest effect on human understanding? In Di Battista, G.,
editor, Proceedings of the 5th International Symposium on Graph Drawing (GD '97), volume 1353 of Lecture
Notes in Computer Science, pages 248-261. Springer.
[25] Richards, W. D. (1999) MultiNet. Software tool. See http://www.sfu.ca/~richards/Multinet/
[26] Scott, J. (2000) Social Network Analysis: A Handbook. Sage Publications, 2nd edition.
[27] Tufte, E. R. (1983) The visual display of quantitative information. Cheshire, CT: Graphics Press
[28] Ulrik Brandes, Dorothea Wagner (2003) visone - Analysis and Visualization of Social Networks. In Michael
Jünger and Petra Mutzel (Eds.): Graph Drawing Software, pp. 321-340. Springer-Verlag.
[29] Ulrik Brandes, Jörg Raab, and Dorothea Wagner (2001) Exploratory Network Visualization: Simultaneous
Display of Actor Status and Connections. Journal of Social Structure 2(4).
[30] Ulrik Brandes, Patrick Kenis, and Dorothea Wagner (2003) Communicating Centrality in Policy Network
Drawings. IEEE Transactions on Visualization and Computer Graphics 9(2):241-253.
[31] Ulrik Brandes, Patrick Kenis, Jörg Raab, Volker Schneider, and Dorothea Wagner (1999) Explorations into
the Visualization of Policy Networks. Journal of Theoretical Politics 11(1):75-106.
[32] Ulrik Brandes, Patrick Kenis, and Dorothea Wagner (1999) Centrality in Policy Network Drawings. Proc. 7th
Intl. Symp. Graph Drawing (GD'99). LNCS 1731, pp. 250-258.
[33] Ware, C. (1999) Information visualization: perception for design. Morgan Kaufman.
[34] Ware, C., Purchase, H.C., Colpoys, L. and McGill, M. (2002) Cognitive Measurements of Graph Aesthetics,
Information Visualization, 1(2), pp 103-110.
[35] Wasserman, S. and Faust, K. (1994) Social Network Analysis: Methods and Applications. Cambridge
University Press.
[36] Wikipedia. (2005) Social network: http://en.wikipedia.org/wiki/Social_network. Accessed 20 March 2005.
17