Towards Understanding the Understandability of
Diagrams
Niek Tax
University of Twente
P.O. Box 217, 7500AE Enschede,
The Netherlands
[email protected]
ABSTRACT
This paper describes models and metrics on conceptual
diagram understandability looking at both the chosen
conceptual modeling notation (language level) and the
individual diagram created in this conceptual modeling
notation (sentence level). Several metrics for conceptual
diagram understandability exist which look at conceptual
diagrams from a different perspective. We could not find any
work that looks at more than one of these diagram
understandability perspectives at the same time. This paper
aims to be a step towards integration of the different
perspectives of conceptual model understandability. We used
a questionnaire to gain insights on the ratio between sentence
level and language level influence in conceptual diagram
understandability. The main insight of this study is that the
tested aspects at sentence level seem to have a bigger impact
on perceived diagram understandability than the tested
notation level aspects. In order to draw valid conclusions
about the overall impact of the sentence level and notation
level on diagram understandability as a whole more research
is needed.
Keywords
Conceptual diagram, modeling notation, diagram aesthetics,
diagram understandability
1. INTRODUCTION
In the communication process between stakeholders within
IT-projects, a commonly used communication medium is
conceptual diagrams. Among popular conceptual diagram
notations are UML, ER and ArchiMate, but many other
conceptual modeling notations are also used in practice.
For efficient stakeholder communication, it is of importance
that the conceptual models used for communication are easily
understandable for all stakeholders. Lindland et al [15] stated
that the quality of a conceptual model is determined largely by
how easily all concerned parties of the model understand it.
How well we are able to understand and process the
information inside a conceptual diagram depends on the
appearance of the diagram. Aesthetic properties like the
number of Line Crossings and Line Bends influences the
degree in which stakeholders are able to read and cognitively
process the information in the conceptual diagram [21]. Other
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
16thTwente Student Conference on IT, January 27st, 2012, Enschede, The
Netherlands.
Copyright 2012, University of Twente, Faculty of Electrical Engineering,
Mathematics and Computer Science.
non-aesthetic factors influencing the degree in which
stakeholders are able to read and cognitively process the
information in the conceptual diagram are diagram structure
(diagram complexity) [12] and modeling notation choice [18].
Study shows that UML can be hard to understand for novices
[22], which could make UML a less suitable modeling
notation choice for stakeholder communication in case
conceptual modeling novices are involved.
For said reasons it is important to make well thought choices
on diagram aesthetics and used modeling notation in order to
avoid miscommunication between stakeholders.
2. PROBLEM STATEMENTS &
RESEARCH QUESTIONS
As understandable conceptual diagrams are essential for
effective stakeholder communication, some insight is required
in how stakeholders and diagram designers can improve
conceptual diagram understandability.
Some research has already been done on the topic of
conceptual diagram understandability. Several studies
approach the research area of conceptual diagram
understandability from a different perspective.
In Table 1, an overview of studies in the diagram
understandability field is given where studies were
categorized in three perspectives:
1.
2.
3.
Notation Perspective
Studies in the notation perspective focus on the fit
between a chosen modeling notation, the people
working with the diagram and the diagramming
task.
Aesthetics Perspective
Studies in the aesthetics perspective focus on visual
properties of the diagram. Some examples of a
diagram’s visual properties are diagram symmetry
and the number of Line Crossings in a diagram.
Structure Perspective
Studies in the structure perspective focus on
structural properties of a diagram like the number of
entities and the number of relationships in a
diagram. The structure of a diagram is closely
related to the thing that is actually modeled in the
diagram, where the aesthetics of a diagram are
completely unrelated to this. This is an important
difference between the aesthetics perspective and
the structure perspective.
We based this three perspective categorization on Moody’s
[18] categorization in sentence level perspective and language
level perspective. Moody defines language perspective as the
fit between a chosen modeling notation, the people working
with the diagram and the diagramming task (our notation
perspective). Sentence level can be described as all aspects
that have nothing to do with the notation (which is both our
aesthetics perspective and structure perspective).
diagram designer. This will allow us to answer the second
research question.
Table 1: Different perspectives of conceptual diagram
understandability
3.2 Questionnaire
Notation Perspective
Moody defines cognitive fit as the three-way fit
between the problem representation, task
characteristics and problem solver skills.
Eight principles are specified for constructing a
conceptual modeling notation which influences the
cognitive fit of the language. [18]
Lindland defines conceptual diagram quality as a
combination of the way the language fits the domain,
the way the language fits the audience and the way the
audience fits the domain [15].
Aesthetics
Perspective
Perspectives on diagram understandability
Several authors have specified notation and situation
independent factors and aspects which influence
conceptual diagram understandability. These factors
contain visual aspects like symmetry, Line Bends and
Line Crossings. [4, 7, 8, 19, 20, 23]
Structure
Perspective
Genero defined and validated metrics for structural
diagram properties which affect conceptual diagram
understandability for ER diagrams [12].
Several metrics on understandability have been
proposed for the UML class diagram notation [3, 6, 14,
16]. These metrics mostly lack proper validation and
are often only in early stages of development [11].
Although several models exists that look at different
perspectives of conceptual diagram understandability, we
could not find any papers that looked at sentence level as well
as language level understandability by considering and
comparing the different aspects and metrics and critically
validating them. To be able to improve conceptual diagram
understandability, all perspectives are relevant and cannot be
neglected. This leads to the following research questions:
-
Which perspective has a higher impact on
understandability given a communication situation?
Which factors and perspectives can conceptual
modelers influence to improve conceptual diagram
understandability?
Both questions help to gain some insight in the question what
conceptual modelers can do to improve conceptual model
understandability.
3. METHOD OF RESEARCH
We approach this research with a literature review and then
use a structured questionnaire to test and gather results on the
impact of the perspectives on conceptual diagram
understandability.
3.1 Literature Research
The goal of this literature research is to get a list of metrics
and aspects for conceptual diagram understandability. With
the different perspectives on conceptual diagram
understandability and their metrics, an analytical step needs to
be made on how these perspectives fit together in a complete
model on conceptual diagram understandability. With this
literature research and analytical step the answer to the first
research question can be found.
The literature research will also focus on the question of
which of the identified perspectives are within control of the
After the literature research and the analytical step a
structured questionnaire will be used to get insight in what
degree the identified perspectives on conceptual diagram
understandability influences the overall understandability of a
conceptual diagram.
As this study will mainly focus on giving insight in the
different conceptual diagram understandability perspectives
and linking them together, there will not be enough time to
conduct a survey thoroughly enough to draw valid
conclusions on the degree in which each conceptual diagram
understandability perspective influences total conceptual
diagram understandability. Still, this empirical study can give
some insight in this field and can function as a proof of
concept for future studies.
4. MODEL APPEARANCE
In literature, four aspects can be distinguished that influence
the appearance of a diagram:




Modeling notation [18]
Model semantics
Sentence level design choices [2, 21]
Modeling tool [10]
In which perspective each of the four aspects fit can be seen in
Table 2.
Table 2: Model appearance perspectives and their aspects
Perspective
Aspect
Notation perspective
Modeling notation
Modeling tool
Aesthetics perspective
Sentence level design choices
Modeling tool
Structure perspective
Model semantics
In the following sections, the aspects influencing diagram
appearance will be explained. The perspective categorization
from Table 2 will here be applied to discuss the aspects in a
structured manner.
4.1 Notation perspective
4.1.1 Choosing a Modeling Notation
What a diagram looks like is dependent on the modeling
notation which is used. We can easily see that this the case, as
each conceptual modeling language has its own semantic
constructs and its own graphical symbols representing these
semantic constructs.
4.1.1.1 Moody’s Principles
Moody [18] determined eight principles determining the
degree in which the chosen conceptual modeling language fits
the characteristics of the task that it’s used for and the
problem solver skills of the stakeholders.
The existence of such an optimal fit suggests that different
modeling notations or visual dialects may be required for
different tasks and audiences. As there is a big difference in
their understanding of conceptual diagrams between novices
and experts [24], this is conceivable.
Moody defines the following eight principles:








Perceptual Discriminability
Graphic Economy
Semantic Transparency
Semiotic Clarity
Visual Expressiveness
Cognitive Integration
Manageable Complexity
Dual Coding
4.1.1.2 Metrics based on Moody’s Principles
In a work in progress paper, Amrit and Iacob define metrics
for Moody’s eight principles, which allows us to measure with
the Moody principles [1]. Amrit and Iacob’s metrics can be
found in Table 3. In this paper we simultaneously validate
some of these metrics, as well as use them to draw
conclusions on the objective application of Moody’s
principles to diagrams. The metrics can be used to gain some
insight on how well the several Moody principles are applied
in a particular diagram.
the used notation with the used tool and task characteristics
and involved stakeholders.
4.2 Aesthetics perspective
4.2.1 Sentence Level Design Choices
The modeling notation and semantics are not the only
influencers for diagram appearance. For a given modeling
notation and semantics, the graph designer himself can still
make a lot of design decisions. These decisions include size of
graphical elements, place of graphical elements, and font
selection.
4.2.2 Diagramming Tool influencing Aesthetics
At sentence level, the tool influences the appearance of a
diagram in two ways:

By applying an Automatic Graph Layout algorithm.

A modeling notation determines the appearance of a
graphical symbol largely by the graphical symbols it
uses, but still there are some free variables available
for the tool to vary. Examples of free aesthetic
variables often implemented by the tool are line
thickness and font choices within graphical
symbols.
Table 3: Sentence Level metrics based on Moody's
Principles as defined by Amrit and Iacob (derived from
[1])
Pattern Name
Semiotic Clarity
Perceptual
Discriminability
Semantic
Transparency
Complexity
Management
Cognitive
Integration
Visual
Expressiveness
Dual Coding
Graphic Economy
Cognitive Fit
Metric Description at the Sentence
Level
Number of instances of symbol
redundancy, symbol overload and
symbol excess in a diagram
Visual Distance in a diagram
Number of symbols that are
semantically opaque and perverse in
a diagram
Diagrammatic complexity = no. of
instances/tokens in a diagram
# Instances of lack of conceptual and
perceptual integration
Instances of: 8 - # information
carrying variables
+1 for Instances of lack of dual
coding
NA
Dependent Variable
4.1.2 Role of the Tool at Notation Perspective
On language level, the tool influences the graph appearance in
the way it fits the notation. A tool may for example not have
all graphical symbols of a modeling notation implemented.
This limits the number of graphical symbols the user can use
and may thereby influence the appearance of a diagram.
No research on notation-tool-fit could be found. Still, we can
draw some conclusions on how the tool influences conceptual
diagram understandability on language level. When a tool
uses a graphical symbol for some semantic construct(s) that
differs from the formal specification of the notation, we can
consider the modeling language offered by the tool to be a
different dialect of the notation that the tool says it offers.
When we keep this in mind, we can see that for determining
cognitive fit of a conceptual diagram one should look at the
notation in the way it is offered by the tool instead of the way
the notation is formally specified. By using Moody’s
principles and Amrit’s metrics on the tool implementation of a
notation, conclusions can be drawn regarding the fit between
Automatic Graph Layout algorithms make some aesthetic
choices independently, thus without influence of the diagram
designer. At the same time, it is conceivable that an
independently made aesthetic choice of the Automatic Graph
Layout algorithm persuades the designer to make different
sentence level design choices. The Automatic Graph Layout
algorithm assists the diagram designer in several ways,
including automatic alignment of graphical elements and
prevention of Line Crossings and Line Bends. Sentence level
graph aesthetics are an interplay between sentence level
design choices and the Automatic Graph Layout algorithm of
the tool.
Currently most diagram modeling tools do not implement
state-of-the-art Automatic Graph Layout algorithms and are
not capable of generating very understandable diagrams [9].
4.2.3 Purchase Metrics
Purchase [20] defines a set of metrics defining
understandability for several sentence level diagram
aesthetics. Purchase defines seven principles that reach
optimal understandability. For each principle, a metric is
defined to identify in what degree a diagram fulfills this
principle.
The following seven principles are defined by Purchase:







Minimizing edge crossings
Minimizing edge bends
Maximizing symmetry
Maximizing the minimum angle between edges
leaving a node
Maximizing Edge Orthogonality
Maximizing Node Orthogonality
Maximizing consistent flow direction (directed
graphs only)
The metrics for these seven principles as defined by Purchase
will not be discussed in this section of the paper as they are of
such complexity that it takes a complete paper to explain them
all.
4.3 Structure perspective
4.3.1 Model Semantics
The real world objects and events which are being modeled
have influence on the appearance of the graph. This is selfevident, as a different real world model implies different
semantic constructs which results in different symbol
instances. As we define the structure of a diagram as the use
of symbol instances in the chosen modeling language (e.g. the
amount of entities, the amount of relationships), the structure
of a diagram heavily depends on the real world thing that is
being modeled.
The metrics that can be used to determine the
understandability of the conceptual diagram structure depend
on the notation in which the diagram is created. Genero
created a set of metrics which can be used to determine
understandability of the diagram structure for ER diagrams
[12]. Several metrics exists for UML diagrams, but they are
still in early stages of development and mostly not validated
[11].
The influence that semantics have on diagram appearance is
less relevant for our search for better graph understandability.
The real world objects and events being modeled are almost in
all cases not changeable by the diagram designer. As we have
no influence on the semantics, it is less relevant to our search
for more understandable diagrams.
4.4 Power of Choice
An interesting question is which of the graph appearance
aspects are within power of the diagram designer. It is
conceivable that a diagram designer is ought to use a
modeling notation used and known by other stakeholders and
thus is not free to choose his own notation. The designer
might also be ought to use the tool available at his/her
working environment. No research on obligatory use or free
choice of modeling notations of modeling notations and
modeling tools by diagram designers could be found.
However, it could be possible that not all diagram designers
are free to choose the modeling notation and tool. The amount
of freedom in diagramming tool choice for conceptual
modelers would be an interesting topic for further research as
no work in this field could currently be found.
It can easily be seen that within a specific task the diagram
designer can have no influence on the sentence level
semantics. The real world objects and events being modeled
follow directly from the task. As sentence level semantics is
defined as the real world objects and event of the model, the
task directly determines the sentence level semantics of its
conceptual model.
The one aspect that a diagram designer always has influence
on are his sentence level design choices. If notation and tool
choices are not within power of choice of the diagram
designer, he/she can still aim at the most understandable
sentence level design choices. When a graphic designer is able
to choose the notation and tool he can also aim at the most
cognitively fitting modeling notation for the task and
audience.
to differ the diagrams on, four aspects of aesthetics
perspective and two aspects of notation perspective were
chosen. By testing understandability of diagrams with both
varying aesthetic perspective and notation perspective aspects,
some insight can be gained on how much impact these aspect
are for conceptual diagram understandability.
The structure perspective is left out of the questionnaire on
purpose, as diagram designers are often not able to modify the
aspects within the structure perspective.
5.1 Aesthetics perspective aspects
Within the aesthetics perspective Line Crossings, Line Bends,
Node Orthogonality and Edge Orthogonality were chosen to
vary the diagrams on, as these are aspects that are validated to
make a difference on understandability of UML class
diagrams [21].
In the following sections the metrics for Line Crossings, Line
Bends, Node Orthogonality and Edge Orthogonality as
defined by Purchase [20] will be explained briefly. The
explanations of the metrics in the following sections are
derived from Purchase’s study [20] in which the metrics were
first presented and explained. In these sections, the following
definitions are applicable:








the number of nodes in the diagram
the number of edges in the diagram
the ith node in the diagram
the number of vertical grid point in the
diagram
the number of horizontal grid point in the
diagram
the number of nodes in a diagram if all
bends in the edges are promoted to nodes
the number of edges in a diagram if all
bends in the edges are promoted to nodes
the positive angle between the ith edge and
the x-axis
5.1.1 Line Crossings
First Purchase defines an approximation for the upper bound
of the number of edge crossings. The number of edge
crossings are calculated as if every edge were to cross every
other edge.
∑
This is an overestimate from the total number of edge
crossings known to be impossible in a straight-line diagram
are subtracted. In straight-line drawings of connected graphs
with at most one edge between nodes, adjacent edges cannot
cross. The total number of such impossible crossings is
therefore:
∑
( )
The upper bound on the number of edge crosses in a diagram
is therefore:
5. QUESTIONNAIRE DESIGN
An interesting question is whether it would be better to focus
on notation perspective or aesthetics perspective differences,
which is also one of the research questions of this paper. In
order to give insight in this problem, eleven conceptual
models were created modeling the same real world scenario.
We varied these eleven conceptual models both on aspects of
notation perspective and aesthetics perspective. As variables
The actual number of crosses in a diagram is scaled against
the maximum number of crossings. To reach a scale in which
1 represents maximum ‘crosslessness’, the scaled
measurement of crosses is subtracted from 1.
{
5.1.2 Line Bends
It is impossible to scale against an upper bound of the number
of bends, which is infinite, so it is scaled by the total number
of edge segments:
Symbol redundancy was added on purpose to several of the
diagrams in order to obtain worse Semiotic Clarity. We expect
that the introduction of symbol redundancy to the UML
diagrams will have a negative influence on diagram
understandability, as it creates worse Semiotic Clarity.
5.2.2 Perceptual Discriminability
So that 1 represents maximum ‘bendlessness’, this scaled
measurement of bends is subtracted from 1.
5.1.3 Edge Orthogonality
The edge deviation factor of the ith edge segment ( )
represents how far away from an orthogonal angle the edge
segment has deviated. It is computed as a proportion of the
angular deviation of the ith edge segment from the horizontal
or vertical gridlines:
|
|
So that 1 represents drawings with optimal edge deviation
with respect to the orthogonal grid (i.e. assumed easier to
read), the average edge deviation factor over all edge
segments is subtracted from 1.
The other notation perspective aspect that we vary in the
diagrams is the Perceptual Discriminability between symbols
within the notation. Perceptual Discriminability is the ease
and accuracy with which different graphical symbols can be
differentiated from each other. Accurate discrimination
between symbols is a necessary prerequisite for accurate
interpretation of diagrams.
Perceptual Discriminability is primarily determined by the
visual distance between symbols, which is defined by the
number of visual variables on which they differ and the size
of these differences. The existing visual variables can be
found in Figure 3. In general, the greater the visual distance
between symbols used to represent different constructs, the
faster and more accurately they will be recognized. Bad
Perceptual Discriminability affects modeling novices more
than modeling experts [5].
∑
5.1.4 Node Orthogonality
The number of available grid-point intersections in the
imaginary unit grid is:
Purchase defines the Node Orthogonality metric as the extent
to which the drawing makes maximal use of the grid area:
Figure 2: Visual variables (from [18])
5.2 Notation perspective aspects
The UML class diagram uses only the visual variable shape to
differ between types of symbols, and even within this visual
variable mostly rectangular shapes are used [17]. The use of
color as visual variable is specifically avoided in UML:
At notation perspective we chose to vary the diagram at
Perceptual Discriminability and Semiotic Clarity, which are
two notation principles defined by Moody [18]. Those
particular Moody principles were chosen as both (directly or
indirectly) influence the cognitive fit of the diagram while
they are almost independent of other Moody principles [1,
18].
In the following sections a brief explanation of Perceptual
Discriminability and Semiotic Clarity is given. Metrics for
both aspects can be found in Table 3.
5.2.1 Semiotic Clarity
“UML avoids the use of graphic markers, such as colour, that
present challenges for certain persons (the colour blind) and
for important kinds of equipment (such as printers, copiers,
and fax machines).” [13]
In several of the questionnaire diagrams, color is added to the
UML notation by assigning each used graphical symbol a
different color for better symbol discriminability. We expect
that this increased symbol discriminability will lead to better
understanding of the diagram, especially with users that are
less familiar with conceptual modeling.
5.3 Variance of aspects in questionnaire
diagrams
To measure the impact of each line, crossings, Line Bends,
Node Orthogonality, Edge Orthogonality, Semiotic Clarity
and Perceptual Discriminability on the perceived
understandability of a diagram, several diagrams are created
which differ on these aspects. The metrics defined by
Purchase [20] as explained in section 5.1 are used.
Figure 1: Semiotic Clarity: In an ideal diagram, there will
be a 1:1 correspondence between semantic constructs and
graphical symbols (from [18])
For Semiotic Clarity and Perceptual Discriminability the
metrics defined by Amrit and Iacob [1] are used which are
shown in Table 3. The metric for the Perceptual
Discriminability is defined as the visual distance in a diagram.
In UML, class diagrams only use shape as visual variable.
When color is added as extra visual variable to discriminate
between symbols, the Perceptual Discriminability metric of
that UML class diagram increases from 1 to 2. Moody states
that the more visual variables the graphical symbols differ on,
the more each graphical symbol pops out. This can have a
positive effect on diagram understandability [18].
The metric for Semiotic Clarity is defined as the number of
instances of symbol redundancy, symbol overload and symbol
excess in a diagram. Most UML class diagrams do not include
instances of symbol redundancy, symbol overload and symbol
excess in a diagram. Those diagrams have a Semiotic Clarity
score of 0. In several diagrams an instance of symbol
redundancy is added, those diagrams have a score of 1 on the
Semiotic Clarity metric.
In table 3 an overview can be found of how well each diagram
scores on each aspect. The aesthetic perspective aspects Line
Crossings, Line Bends, Node Orthogonality and Edge
Orthogonality all have score on a scale from 0 to 1, where
Purchase perceives a higher score to be more understandable
[20].
The questionnaire can be viewed online at:
http://www.thesistools.com/web/?id=236082
6. RESULTS & DISCUSSION
We obtained 517 data points, from 47 subjects and 11
diagrams per subject. We start by using a one-sample T test to
determine if the improving knowledge of the model subject
throughout the questionnaire has influence on the diagram
understandability.
6.1 One-sample T test
Line Bends
Node
Orthogonality
1
0.57
1.00
0.29
0.58
2
0
2
1.00
0.80
0.32
1.00
1
1
Diagram 4
47
3.53
0.905
0.132
3
0.79
1.00
0.32
0.81
1
1
Diagram 5
47
4.79
0.463
0.068
4
1.00
0.33
0.04
0.49
1
0
Diagram 6
47
3.13
1.076
0.157
5
1.00
0.80
0.28
1.00
1
0
Diagram 7
47
3.64
0.942
0.137
Diagram 8
47
3.23
1.127
0.164
Diagram 9
47
3.02
0.944
0.138
Diagram 10
47
3.87
0.924
0.135
Diagram 11
47
3.28
0.902
0.132
Edge
Orthogonality
Line Crossings
Semiotic Clarity
Table 5: One-sample T test results on the questionnaire
diagrams
Diagram
Perceptual
Discriminability
Table 4: Properties of the questionnaire diagrams
The participants of the questionnaire were students enrolled in
the study Business Information Technology, Industrial
Engineering or Computer Science at the University of Twente
in the Netherlands. Students of these studies where used as
questionnaire participants as these students can be expected to
have at least basic knowledge of conceptual modeling. All
questionnaire participants have at least completed the
introductory course on Information Systems.
One-Sample Statistics
6
0.79
1.00
0.28
0.49
1
0
7
1.00
0.50
0.06
0.81
2
0
8
0.57
1.00
0.29
0.58
2
0
9
1.00
0.33
0.04
0.49
1
1
10
1.00
0.17
0.20
1.00
1
0
11
1.00
1.00
0.38
0.41
1
0
The properties of the diagrams were chosen in such a way that
most diagrams differ heavily one or several of the defined
aesthetic perspective properties or one or several of the
defined notation perspective properties or a combination of
both. To be able to compare the perceived understandability
of these diagrams to the perceived understandability of a clean
UML diagram without the defined flaws, diagram 5 is created
which scores quite well on all aspects.
All diagrams have the same real world objects that are being
modeled. This is needed to exclude the complexity of the
modeling subject from the data. On the other hand, this
approach has the potential drawback that participants in the
test might get more familiar with the modeling subject
throughout the questionnaire. To be able to check in what
degree the familiarity with the modeling subject influences the
perceived model understandability, the first diagram of the
questionnaire is shown again at diagram number eight.
Any learning-effect is minimized by choosing a model
semantics such that all participants have at least basic domain
knowledge.
N
Mean
Std. Deviation
Std. Error
Mean
Diagram 1
47
3.28
1.136
0.166
Diagram 2
47
4.11
0.914
0.133
Diagram 3
47
2.62
0.922
0.134
Diagram 1 and diagram 8 are in fact identical, but diagram 1
is shown as first diagram in the questionnaire and diagram 8 is
shown as eighth diagram. If the improving knowledge
throughout the questionnaire does not influence diagram
understandability, both diagrams are expected to have equal
perceived understandability.
In table 5, it can easily be seen that the mean perceived
diagram understandability and standard deviation of diagram
1 and diagram 8 are almost identical. It can therefore be
expected that the improving knowledge of the diagram subject
does not significantly influence perceived diagram
understandability.
6.2 Spearman’s rank correlation
coefficient
Table 6 shows the correlations between the chosen aspects
and the perceived diagram understandability. The results
presented in table 6 are obtained using spearman’s rank
correlation coefficient.
The findings that Table 6 reveals are:

Line Crossings and Edge Orthogonality seem to
have a positive correlation with perceived diagram
understandability. This was to be expected, as a
higher score on the Line Crossings metric actually
means the diagram has less Line Crossings. A
higher score on the Edge Orthogonality metric
means that the diagram has more orthogonal edges.
The positive correlation between Line Crossings,
Edge Orthogonality and the perceived diagram
understandability is consistent earlier research [20,
21].

Table 6: Spearman's rank correlation coefficient for the
defined aspects
Correlations
Spearman’s rho
Line Crossings
Line Bends
Node
Orthogonality
Edge
Orthogonality
Perceptual
Discriminability
Semiotic Clarity
**
*
Understanda
bility
Correlation Coefficient
.275**
Sig. (2-tailed)
.000
N
517
Correlation Coefficient
-.209**
Sig. (2-tailed)
.000
N
517
Correlation Coefficient
-.069
Sig. (2-tailed)
.117
N
517
Correlation Coefficient
.000
N
517
Correlation Coefficient
-.080
Sig. (2-tailed)
.069
N
517
Correlation Coefficient
-.137**
Sig. (2-tailed)
.002
N
517
Correlation is significant at the 0.01 level (2-tailed).
Correlation is significant at the 0.05 level (2-tailed).


Of the two notation perspective aspects, namely Semiotic
Clarity and Perceptual Discriminability, one aspect had a
weak but significant correlation with perceived diagram
understandability, while the other did not have a significant
correlation with diagram understandability.
Of the four aesthetic perspective aspects, namely Line
Crossings, Line Bends, Node Orthogonality and Edge
Orthogonality, three aspects had a significant correlation with
perceived diagram understandability. Those three aspects all
have a stronger correlation with perceived diagram
understandably compared with the notation perspective aspect
Semiotic Clarity.
7. CONCLUSIONS
Now it is time to look back at the research questions stated in
the beginning of this paper:
-
.325**
Sig. (2-tailed)
Line Bends seem to have a negative correlation with
perceived diagram understandability. This is
inconsistent with earlier research [20, 21], where a
positive correlation between Line Bends and
diagram understandability has been shown. The
diagrams with line bends were created by adding
line bends to a different diagram from the test.
Therefore the participants are shown a line bends
version of a diagram after having seen an identical
diagram (in all other aspects) earlier. It might be the
case that the participants learned from this earlier
identical diagram and as a result neglects the line
bends in his understandability rating.
Node Orthogonality seems to have no significant
correlation
with
perceived
diagram
understandability. The notation perspective aspect
Perceptual Discriminability also does not seem to
have a significant correlation with perceived
diagram understandability. It is possible that these
two aspects have so little influence on perceived
diagram understandability compared to the
influence of other aspects used to differ the
diagrams that their influence on perceived
understandability is covered by the other aspects.
Semiotic Clarity has a weak but significant negative
correlation
with
perceived
diagram
understandability. As the Semiotic Clarity metric is
defined as the number of instances of symbol
redundancy, symbol overload and symbol excess in
a diagram, it was to be expected that a higher score
on this metric would result in a less understandable
diagram.
-
Which perspective has a higher impact on
understandability given a communication situation?
Which factors/perspectives can conceptual modelers
influence to improve conceptual diagram
understandability?
It can be concluded that the two notation perspective aspects
used in the questionnaire have on average less impact than the
four aesthetic perspective aspects.
It cannot be concluded that the aesthetic perspective has
overall a larger impact on perceived diagram
understandability than notation perspective, as the notation
and aesthetics perspective both include more aspects than the
aspects included in the questionnaire. However, this study
provides us with insight into the comparative impact of two
notation perspective aspects and four aesthetic perspective
aspects on perceived diagram understandability.
Looking at the second research question, we can conclude that
aspects in the aesthetics perspective can always be influenced
by conceptual modelers. Conceptual modelers are however
not in all occasions free to choose the modeling notation and
the modeling tool they want. As mentioned earlier, exact
numbers on free choice of modeling notation and tool could
not be found. Therefore, the free choice of modeling tools
would be an interesting area of research for future studies.
8. FUTURE WORK
The questionnaire can only compare the impact of the four
aesthetic perspective aspects with the impact of the two
notation perspective aspects. To be able to fully compare the
impact of the notation perspective with the aesthetics
perspective, more research is needed where the impacts off all
aesthetic perspective aspects and notation perspective aspects
are taken into account.
Further research is also needed in the area of free choice of
modeling tools and modeling notations for conceptual
modelers. Such a study could influence the answer to the
question what conceptual modelers can actually do to improve
conceptual model understandability.
9. ACKNOWLEDGMENTS
My special thanks to Dr. Chintan Amrit, for guiding me
through the research as my supervisor. In addition, I want to
thank all my student colleagues for reviewing this work and
providing insightful comments.
10. REFERENCES
1.
Amrit, C. and Iacobs, M.E. Patterns for Communicating
with Models. Not yet published.
2. Andriyevska, O., Dragan, N., Simoes, B. and Maletic,
J.I., Evaluating UML class diagram layout based on
architectural importance. in, (2005), IEEE, 1-6.
3. Bansiya, J. and Davis, C.G. A hierarchical model for
object-oriented design quality assessment. Software
Engineering, IEEE Transactions on, 28 (1). 4-17.
4. Bennett, C., Ryall, J., Spalteholz, L. and Gooch, A. The
aesthetics of graph visualization. Proceedings of
Computational Aesthetics in Graphics, Visualization,
and Imaging, 5.
5. Britton, C. and Jones, S. The untrained eye: how
languages
for
software
specification
support
understanding in untrained users. Human–Computer
Interaction, 14 (1-2). 191-244.
6. Chidamber, S.R. and Kemerer, C.F. A metrics suite for
object oriented design. Software Engineering, IEEE
Transactions on, 20 (6). 476-493.
7. Dunne, C. and Shneiderman, B. Improving graph
drawing readability by incorporating readability metrics:
A software tool for network analysts. University of
Maryland, HCIL Tech Report HCIL-2009-13.
8. Eichelberger, H., Aesthetics of class diagrams. in,
(2002), IEEE, 23-31.
9. Eichelberger, H., Nice class diagrams admit good
design? in, (2003), ACM, 159-ff.
10. Eichelberger, H. and Schmid, K. Guidelines on the
aesthetic quality of UML class diagrams. Information
and Software Technology, 51 (12). 1686-1698.
11. Genero, M., Piattini, M. and Calero, C. A survey of
metrics for UML class diagrams. Journal of Object
Technology, 4 (9). 59-92.
12. Genero, M., Poels, G. and Piattini, M. Defining and
validating metrics for assessing the understandability of
entity-relationship diagrams. Data & Knowledge
Engineering, 64 (3). 534-557.
13. OMG Group. OMG Unified Modeling Language (OMG
UML), Superstructure, V2.1.2, 2007.
14. Li, W. and Henry, S. Object-oriented metrics that predict
maintainability. Journal of systems and software, 23 (2).
111-122.
15. Lindland, O.I., Sindre, G. and Solvberg, A.
Understanding quality in conceptual modeling. Software,
IEEE, 11 (2). 42-49.
16. Marchesi, M., OOA metrics for the Unified Modeling
Language. in, (1998), IEEE, 67-73.
17. Moody, D. and Van Hillegersberg, J. Evaluating the
visual syntax of UML: An analysis of the cognitive
effectiveness of the UML family of diagrams. Software
Language Engineering. 16-34.
18. Moody, D.L. The “physics” of notations: Toward a
scientific basis for constructing visual notations in
software engineering. IEEE Transactions on Software
Engineering. 756-779.
19. Poranen, T., Mäkinen, E. and Nummenmaa, J., How to
draw a sequence diagram. in, (2003), 91-102.
20. Purchase, H.C. Metrics for graph drawing aesthetics.
Journal of Visual Languages & Computing, 13 (5). 501516.
21. Purchase, H.C., Carrington, D. and Allder, J.A.
Empirical evaluation of aesthetics-based graph layout.
Empirical Software Engineering, 7 (3). 233-255.
22. Siau, K. and Cao, Q. Unified modeling language: A
complexity analysis. Journal of Database Management
(JDM), 12 (1). 26-34.
23. Sun, D. and Wong, K., On evaluating the layout of UML
class diagrams for program comprehension. in, (2005),
IEEE, 317-326.
24. Winn, W. An account of how readers search for
information in diagrams. Contemporary Educational
Psychology.