MSc Software Maintenance
MS Viðhald hugbúnaðar
Fyrirlestrar 25 & 26
The Importance of Typographic Style
“Hence, the book format is a more appropriate form
for representing program source code.”
28/07/2017
Dr Andy Brooks
typographic/prentlegur
1
Case Study
Dæmisaga
Reference
Typographic Style is More than Cosmetic,
Paul W. Oman and Curtis R. Cook,
Communications of the ACM, Volume 33, Number 5,
pp 506-520, 1990. © ACM
“... about one half of a maintenance programmer´s time is
spent studying source code and related documentation.”
“Unfortunately, in many instances the only reliable description
of a program is the source code itself.”
28/07/2017
Dr Andy Brooks
2
Macro-typographic principles
• Code areas for global definitions, the main program,
support routines, and included code segments should be
easily identified.
• Module separation should be obvious.
– module separation conventions
• The purpose and use of each component should be
identified.
– intermodule commenting
• The execution control and information flow between
components should be clear.
• The program should be readable and different access
paths available when browsing.
28/07/2017
Dr Andy Brooks
3
Micro-typographic principles
• Code areas for constants, data declarations, and code
body should be easily identified within a module.
• The purpose and use of each section within a module
should be identified.
– intramodule commenting
• The execution control and information flow within a
module should be clear.
• Statements should be readable and easily scanned.
– statements written as sentences
• White space to separate groups of statements.
– use of blank lines
28/07/2017
Dr Andy Brooks
4
Typographic principles and factors
• The principles can be implemented with
regard to several typographic factors:
– commenting
– naming
– blank lines
– type and font usage
– CharacterCase
etc.
28/07/2017
Dr Andy Brooks
5
Book Format Paradigm
•
•
•
•
•
•
Preface
Table of contents
Indices
Pagination
Chapters
Sections
28/07/2017
•
•
•
•
•
Paragraphs
Sentences
Punctuation
Type style
Character case
Dr Andy Brooks
6
Figure 1. Pages from a Book Format Listing ©ACM
Preface
28/07/2017
Dr Andy Brooks
access paths
7
Figure 1. Pages from a Book Format Listing ©ACM
Index
Called from:
Calls to:
28/07/2017
Dr Andy Brooks
access paths
8
Figure 1. Pages from a Book Format Listing ©ACM
indentation
italics
horizontal sentences
comment indicating end of code section
CharacterCase
28/07/2017
Dr Andy Brooks
9
Experiment 1:
Testing Macro-Typographic Principles
• Two versions of a line-oriented text editor written in
Pascal were created:
– traditional listing
– book format listing
• table of contents, chapters, and module index
• The Skip_Blanks procedure and five calls to it were
removed from the original 1,532-line program.
• Procedures unrelated to command parsing were also
removed to reduce the original program to 1,011 lines.
– “a size that could be managed by student programmers in one
hour”.
28/07/2017
Dr Andy Brooks
10
Experiment 1
Subjects/Þátttakendur
• 53 senior and graduate level computer science
students.
• Subjects randomly assigned into two groups.
– 28 subjects received program version 1
– 25 subjects received program version 2 (book format)
• The same instructions were given to everybody.
• No special instructions or explanations for the
book format listing.
– “this was deliberately done to test whether subjects
could naturally use the book format listing”
28/07/2017
Dr Andy Brooks
11
Experiment 1
Maintenance Task
• “Write a Skip_Blanks procedure that would
enable free-form command inputs, and
indicate where (on the listing) the
procedure would be called.”
• Subjects needed to understand the
command line structure and the execution
flow of the routines that manipulated the
command line.
• Subjects were given 55 minutes.
28/07/2017
Dr Andy Brooks
12
Experiment 1
Results ©ACM
• Regarding a correct Skip_Blanks routine, book format
listing outperforms traditional listing by almost 2:1 (13:7).
• Twice as many traditional listing subjects gave up or did
not finish (11:4).
28/07/2017
Dr Andy Brooks
13
Experiment 1
Chi-square tests/Kí-kvaðrat próf
Trad
4
3
10
11
Book
9
4
8
4
• Pearson chi-square = 10.45
• p < 0.025, d.f. = 3 (degrees of freedom, frígráður)
Trad
7
Book 13
21
12
• Pearson chi-square = 4.09
• p < 0.05, d.f. = 1
p < 0.05 ...something is not purely random
reject the null hypothesis
28/07/2017
Dr Andy Brooks
14
Experiment 1
Results ©ACM
5 calls
• Data only for subjects who had a correct Skip_Blanks procedure.
• Percentage accuracy is greater for book format (47.6% > 34.2%).
– but no statistical test reported
28/07/2017
Dr Andy Brooks
15
Experiment 1
Results & Discussion
• No difference found in average times.
– Version 1, 53.5 minutes
– Version 2, 52.2 minutes
• Macro-typographic style improved programmer
performance in terms of accuracy but not time.
– 2:1 ratio for a correct Skip_Blanks procedure
• “... book format listing performed better without
any explanation, description, or justification of
the book format listing.”
28/07/2017
Dr Andy Brooks
16
Experiment 2:
Testing Micro-Typographic Principles
• Two Pascal procedures (94 LOC) from a toolbox:
– traditional listing in the manner of Lightspeed Pascal
– book format listing
• “section headings were highlighted, sections and
control constructs were separated by blank lines,
statements were written as sentences when
possible, procedure calls were highlighted, and
related clauses were aligned and/or chunked
together”
28/07/2017
Dr Andy Brooks
17
Figure 2. Experiment 2: Pascal Code Excerpts ©ACM
28/07/2017
Dr Andy Brooks
18
Experiment 2
Subjects/Þátttakendur
• 36 intermediate computer science students.
• Subjects randomly assigned into two groups.
– 18 subjects received program version 1
– 18 subjects received program version 2 (book format)
• The same instructions were given to everybody.
28/07/2017
Dr Andy Brooks
19
Experiment 2
Short Comprehension Test
• Ten multiple choice and short answer
questions – altogether 14 answers.
• Subjects were given ten minutes to
answer the questions.
• Subjects were also asked to rate the
readability of the code.
– 1 very poor
– 5 very good
28/07/2017
Dr Andy Brooks
20
Experiment 2
Results ©ACM
maximum test score 14
• score/time helps deal with the time/accuracy tradeoff
28/07/2017
Dr Andy Brooks
21
Experiment 2
Results & Discussion
• No statistically significant difference found in average
times.
– Version 1, 9.31 minutes
– Version 2, 8.90 minutes
• Micro-typographic style improved programmer
comprehension in terms of accuracy but not time.
– Using the book format listing resulted in 3 more correct answers
on average.
• Subjects rated higher the readability of the book format
listing.
Andy asks: What were the questions?
28/07/2017
Dr Andy Brooks
22
testing language independence
Experiment 3:
Testing Micro-Typographic Principles in C
• A reverse-Polish desk calculator program written in C.
– traditional listing taken from Kernighan and Ritchie´s
book “The C Programming Language”.
– book format listing
• “section headings were highlighted, sections and
control constructs were separated by blank lines,
statements were written as sentences when
possible, procedure calls were highlighted, and
related clauses were aligned and/or chunked
together”
28/07/2017
Dr Andy Brooks
23
Figure 4. Experiment 3: C Code Excerpts ©ACM
28/07/2017
Dr Andy Brooks
24
Experiment 3
Subjects/Þátttakendur
• 44 advanced computer science students.
• Subjects randomly assigned into two groups.
– 22 subjects received program version 1
– 22 subjects received program version 2 (book format)
• The same instructions were given to everybody.
Short Comprehension Test
• Nine questions, one with a two-part answer,
resulting in a total of 10 answers.
28/07/2017
Dr Andy Brooks
25
Experiment 3
Results ©ACM
maximum test score 10
28/07/2017
Dr Andy Brooks
26
Experiment 3
Results & Discussion
• No statistically significant difference found in average
times.
– Version 1, 9.52 minutes
– Version 2, 8.84 minutes
• Micro-typographic style improved programmer
comprehension in terms of accuracy but not time.
– Using the book format listing resulted in 1 more correct answer
on average.
• Subjects did not rate higher the readability of the book
format listing, perhaps because subjects knew the
traditional listing was from the C book by respected
authors Kernighan and Ritchie.
Andy asks: What were the questions?
28/07/2017
Dr Andy Brooks
27
Experiment 4:
Complete Book Format Listing
• A portion of the X_Windows package: 1,057
lines of C code.
– traditional listing (laser printed with pagination)
– complete book format listing
• The table of contents and index added 269 lines
of comments, but the micro-typographic
rearrangement meant the complete book format
listing was only 41 lines longer than the original
code.
28/07/2017
Dr Andy Brooks
28
Experiment 4
Subjects/Þátttakendur
• 12 professional programmers with at least two years of C
programming experience.
• Subjects were paired by experience and job function,
creating 6 pairs of subjects.
– 6 subjects received program version 1
– 6 subjects (the other members of the pairs) received program
version 2 (book format)
– within a pair, assignment was random (coin flip)
• Two subjects, highly experienced with the X_Windows
system, helped establish expert performance levels.
– subjects Xt and Xb (traditional and book)
28/07/2017
Dr Andy Brooks
Andy says: good.
29
Experiment 4
Tasks
• 30 minutes of think-aloud in which the task was
to become familiar with the program.
• Oral comprehension test
Andy asks: What
– time up to 30 minutes
– accuracy (maximum score 10)
were the questions?
• Complete an incomplete call graph
– time up to 30 minutes
– accuracy (maximum score 39)
• one point for each missing node (11) and edge (28)
• The same instructions were given to everybody.
28/07/2017
Dr Andy Brooks
30
Experiment 4
Results ©ACM maximum test score 10
t traditional
b book format
28/07/2017
Dr Andy Brooks
31
Experiment 4
Comprehension Test Results
• Small sample sizes of 6 and 6, so no statistical
tests were performed.
• For each pair of subjects, the subject working
with the book format listing had better accuracy
and took less time than the subject working with
the traditional listing.
• The subjects working with the book format listing
performed well compared to the two experts.
28/07/2017
Dr Andy Brooks
32
Experiment 4
Results ©ACM maximum score 39
t traditional
b book format
28/07/2017
Dr Andy Brooks
33
Experiment 4
Call Graph Results
• Small sample sizes of 6 and 6, so no statistical
tests were performed.
• For four pairs of subjects, the subject working
with the book format listing had better accuracy
and took less time than the subject working with
the traditional listing.
• Et presented exceptional behaviour. Et worked
the full 30 minutes tracing every procedure call.
• The subjects working with the book format listing
performed well compared to the two experts.
28/07/2017
Dr Andy Brooks
34
Experiment 4
Imperfect experimental design/
Ófullkomið tilraunasnið
“Before starting the exercise, five of the six subjects
working with the version 2 listing indicated that they
could use the index to complete the call graph
without looking at the code. They were told it was
an exercise in code reading and they were to build
the call graph from the code, not the index. They
were permitted to use the index and table of
contents only to find modules when tracing the
execution of the code they were reading.”
Andy says: experiment 4 is flawed. The book format listing had
all the necessary calling information in the index.
28/07/2017
Dr Andy Brooks
35
Andy´s View
• In Experiment 1, the Pearson chi-square test was used. This
test is approximate. Nowadays, Fisher´s exact test would used
and the resulting p value may not be less than 0.05.
– see http://www.graphpad.com/quickcalcs/index.cfm
• In Experiments 2, 3, and 4, without knowledge of the
questions actually asked in the comprehension tests, it is
difficult to form an opinion about how useful the book format
listing really is.
• I prefer experiments where subjects actually perform a
maintenance task rather than answer a comprehension test.
• The call graph exercise of Experiment 4 was flawed.
• The “statements as sentences” principle is interesting and I
personally prefer this arrangement of program text.
28/07/2017
Dr Andy Brooks
36