A structured programming experiment
H. C. Lucas, Jr.* and R. B. Kaplant
A number of advantages have been claimed for structured programming, though most of the claims
have yet to be substantiated with experimental orfieldstudy data. The purpose of this paper is to
present the results of an experiment comparing the performance and attitudes of a group of student
programmers writing structured code with a group writing unstructured programs.
(Received October 1974)
Though there is little agreement on the definition of structured
programming, almost all writers suggest that at a minimum, it
involves avoiding the use of unconditional transfer of control
statements (GO TO's) from programs (Knuth, 1974; Mills,
1971; Dijkstra, 1968b). However, structured programming is
more than programming without GO TO's; it is an approach
to solving problems using a top-down, hierarchical method
(Dahl, et al., 1972; Dijkstra, 1968a). Unrestricted GO TO
statements are replaced in their various contexts by language
structures such as iterations, conditionals, and procedure
calls. The basic objective is to formulate programs so that
they are easily understood (Knuth, 1974).
Structured programming produces a better documented
program with greater readability. A programmer can read a
program segment from top to bottom without having to follow
through a number of transfers of control. Structured programs
should be easier to debug and understand at some later date.
It should also be easier to modify a program, especially when
the changes are made by someone other than the original
programmer. Modifications can be made to isolated blocks of
code without affecting other major segments of the program.
Based on these advantages claimed for structured programming,
the basic contention of this study is that programmers will
perform better and have more favourable attitudes when
writing structured code than when writing unrestricted
programs.
Experimental Design
The subjects in the experiment were MBA and engineering
students in a Stanford Graduate School of Business course
entitled Information Systems Technology. As a part of the
course, the students learned the PLC programming language
(an in core PL/I compiler developed at Cornell University)
and wrote three batch programs on the IBM 360/67 at the
Stanford Computation Centre.
The first problem assigned was a simple binary search of a
sorted list, while the second was a simplified sequential file
update in which only transactions were processed against
master file records. The third problem involved maintenance
programming. A standard version of the second problem was
provided to each student and the assignment was to modify the
program by extending it to add and delete records from the
master file.
Each completed program was submitted for execution by the
instructor using standard test data for grading purposes. In
addition to correct processing of the grading data, the quality
of the documentation, use of comments in the code, the student's
flowchart, source language identification, and program
readability were used to assign a grade. Students submitted all
runs plus an estimate of the time spent on the assignment.
After the submission of the third problem, the students
completed a questionnaire containing attitudinal and
background questions.
All students wrote the first program with no restrictions since
its major purpose was to familiarise the students with the PLC
language. For the second and third assignments (the update
and maintenance programs) the class was randomly divided
into two groups. A total of 32 students, 16 in each group,
completed the programming assignments in time to be included
in the analysis. A series of t tests on the background variables
showed no statistically or practically significant differences
between the two groups after the random assignment of subjects.
This means that the random assignment was successful in
controlling for programmer differences such as experience and
knowledge of different languages, etc.
The structured treatment group was not allowed to use GO TO
statements while the unstructured treatment group had no
restrictions placed on it. For the third problem, each group
received a standardised version of the appropriate structured
or unstructured program as a starting point for the modifications that were to be made. The standardised structured and
unstructured programs distributed in class were written by the
same programmer one month apart and an attempt was made
to be sure the two programs were equivalent.
Performance in the experiment is measured by the number of
runs and estimates of time required by the programmer along
with program compile time and object code storage requirements. Attitudes derived from the questionnaire include ratings
of the enjoyment of each assignment and ease of writing and
debugging the programs.
The experimental design is biased against structured programming because the only condition differing between the two
experimental groups is the use of the GO TO statement.
Unfortunately, the class schedule made it impossible to provide
special instructions for the structured programming treatment
group. The entire class, however, was exposed to DO WHILE
and procedure call statements, though the structured group
was not given specific alternatives to the proscribed
GO TO statement or extra exposure to top-down problem
solving strategies. The treatment group was forced to learn a
new structured approach to programming by the restriction
placed on them.
Any laboratory experiment suffers from a lack of realism. A
field study does offer more realism than an experiment, but
would be extremely difficult to design. There are limited
opportunities in a production environment for multiple programmers to be randomly assigned to groups and asked to
write the same program under differing requirements! Also,
many of the advantages claimed for structured programming
are important for long-lived programs that will undergo many
*Associate Professor of Computer Applications and Information Systems, Graduate School of Business Administration, New York University,
100 Trinity Place, New York, NY10006, USA.
jPh.D. Candidate in Computer and Information Systems, Graduate School of Business, Stanford University, Stanford, California 94305, USA.
136
The Computer Journal
Table 1 Structured and unstructured group results for the update and maintenance programs
Maintenance program
Update program
Structured
Number of runs
14-63
Programming time 14-75
Compile time 2
1-27
Bytes object code 2 6,690
Easy to write 1
400
Easy to debug 1
400
Enjoyment1
4-63
Unstructured
t
2-31 +
8-87
11-69
1-34
1-13
1-51
5,943
1-25
-l-89 +
519
4-69
-101
4-88
-0-46
+P<
+
Structured
1006
Number of runs
Programming time 9-13
0-52
Compile time 2
Bytes object code 2 2,462
5-13
Easy to write 1
4-75
Easy to debug 1
3-44
Enjoyment 1
Unstructured
10-25
1306
0-75
4,569
4-63
4-63
4-31
t
-012
-l-83 +
-3-27+ +
-3-86+ +
0-96
019
-1-42
005 3
P< 001
n = 16 for each group
+
'Higher numerical results mean easier to write and debug and greater enjoyment.
'For the maintenance program, the compile time and bytes of storage of the original program the students received to modify were subtracted from each
student's results. The data represent the changes made to the programs.
"One-tailed probability.
modifications over time. In the classroom environment, the
emphasis is on completing an assignment with a working
program. This setting also makes the experiment more conservative since the advantages for structured programming
might not appear given student incentives and the short, 12
week time span of the class. Due to these problems, the results
should be regarded as exploratory.
Results
The data from the experiment were used to test three hypotheses about structured programming and one commonly held
hypothesis on the importance of experience in programmer
performance.
Hypothesis 1: The unrestricted programming group will outperform the structured group and will have more favourable
attitudes on the update program*
Normally one would predict the opposite of Hypothesis 1, that
is, the structured programming group should out-perform the
unrestricted group. However, as mentioned above, it was not
possible to provide the structured group with special instructions or alternatives to the GO TO. Thus, a drop in performance
was anticipated when programmers in the structured programming group were first exposed to this new approach without
special preparation.
The data to test Hypothesis 1 are shown in Table 1. There are
only two statistically significant differences. The average number
of runs and the average difficulty of writing the update program
were greater for the structured group than for the unstructured
group, supporting Hypothesis 1.
Hypothesis 2: The structured programming group will outperform the unstructured group and have more favourable
attitudes in completing the maintenance programming
assignment.
One of the major advantages of structured programming is in
the ease of making changes to the resulting code. Since the
students in the structured group had some experience by now
and because the maintenance assignment involved changing a
program that was already structured, the structured group was
expected to out-perform the unrestricted group.
The results in Table 1 support this hypothesis for the perfor-
mance variables. The means for the structured group on the
time spent modifying the program, program compile time and
program storage requirements were significantly lower than the
means for the unstructured group.t
Hypothesis 3 : Students in the structured group should improve
more in performance and attitudes between the update and the
maintenance program than students in the unstructured group.
The same reasoning as behind Hypothesis 2 applies here. Both
learning and the advantages of structured programming for
changing code should produce greater performance improvements from the update to the maintenance assignments for the
structured group, especially given its initial low performance on
the update program.
The results in Table 2 strongly support Hypothesis 3. These
data are a comparison of the differences in performance
between the two groups. The results for the maintenance
program are subtracted from the results for the update programj
and these difference scores are compared for the structured
and unstructured groups. The structured group improved
more on the number of runs, programming time required and
the rating of programming ease compared with the unstructured
group.
Table 2 Structured and unstructured group changes between
the update and maintenance programs: Mean difference
update program—maintenance program
Structured
4-56
Number of runs
Programming time
5-63
-113
Easy to write 1
-0-75
Easy to debug 1
Enjoyment 1
1-19
t
Unstructured
— 1-56
2-58*
3-49+ +
-1-38
-2-52*
0-56
-1-26
006
105
0-56
*p < 0-052
p <001
n = 16 for each group
++
•Higher numerical results mean easier to write and debug and greater
enjoyment.
!
One-tailed probability.
•The authors are grateful to the referee whose suggestions led to significant changes in the reasoning behind and statement of the first
hypothesis.
fThe compile time and object code storagefiguresfor this assignment were adjusted by subtracting the compile time and the object code size
of the standard program given to each student for modification. The resulting figures in Table 1 represent the changes introduced by the
modifications.
JThe mean enjoyment rating for the update program was 4-75 vs. 3-88 for the maintenance program; t = 2-95; p < 001.
Volume 19 Number 2
137
It is also interesting to note that when both the structured and
unstructured groups are combined the program modification
assignment was rated as significantly less enjoyable than the
assignment to write the update program. These results tend
to confirm the observation that most programmers prefer to
write their own program rather than modify someone else's
code! This evidence suggests that management should consider
rotating programmers between development and maintenance
programming periodically.
Hypothesis 4: Prior programming experience will be positively
associated with high performance and favourable attitudes for
both groups in the experiment.
The data presented in Table 3 generally support the frequent
management practice of recruiting and rewarding programmers
based on experience. The total number of runs and total
programming time for both update and maintenance programs
are negatively related to years of experience, the number of
programs written and the number of languages used. These
experience factors are also positively correlated with average
programmer ratings of the ease of writing and debugging the
programs. It is interesting to note that none of the experience
factors is significantly related to the enjoyment of the programming task or to the compile time or object code requirements of
the programs.
Implications
This experiment represents an initial attempt to evaluate
structured programming. Because of the classroom environment, the experimental design suffers from certain defects.
First, students in the experimental group were not adequately
taught structured programming; they had to realise for
themselves that DO-WHILE and procedure calls could handle
most GO TO situations. Secondly, some of the gains by the
structured group between the update and maintenance programs are probably due to the relative disadvantage of the
structured programming group in writing the update program.
Even with these limitations the results of the experiment
provide some support for the advantages claimed for structured
programming, especially for the ease of modifying structured
programs and the efficiency of the resulting code. These results
are particularly important given the conservative design and
exploratory nature of the experiment, which, if anything, was
biased against structured programming. The data suggest that
programmers should be trained to write structured programs,
particularly when their past experience is primarily with unstructured programming techniques. Based on the experimental
results and class comments, several training sessions are recommended when structured programming is adopted by an
organisation. It is especially important to teach the hierarchical
approach associated with structured programming and point
out useful language elements in writing structured programs.
Management should also expect to see learning progress at a
fairly rapid rate for structured programming groups.
There is currently much interest in treating programming as a
group activity in which various programmers read each others'
code and suggest modifications (Baker, 1972 a, b; Mills, 1971;
Weinberg, 1971). Given this experimental support for the ease
of modifying structured programs, management should seriously consider implementing structured programming concomitantly with a change to programmer teams and 'egoless
programming' (Weinberg, 1971).
While experience is important, it explains less than 20 per cent
of the variance in performance and attitudes for the groups
involved in the experiment.* It is unlikely that programming is
such a craft-like trade that only experience affects performance.
Management actions like the adoption of structured programming can contribute to programmer productivity.
Table 3 Pearson correlations of performance and attitudes
with background variables
Total1
Number of
Runs
Years experience -0-30*
Number of
programs written -0-15
Number of file
programs written -0-13
Number of
languages used
-0-07
Total1
Easy2 Easy1
Programming to
to
Time
Write Debug
-0-34*
0-30* 0-42+ +
-0-38*
0-17
0-36*
-0-17
0-13
0-25+
-0-37*
0-56 ++ 0-37*
. V < 010
*p < 0-05
++
p < 001
n = 32
1
For
both
programs
combined.
2
Higher numerical results mean easier to write and debug. (The results for
each program were averaged to form a single score.)
*The square of the Pearson correlation coefficient between two variables in Table 3 represents the percentage of the variance in one variable
explained by the other.
References
BAKER, F. T. (1972a). Chief Programmer Team Management of Production Programing, IBM Systems Journal, Vol. 11, No. 1, pp. 56-73.
BAKER, F. T. (1972b). Systems Quality Through Structured Programming, AFIPS Conference Proceedings, Vol. 41, Part I.
DAHL, O. J., DIJKSTRA, E. W., and HOARE, C. A. R. (1972). Structured Programming, London: Academic Press.
DIJKSTRA, E. W. (1968a). The Structure of'THE' Multiprogramming System, Communications of the ACM, Vol. 11, No. 5, pp. 341-346.
DIJKSTRA, E. W. (1968b). GO TO Statement Considered Harmful, (Letter to the Editor), Communications of the ACM, Vol. 11, No. 3,
pp. 147-148.
KNUTH, D. (1974). Structured Programming with GO TO Statements, Computing Surveys, Vol. 6, No. 4, pp. 261-302.
MILLS, H. D. (1971). Chief Programmer Teams: Principles and Procedures, International Business Machines Corporation, FSC 71-5108,
June 1971.
WEINBERG, G. M. (19?1). The Psychology of Computer Programming, New York: Van Nostrand Reinhold.
138
The Computer Journal