MSc Software Maintenance
MSc Viðhald hugbúnaðar
Fyrirlestrar 45 og 46
Essential Documentation
29/07/2017
Dr Andy Brooks
1
Case Study
Dæmisaga
Reference
A Study of the Documentation Essential to Software
Maintenance, Sergio Cozzetti B. de Souza, Nicolas Anquetil,
and Káthia M. De Oliveria.The 23rd International Conference
on Design of Communication, SIGDOC´05, pp 68-75, 2005
©ACM
29/07/2017
Dr Andy Brooks
2
1. Introduction
• Documentation continues to be “renowned for its
absence” despite the emphasis placed by educators.
• Agile software development actually questions the
importance of documentation.
– “Combined with the preference for face-to-face communication,
agile methods usually produce less written documentation than
other methods.”
• http://en.wikipedia.org/wiki/Agile_software_development
• There is pressure to re-document legacy software.
– to support ongoing maintenance or re-engineering
• All this raises the question: “What documentation would
be most useful to software maintenance?”
29/07/2017
Dr Andy Brooks
3
1. Introduction
Examining the Agile Manifesto
http://www.ambysoft.com/essays/agileManifesto.html
“Working software over comprehensive documentation.
When you ask a user whether they would want a fifty page document
describing what you intend to build or the actual software itself, what do
you think they’ll pick? My guess is that 99 times out of 100 they’ll choose
working software. If that is the case, doesn’t it make more sense to work in
such a manner that you produce software quickly and often, giving your
users what they prefer?
Furthermore, I suspect that users will have a significantly easier time
understanding any software that you produce than complex technical
diagrams describing its internal workings or describing an abstraction of its
usage, don’t you? Documentation has its place, written properly it is a
valuable guide for people’s understanding of how and why a system is built
and how to work with the system. However, never forget that the primary
goal of software development is to create software, not documents –
otherwise it would be called documentation development wouldn’t it?”
29/07/2017
Dr Andy Brooks
4
1. Introduction
Examining the Agile Manifesto
http://www.ambysoft.com/essays/agileManifesto.html
http://en.wikipedia.org/wiki/Agile_software_development
“Continuous attention to technical excellence and good design
enhances agility.
It's much easier to understand, maintain, and evolve high-quality source
code than it is to work with low-quality code. Therefore, agilists know that
they need to start with good code, to keep it good via refactoring, and take
a test-driven approach so that they know at all times that their software
works. We also adopt and follow development guidelines, in particular
coding guidelines and sometimes even modeling guidelines.”
• Specific Agile tools and techniques include continuous integration,
automated unit testing, pair programming, test driven development,
design patterns, and code refactoring.
• Ongoing code refactoring in Agile software development means that
(preventative) maintenance is a normal state of affairs.
29/07/2017
Dr Andy Brooks
5
1. Introduction
• Agile software development, however, does not adress
what happens over extended periods of time “when a
development team is sure to disperse with the
knowledge it has of the implementation details”, so
documentation is still a “highly relevant artifact”.
– teams eventually break up and take their knowledge with them
– there is a need for communication across time between the
development team and the maintenance team.
• Re-documenting legacy software or software redocumentation is about trying to make up for past
deficiencies. This is, however, a costly activity whose
expense is difficult to justify to customers.
29/07/2017
Dr Andy Brooks
6
3.1 Some documentation problems
•
non-existent or poor quality or out of date
–
•
Last Updated July 17 2002.
too much (!)
–
•
volumes of documentation are easier ignored than read
difficult to access
–
•
tables, text, and diagrams across multiple file types
lack of programmer interest in documentation
–
do you find it boring writing JavaDoc? I know I do...
•
•
http://java.sun.com/j2se/javadoc/
difficult to produce standardised documentation
–
how many developers know all of UML well? I know I don’t...
•
29/07/2017
JPEG
TIFF
PNG
GIF
BMP
...
http://www.uml.org/
Dr Andy Brooks
7
3.2 Documentation for maintenance
related work
F. A. Cioch and M. Palazzolo. A documentation suite for maintenance
programmers. In Proceedings of the 1996 International Conference on
Software Maintenance (ICSM’96), pages 286–95. IEEE,1996.
• Documentation is designed specifically for new staff
joining a software maintenance team.
• Staff begin as newcomers and progress through the
stages of student and intern until they become experts.
• Newcomers receive short contextual overviews: project
mission, product review, and module role in product.
– The overviews are like the high-level information that is found in
marketing presentations or briefings to management.
• Students study what modules do, how they work, and
how they interact with other code and the external
environment. Architecture diagrams and design stories
are used.
29/07/2017
Dr Andy Brooks
8
3.2 Documentation for maintenance
related work
F. A. Cioch and M. Palazzolo. A documentation suite for maintenance
programmers. In Proceedings of the 1996 International Conference on
Software Maintenance (ICSM’96), pages 286–95. IEEE,1996.
• Interns undergo practical training. Task-oriented
manuals are used to explain how code should be
organised, compiled, executed and tested. Interns are
not exposed to complete standards documents (e.g. The
configuration management standards) as this may
“trigger the feeling of information load”.
• Experts make use of traditional documents produced
during software development such as requirements and
design specifications.
– Traditional documents are said to provide too much information
to newcomers, students, and interns.
29/07/2017
Dr Andy Brooks
9
3.2 Documentation for maintenance
related work
Scott W. Ambler
http://www.agilemodeling.com/essays/agileDocumentation.htm
• Ambler recognises the need in Agile software
development to provide various kinds of documentation
for software maintainers: design decisions, project
overview, and system documentation.
• Decision decisions is a summary of critical decisions
regarding design and architecture made by the team
throughout development.
– Ambler advises to focus on decisions:
• which are not obvious
• which had other reasonable alternatives
29/07/2017
Dr Andy Brooks
10
3.2 Documentation for maintenance
related work
Scott W. Ambler
http://www.agilemodeling.com/essays/agileDocumentation.htm
• Project overview is a summary of:
–
–
–
–
–
system vision
tools and technologies used
build details
procedures for the backing up of data
references to the source code and available documents
• System documentation is an overview of the technical
architecture, the business architecture, and high-level
requirements. (A detailed technical architecture may be provided.)
– “System documentation is said to provide truck insurance, the
assurance that if the development team leaves, or gets hit by a
truck, that critical information about the project is left behind.”
Andy says: Ambler´s essay on Agile documentation is worth a read.
29/07/2017
Dr Andy Brooks
11
3.2 Documentation for maintenance
related work
A. Forward and T. C. Lethbridge. The relevance of software documentation,
tools and technologies: a survey. Proceedings of the 2002 ACM symposium on
Document engineering (DocEng’02) ,pages 26–33, 2002. ©ACM
• 48 respondents (mainly managers and developers)
• Question 4: How often is documentation updated when
changes occur in a software system?
1 = ‘never made’ … 5 ‘updates are made within a few days’
29/07/2017
mode/tíðasta gildi
Table 2 ©ACM
Dr Andy Brooks
12
3.2 Documentation for maintenance
related work
A. Forward and T. C. Lethbridge. The relevance of software documentation,
tools and technologies: a survey. Proceedings of the 2002 ACM symposium on
Document engineering (DocEng’02) ,pages 26–33, 2002. ©ACM
• Question 6: How often do you consult available software
documentation?
1 ‘never’ ... 5 ‘always’
• Across all 48 respondents, the most consulted document
was Specifications (average score 3.85).
• Across 12 respondents identifying one role as that of
manager, the most consulted document was
Requirements (average score 3.60).
• Across 17 respondents identifying as a developer, but
not manager, the most consulted document was
Architectural (average score 4.33).
29/07/2017
Dr Andy Brooks
13
convenience sample/hentugleikaúrtak
drop-out/brottfall
Questionnaire survey one
• The questionnaire could be answered on paper or over
the internet.
– Voluntarily and anonymously.
• Convenience (or accidental) sampling was used.
– One of the authors used his list of contacts.
• 76 software maintainers “from various parts of Brazil”
answered the questionnaire in the second half of 2004.
• Two part questionnaire: respondent details and ratings of
importance of various types of documentation.
29/07/2017
Dr Andy Brooks
14
drop out/brottfall
bias/skekkja
Can the results be generalized?
Critical commentary from Andy
• For external validity , a random sample from a population
is required. In this study, a convenience sample was
used which can introduce bias.
– Asking a professional society to distribute or publicise the
questionnaire is better, but does not in itself guarantee a lack of
bias. For example, only those who have had problems with
documentation might answer the questionnaire.
• We are not informed how many of the contacts declined
to answer the questionnaire. This is the drop-out or nonresponse rate which can be used to help judge the
results of a survey.
– If you obtain only 10 replies from a random sample of 1,000 then
it is likely some sort of bias is present.
29/07/2017
Dr Andy Brooks
15
Questionnaire survey one
Respondent details
• Position:
– manager, analyst or programmer
• Years of experience in maintenance:
– 1-3, 3-5, 5-10, >10 years
• Number of systems maintained:
– 1-5, 6-10,11-20, >20 systems
• Known approaches:
– structured analysis, object-oriented
“In Brazil, all software engineers usually know a technique called structured analysis.”
29/07/2017
Dr Andy Brooks
16
Questionnaire survey one
original questionnaire was in portugese
Ratings
• “Based on your practical experience, indicate what
importance each documentation artifact has in the
activity of understanding a software to be maintained”.
–
–
–
–
–
1 = “no importance”,
2 = “little importance”
3 = “important”,
4 = “very important”
don´t know
• There were 34 documentation artificats in total, arranged
in the following categories: requirement elicitation,
analysis, design, coding, test, and transition.
29/07/2017
Dr Andy Brooks
17
Questionnaire survey one
MER likely ERM Entity Relationship Model
Documentation artefacts
• Requirement elicitation
– Structured analysis: (1) requirements list, (2) context diagram,
(3) requirement description
– Object-oriented: (4) vision document, (5) use case diagram
– Structured and OO: (6) conceptual data model, (7) glossary
• Analysis
– Structured analysis: (8) functions derived from the requirements,
(9) hierarchical function diagram, (10) data flow diagram.
– Object-oriented: (11) use cases specifications, (12) class
diagram, (13) activity diagram, (14) sequence diagram, (15) state
diagram
– Structured and OO: (16) non functional prototype, (17) logical
data diagram (MER), (18) data dictionary
29/07/2017
Dr Andy Brooks
18
Questionnaire survey one
Documentation artefacts
• Design:
– Structured analysis: (19) architectural model, (20) general
transaction diagram, (21) components specification.
– Object-oriented: (22) collaboration diagram, (23) components
diagram, (24) distribution diagram.
– Structured and OO: (25) physical data model, (26) functional
prototype
• Coding
– Structured and OO: (27) comments in source code, (28) source
code
29/07/2017
Dr Andy Brooks
19
Questionnaire survey one
Documentation artefacts
• Test:
– Structured and OO: (29) unitary test plan, (30) system test plan,
(31) acceptance test plan
• Transition
– Structured and OO: (32) data migration plan, (33) transition plan,
(34) user manual
29/07/2017
Dr Andy Brooks
20
convenience sample/hentugleikaúrtak
drop-out/brottfall
Questionnaire survey two
• The questionnaire was answered only on paper to allow
the same person to provide answers for up to 8 different
maintenance projects.
– Voluntarily and anonymously.
• Convenience (or accidental) sampling was used.
– No effort was made to have all the first survey participants
answer the second survey.
• “some maintainers actually answered both surveys, others only the first or
only the second”
• 52 software maintainers “from various parts of Brazil”
provided answers for 237 maintenance projects.
• Two part questionnaire: respondent and system details
and actual usage of various types of documentation.
29/07/2017
Dr Andy Brooks
21
Questionnaire survey two
Respondent and system details
• Years of experience in maintenance:
– 1-3, 3-5, 5-10, >10 years
• Months of experience with the system maintained:
– 1-3, 3-6, 6-12, >12 months
• Application domain:
– banking, education, ...
• Size of the maintenance team:
– 1 person, 1 team, several teams
• System age:
– <1 year, 1-2 years, 3-5 years, >5 years
29/07/2017
Dr Andy Brooks
22
Questionnaire survey two
Respondent and system details
• System maturity:
– recently developed, mostly corrective maintenance, growing –
corrective and evolutive maintenance, stablised – few corrective
and mostly evolutive maintenance, phasing out – new product
being developed
• Documentation quality:
– complete? –yes/no, up-to-date? -yes/no, readable? -yes/no
• Approach:
– structured analysis, object-oriented
• Maintenance type realized during the survey:
– corrective, evolutive
• Defined maintenance process?
– yes,no
29/07/2017
Dr Andy Brooks
23
Questionnaire survey two
Documentation artefacts
• The documentation artefacts were as before, but
rather than one long list, questionnaires were
arranged either for structured analysis (24
documentation artefacts) or object-oriented (25
documentation artefacts.)
structured analysis (24)
29/07/2017
object-oriented (25)
Dr Andy Brooks
24
structured analysis, 70 replies
29/07/2017
Table 1 ©ACM
Dr Andy Brooks
Questionnaire survey one
25
object-oriented, 54 replies
29/07/2017
Table 1 ©ACM
Dr Andy Brooks
Questionnaire survey one
26
Survey one results
• Across both structural analysis and object-oriented, the
two most important artificats are the source code and
comments.
– “This result was expected.”
• Across both structural analysis and object-oriented, the
following were ranked highly:
– data models (circles ●)
– requirements (squares ■)
– acceptance test plan (diamond ♦)
• Across both structural analysis and objection-oriented,
unit and system test plans are ranked relatively well.
• “A big surprise” was the low rankings of general views of
the system (triangle ►).
29/07/2017
Dr Andy Brooks
27
structured analysis, 142 replies Table 2 ©ACM
29/07/2017
Dr Andy Brooks
Questionnaire survey two
28
object-oriented, 95 replies
29/07/2017
Table 2 ©ACM
Dr Andy Brooks
Questionnaire survey two
29
Survey two results
• Across both structural analysis and object-oriented, the
source code and comments are ranked very highly.
– 1st and 3rd.
• Across both structural analysis and object-oriented, data
models (circles ●) “lost the importance they had in the
first survey”.
• Across both structural analysis and object-oriented, the
test plans are well used.
• The low rankings of general views of the system “seems
partially confirmed “ (triangle ►).
29/07/2017
Dr Andy Brooks
30
Survey two results
System maturity
• Total replies = 51
–
–
–
–
3 (6%) recently developed
32 (64%) growing
14 (27%) stabilized
2 (4%) phase out
• The authors did not try to analyse survey
responses by system maturity.
– Systems that are stabilized or being phased out are
unlikely to undergo architectural changes. Are general
views less important for these systems?
29/07/2017
Dr Andy Brooks
31
Survey two results
Documentation quality
• Completeness: total=51
– complete = 14 (27%),
– not complete = 37 (73%).
• Actuality: total=51
– up-to-date = 17 (33%)
– out-of- date = 34 (67%).
• Readability: total=51
– readable = 41 (80%)
– unreadable = 10 (20%)
“The quality of the documentation is as could be expected:
generally incomplete and out-of-date.”
29/07/2017
Dr Andy Brooks
32
Survey two results
Maintenance type
• Total replies = 237
– 64 (27%) corrective
– 125 (53%) evolutive
– 48 (20%) both
• The authors did not try to analyse survey
responses by maintenance type.
– Is the scope of corrective maintenance usually so
small (e.g. fixing a method) that a maintainer does not
need to make use of general views of the system?
29/07/2017
Dr Andy Brooks
33
6. Conclusion
• “The surveys confirmed that source code and comments
are the most important artifact to understand a system to
be maintained.”
• “Data models and requirement descriptions were other
important artifacts.”
– test plans too were also relatively important
• “Surprisingly, and contrary to what we found in the
literature, architectural models and other general views
of the system are not very important.”
– To explain this, the authors’ suggest that such models providing
general views may be used once, then never consulted again.
• “Further research is needed...”
29/07/2017
Dr Andy Brooks
34
Critical commentary from Andy
• The scope of a single maintenance request can vary
from a single statement to the system in its entirety. An
explicit model of maintenance requests is, however,
absent in this work.
• If the scope is a single statement, a maintainer will need
the source code and comments.
• If the scope is the entire system, then the maintainer is
also likely to need models providing general views.
• The two surveys may have simply indirectly measured
the frequency distribution of maintenance request types.
• General views are obviously very important when
making changes to the system architecture.
29/07/2017
Dr Andy Brooks
35
http://yourdon.com/strucanalysis/wiki/index.php?title=Introduction
Structured Analysis
Data Flow Diagramming
The context diagram shows
the flow of data to and from
external entities.
Figure 0 shows the
decomposition of the system
into main sub-processes.
Figure 3 shows the
decomposition of subprocess 3.
If a maintenance request
resulted in the need for a new
flow of data from a new
external entity, the maintainer
is likely to want to view the
context diagram.
29/07/2017
Dr Andy Brooks
36