Continuing Best Practices
► Slide
information taken in large part from
former Rational Corporation slides.
Considerably modified and supplemented
for classroom use.
21
1
Practice 5: Verify Software Quality
Develop Iteratively
Manage
Requirements
Use
Component
Architectures
Model
Visually
Verify
Quality
Control Changes
Quality, as used within Rational Unified Process, is defined as
“The characteristic of having demonstrated the achievement of producing a product
which meets or exceeds agreed upon requirements as measured by an
agreed upon measures and criteria And is produced by an agreed upon process.
If done this way, the process can be repeated and managed 
In most organizations, testing accounts for 30-50%21of development costs! Yet most people believe
software is not adequately tested when delivered.
Testing is difficult; complete testing is impossible; a good process and automated tools help!
2
Practice 5: Verify Software Quality
Software problems are 100 to 1000 times more
costly to find and repair after deployment
Cost
Development
Deployment
One of the ways to improve quality: Test early and continuously!
21
Test functionality, reliability; performance;
Test architectural decisions early. 3
Iterative Development Permits Continuous
Testing – ensuring higher Quality
Iteration 1
Iteration 2
R
R
D
R
D
D
C
T I M E
Test
Life
Cycle
Iteration 3
C
C
T
T
T
Test
Test
Test
Plan
Design
Implement
Execute
Assess
Plan
Design
Implement
Execute
Assess!
21
Plan
Design
Implement
Execute
Assess!
4
Verifying Quality – and Continuous Testing
(continued)
Rather than test one time, spread testing out continuously.
 Part of each iteration – BUT (see below)
► Each iteration produces an executable release (not necessarily
a product release…)
►
 Don’t think of an ‘executable’ as just an .exe or .dll. The executables
may be part of an architecture……
 Each iteration is tested and integrated into an evolving application.
►
►
►
Note: Each ‘phase’ has one or more iterations, and each phase
has milestones!
Be careful: The ‘extent (how much)’ of R, D, C, T depends on
which phase the iteration is in!
(See your drawings of the RUP)
21
5
More…recall – talking about Verifying Quality
Cannot ‘engineer in’ quality; Must be threaded throughout
development!
► Notice: Continuous Testing and integration!
 Distributes testing….ensures higher quality
 A fundamental design concept: Divide and Conquer!
► At end, entire system tested as a whole.
► Many errors can be found early; fixed while repair costs are
inexpensive
► Architectural decisions (key decisions) tested early avoiding
disastrous problems later.
► These features greatly reduce risks and liability of delivering
poor quality systems.
►
21
6
Requirements
Testing in an Iterative Environment
Iteration 1
Iteration 2
Iteration 3
Iteration 4
Continuous integration!!! (one of the major problems of SDLC!)
Automated Tests
We will produce automated tests (IF AVAILABLE in your ‘environment’)
As new requirements are added in iterations, new tests are generated and run.
This means that some tests will be rerun – part of ‘Regression Testing.’
Test Suite 1
Test Suite 2
21
Test Suite 3
Test Suite 4
7
Quality – What is it?
► It
is an elusive term
► Means different things to different people
21
Next few slides from OOSE text slides; modified
8
Software Quality...
►Usability
- Users can learn it fast and get their
job done easily (usually addressed in Interface)
►Efficiency - It doesn’t waste resources like CPU
time and memory (addressed in execution)
►Reliability - It does what it is required to do
without failing (MTBF, MTTR…)
►Maintainability - It can be easily changed
►Reusability - Its parts can be used in other
projects, so reprogramming is not needed
21
9
Software Quality...again, means different
things to different stakeholders
Customer:
solves problems at
an acceptable cost in
terms of money paid and
resources used
User:
easy to learn;
efficient to use;
helps get work done
QUALITY
SOFTWARE
Development manager:
sells more and
pleases customers
while costing less
to develop and maintain
Developer:
easy to design;
easy to maintain;
easy to reuse its parts
21
10
Software Quality
►The
different qualities can conflict.
Increasing efficiency can reduce maintainability or
reusability
 Increasing usability can reduce efficiency
  Increasing usability can reduce maintainability
  Increasing maintainability can reduce efficiency, etc!
►Setting
objectives for quality is a key
engineering activity
 Then design to meet these objectives
 Avoids ‘over-engineering’ which wastes money
 Obtain the highest possible reliability
using a fixed budget 11
21
Short Term Vs. Long Term Quality
►Short
term:
 Does the software meet the customer’s immediate needs?
 Is it sufficiently efficient for the volume of data we have
today?
►Long
term:
 Maintainability
 Customer’s future needs
 Scalability
21
12
Dimensions of Software Quality
Type
Why?
How?
Functionality
Does my app do what’s
required?
Test cases for each
scenario implemented
Reliability
Does my app leak memory?
Analysis tools and code
instrumentation
Application
Performance
Does my app respond
acceptably?
Check performance for
each use-case/scenario
implemented
System
Performance
Does my system perform
under production load?
Test performance of all usecases under authentic and
worst-case load
For each iteration, do the ‘above’ software quality checks.
Remember: tests are ‘driven’ by Use Cases and Supplementary Specifications!
21
13
Problems Addressed by Verifying Quality
Root Causes
Solutions
Insufficient requirements
Ambiguous communications
Brittle architectures
Overwhelming complexity
Subjective assessment
Undetected
inconsistencies
Testing provides objective
project status assessment

Poor testing
Waterfall development
Uncontrolled change
Defects are found earlier and
are less expensive to fix
(because ‘testing’ is
distributed…

Insufficient automation








Objective assessment exposes
inconsistencies early
(continuous integration helps!)
Testing and verification are
focused on high risk areas
Automated testing tools
provide testing for reliability,
21
functionality, and performance 14
Practice 6: Control Changes to Software
Develop Iteratively
Manage
Requirements
Use
Component
Architectures
Model
Visually
Verify
Quality
Control Changes
Must recognize that we CANNOT STOP CHANGE. Our goal is to Manage Change!
The only ‘constant’ is ‘change!’
Must control How and When control is introduced and who introduces the changes.
This DOES NOT MEAN that we absolutely accept ALL changes, But…(Discuss!)
Want a process that can respond to change…(RUP)
21
15
Must synchronize Change across development
teams and locations too.
(What impacts do proposed changes have on our architecture!)
Practice 6: Control Changes to Software
► Consider:







Multiple
Multiple
Multiple
Multiple
Multiple
Multiple
Multiple
we often have:
developers
teams
sites
iterations
releases
projects
platforms
All at the same time!
May have multiple developers organized into different teams at multiple sites all working
together on multiple iterations, releases, products, and platforms
(mostly based on the software architecture)
Without explicit control, parallel development
degrades to chaos!!!!
21
16
Problems Addressed by Controlling Changes
Root Causes










Insufficient
requirements
Ambiguous
communications
Brittle architectures
Overwhelming
complexity
Subjective assessment
Undetected
inconsistencies
Poor testing
Waterfall development
Uncontrolled change
Insufficient automation
Solutions
Requirements change workflow
is defined and repeatable
Change requests facilitate clear
communications
Isolated workspaces reduce
interference from parallel work
Change rate statistics are good
metrics for objectively assessing
project status
Workspaces contain all artifacts,
facilitating consistency
Change propagation is
controlled
Changes maintained in a robust,
21
19
customizable
system
Best Practices Reinforce Each Other
What does iterative
development do??
Ensures users involved
as requirements evolve
Manage
Requirements
Use
Validates architectural
Component
decisions early on. Drives development,
Architectures
planning, change control. ….
Develop
Iteratively
Addresses complexity of
design/implementation incrementally.
Need tools/support environment!
Measures quality early and often.
Continuous testing and integration
Evolves baselines incrementally
Architecture  teams  localizing
changes; Need CMS, Conf Control…
Model
Visually
Verify
Quality
Control
Changes
20
Remember: best practices yield best21results WHEN USED COLLECTIVELY!
Summary: Best Practices of Software
Engineering
The result is software that is
 On Time
 On Budget
 Meets Users Needs
Performance
Engineer
Analyst
Project
Manager
Develop Iteratively
Developer
Use
Manage
Component
Requirements Architectures
Model
Visually
Verify
Quality
Tester
Control
Changes
21
Release
Engineer
21