Unit Testing
CSSE 376, Software Quality Assurance
Rose-Hulman Institute of Technology
March 27, 2007
Outline
Role of unit testing
Testing strategies
Black box methods
White box methods
2
Role of Unit Testing
Assure minimum quality of units
before integration into system
Focus attention on relatively small
units
Marks end of development step
3
Testing versus Debugging
Testing
Debugging
planned
unplanned
results are
recorded
may be done by
non-developers
look for errors
unrecorded
always done by
developers
look for causes of
errors
4
Limits of Testing
Testing can never demonstrate the
absence of errors
Exhaustive testing is infeasible
Testing may be done imperfectly
5
Strategies for Unit Testing
Black box
use only specification of program
test implementation against its
specification
White box
use structure or other properties of a
program to generate tests
6
Black Box Methods
Equivalence partitioning
Boundary value analysis
7
White Box Methods
8
White Box Methods
Coverage
statement
branch
path
Cyclomatic
Dataflow
Mutation
Error Based
9
Coverage Methods
Statement
execute each statement
Branch
execute each branch
Path
execute each path
10
Statement Coverage
Execute each statement in the
program
Considered minimum criterion for
most unit testing
May be difficult to achieve for error
cases
11
Example Program
1:
2:
3:
4:
5:
6:
7:
8:
if (a < 0) {
return 0 }
r = 0;
c = a;
while (c > 0) {
r = r + b;
c = c - 1; }
return r;
12
Statement Tests
a = 3, b = 4
executes 1, 3, 4, 5, 6, 7, 5, 6, 7, 5, 6, 7, 5,
8
a = -3, b = 2
executes 1, 2
13
Branch Coverage
Execute each branch of the program
at least once
Differs from statement coverage only
for "if" statements without "else"s and
case statements without default
cases.
14
Dataflow Testing
More than branch coverage, but less
than path coverage
Uses information about references to
variables to select paths
15
Definitions and Uses
Defining node
input statement
lhs of assignment
Usage node
output statement
rhs of assignment
control statement
16
Suspicious Paths
Variable is defined (set to a new
value) but never referenced
Variable is referenced but never
defined
Variable is defined twice before it is
used
17
DU Paths
Definition-use (du) path wrt V:
a path containing a defining node for
V at the beginning a usage node for V
at the end, and no def's or use's in
between
DU testing: execute each du-path of
each variable
18
Example Program
1:
2:
3:
4:
5:
6:
7:
8:
if (a < 0) {
return 0 }
r = 0;
c = a;
while (c > 0) {
r = r + b;
c = c - 1; }
return r;
19
Example DU Paths
(corrected 3/28/07)
Def (c) = {4, 7}
Use (c) = {5, 7}
Def (r) = {3, 6}
Use (r) = {6, 8}
du-paths for c:
4 - 5, 7 – 5
du-paths for r:
3 - 4 - 5 - 6, 6 - 7 - 5 - 6,
6-7-5-8
3 - 4 - 5 - 8,
20
Test Cases for DU Paths
(corrected 3/28/07)
a=2
1-3-4-5-6-7-5-6-7-5-8
Covers du-paths:
4 - 5, 7 - 5, 3 - 4 - 5 - 6, 6 - 7 - 5 - 6
21
Cartoon of the Day
22
Mutation Testing
(As Applied to White-Box)
Path testing exercises the control of a
program, not the computations along
the paths
Most programs under test are
"almost" right
23
Mutation Method
1. Pre-process program to generate
mutants
2. Execute all the mutants and the
original program
3. If a mutant differs from the original,
then it is "killed"
4. If all mutants are killed, then test set
is adequate.
24
Mutation Metaphor
25
Example Program
function square( x: integer): integer;
begin
square := x * x
end
26
Example Mutant
function square( x: integer): integer;
begin
square := x + x
end
27
Executing Mutants
Test set {2} does
not kill the mutant
Test set {2, 4} does
kill the mutant, and
it reveals the flaw in
the program
28
Which Mutants?
Examples:
Off by one errors (i+1,
i, i-1)
Different variable (i, j,
k)
29
Assumptions of
Mutation Testing
Competent Programmer: The perfect
program is very close to the program
under test
Oracle Hypothesis: You know the
output for any input
Continuity: Every complicated
mistake can be caught by looking for
simple mistakes.
30
Problems with Mutation
 Very expensive
 each test runs the whole program
 many mutants for every program
 Some mutants fail to halt
 May be difficult to detect when
a mutant is really just an
equivalent program
 Not reliable: may not try the
right mutants
31