Control Structure Testing
Some Alternative Techniques
Copyright © 2016 Curt Hill
Introduction
• The purpose of the Cyclomatic
Complexity metric was to:
– Put an upper bound on basis path
needed tests
– Give a measure of module complexity
• There are other methods for
determining tests
– These all revolve around control flow
• This presentation considers some of
them
Copyright © 2016 Curt Hill
Motivation
• Basis path testing is satisfied that
every statement executed
– This may not be sufficient
• Control flow is one of the places
bugs hide
– By examining closely all of these we
should find a host of other bugs
• Some of the alternatives to basis
path testing should cover those test
cases and provide others that are
useful
Copyright © 2016 Curt Hill
Condition Testing
• The focus of this approach is to
examine predicates
• We look specifically at relational
operations (comparisons) and
Boolean operators
• Thus, in C we only examine the
parentheses following an if, while or
switch
• These are the units that drive all of
the control flow
of
the
module
Copyright © 2016 Curt Hill
Boolean/Relational Errors
• The Boolean result that drives our
control flow may have several
possible errors
• These may be in a:
– Comparison
– Boolean operator
– Arithmetic
• We then generate test cases looking
for the possible errors
Copyright © 2016 Curt Hill
Error Types
• Arithmetic expression error
• Relational operator error
• Usually an equal left out or put in, but
sometimes a reversal of less/greater
• Boolean operator
– Wrong, extra or missing operator
• Wrong Boolean variable
• Precedence error
– Wrong or missing parenthesis
Copyright © 2016 Curt Hill
Test Case Generation
• We set up the test data considering
these errors
• When done systematically we
generally get more complete
coverage than basis path testing
• The computation of Cyclomatic
Complexity or the construction of a
flow graph is not needed
Copyright © 2016 Curt Hill
Branch Testing
• A form of condition testing
• For every Boolean affecting flow we
must generate tests that force the
Boolean to be true and false
• Consider:
if(a==b && c<d)
then four tests are required – each
comparison to be true or false in
every combination
• An if or while with N conditions
needs 2N tests
Copyright © 2016 Curt Hill
Domain Testing
• Includes branch testing and but
increases the test cases for
comparisons
• For any comparison such as a>b we
should test three times
• a<b
• a=b
• a>b
– This should find relational errors
Copyright © 2016 Curt Hill
Combinatorics
• It does not take too many decision
statements to make the number of
tests large
• Whenever we can we do wish to
reduce the number of tests without
reducing effectiveness
• We typically do this by limiting the
tests in a way similar to short circuit
evaluation
Copyright © 2016 Curt Hill
Limits
• Suppose we have an if with a
condition of:
a && b
where a and b are comparisons of
Boolean variables
• We would normally expect to do 4
tests
• We may be able to omit the test
where both a and b is false without
losing effective coverage
Copyright © 2016 Curt Hill
Example
• Consider the following:
if(x>4 || y < 2)
• Each variable > = <
• We could generate 9 tests
–
–
–
–
–
–
–
–
–
(x = 5, y = 1) => t, t, t
(x = 5, y = 2) => t, f, t
(x = 5, y = 3) => t, f, t
(x = 4, y = 1) => f, t, t
(x = 4, y = 2) => f, f, f
(x = 4, y = 3) => f, f, f
(x = 3, y = 1) => f, t, t
(x = 3, y = 2) => f, f, f
(x = 3, y = 3) => f, f, f
Copyright © 2016 Curt Hill
Data Flow Analysis
• In data flow testing we consider the
definition and access of variables
along a path
• A definition of a variable is any
statement that sets the variable to a
new value
• A use is any statement that access
the value of the variable
– Some distinguish between use in a
predicate and use in a computation
Copyright © 2016 Curt Hill
Continued
• We are now interested in definition
use chains for a variable
– AKA DU chain
• A DU chain is a path of statements
starting with a definition and ending
with a use
• A DU chain may contain other
chains since a variable may be used
several times along the path
• One strategy is to cover every DU
chain in a program
Copyright © 2016 Curt Hill
Potential Errors
• We would like Data Flow testing to
find these kinds of errors:
• A variable that is defined but never
used in a significant way
• A variable that is used but has no
valid initialization
• A variable that is defined multiple
times before it is used
• A dynamic variable destroyed
before it is used
Copyright © 2016 Curt Hill
Process
• We typically construct a flow
diagram
• We then decorate this flow diagram
with letters indicating how variables
are handled:
– d indicates a definition
– u indicates a usage
– k indicates killing the variable
• Such as a delete on a pointer
– ~ indicates we do not care about prior
or trailing actions
Copyright © 2016 Curt Hill
Example
• Let’s consider calculation of a bill
• There are three major cases based
on the (integer) amount of usage
Usage
Bill amount
< 100
40.00
101 - 200 40 + .5 for every additional
unit
> 200
90 + .1 for every additional
unit
Copyright © 2016 Curt Hill
Code
double CalculateBill (int Usage){
double Bill = 0;
if(Usage > 0)
Bill = 40;
if(Usage > 100){
if(Usage <= 200)
Bill = Bill + (Usage - 100) * 0.5;
else {
Bill = Bill + 50 + (Usage - 200) * 0.1;
if(Bill >= 100)
Bill = Bill * 0.9;
} // else
} // outer if
return Bill;
}
Copyright © 2016 Curt Hill
Flow Diagram (Bill)
1
D
2
D
3
4
5
7
U
D
8
U
U
D
10
Copyright © 2016 Curt Hill
U
D
6
K
11
9
Paths
Type
Path
1
D
DD
1,2,3
Double
definition
DU
3,4,5,
6
Normal
UD
6
Normal
UK
10,11
Normal
UU
7,8
Normal
2
D
3
4
5
7
U
D
U
U
D
U
D
6
8
10
Copyright © 2016 Curt Hill
K
9
11
Path Combinations
• ~D, DU, UU and UK are normal
– Most others are suspicious or errors
– A few are syntax errors
• DD is a double definition
– May indicate but not guarantee an error
• DK or D~ defined without any use
• ~U indicates use before definition
• KU and KK indicates an error
– Dangling pointer or syntax error
Copyright © 2016 Curt Hill
Example
• Now we look at a bad routine
• It has a number of undesirable path
combinations
• Consider the next three screens
Copyright © 2016 Curt Hill
int fn(AClass * R){
1 AClass * P, * Q = new AClass();
2 if(R != NULL)
3
P = R;
4 if(Q->OK())
5
P->Fix();
6 if(Q->OK())
7
Q = new AClass(*R);
8 if(P->OK()){
9
delete P;
10
P = NULL;
11
}
12 AClass * T = Q;
13 Q->Fix();
14 R->Mod(*Q);
15 delete P;
Copyright © 2016 Curt Hill
16 delete Q;
int fn(AClass * R){
1 AClass * P, * Q = new AClass();
2 if(R != NULL)
3
P = R;
1, 2, 3
4 if(Q->OK())
5
P->Fix();
1,2,4,5
6 if(Q->OK())
7
Q = new AClass(*R);
8 if(P->OK()){
3,4,6,9
9
delete P;
10,15
10
P = NULL;
11
}
12 AClass * T = Q;
13 Q->Fix();
14 R->Mod(*Q);
15 delete P;
Copyright © 2016 Curt Hill
16 delete Q;
P
~D
~U
DK
KK
int fn(AClass * R){
1 AClass * P, * Q = new AClass();
2 if(R != NULL)
3
P = R;
1, 4
4 if(Q->OK())
5
P->Fix();
6 if(R->OK())
1, 6, 7
7
Q = new AClass(*R);
8 if(P->OK()){
13, 16
9
delete P;
10
P = NULL;
11
}
12 AClass * T = Q;
12, 17
13 Q->Fix();
14 R->Mod(*Q);
15 delete P;
Copyright © 2016 Curt Hill
16 delete Q;
Q
DU
DD
UK
T
D~
Test Generation
• Data flow analysis should lead us to
develop test cases
• Often the flows give us something
better than basis path testing with
fewer tests than every path
– A path is only interesting if data flow
suggests variable interaction
Copyright © 2016 Curt Hill
Static Analysis
• Data flow analysis is a form of static
analysis
– We know that static analysis cannot tell
us the whole story
• There may be error paths that are
not possible to actually execute
• Still any assistance they give will be
helpful
Copyright © 2016 Curt Hill
Loop testing
• The whole point of computer
software is looping
– It is the ability to repeat some action
very many times that makes programs
useful
• Unfortunately, bugs like to hide in
and around loops
• Loop testing is a white box
technique to exterminate some of
those bugs
Copyright © 2016 Curt Hill
Common Loop Errors
• The most common loop error is the
off by one error
– We want to go through M times and
instead went through M-1 or M+1
– Usually the condition has an extra
equal or needs an equal or reversed
condition
• Uninitialized loop control variable
– Often skips the loop
• Loop control variable not changed in
a loop
– Usually gives
an infinite loop
Copyright © 2016 Curt Hill
Common Loop Errors
• Of those only the off by one has
much of a chance of making it to unit
testing
– The others are usually detected in
initial or ad hoc testing
• Therefore we need tests that will
detect the off by one loop
Copyright © 2016 Curt Hill
Loop Structures
• There are about four categories of
loop
– The approach depends on the category
•
•
•
•
Simple
Nested
Concatenated
Unstructured
Copyright © 2016 Curt Hill
Simple Loop
• Just what you might think
– One loop construct
– Single entry, single exit
– Not in or adjacent to another loop
• From a testing viewpoint you want
tests that exercise:
– Skipping the loop completely
• If that is possible
– Executing the loop once
– Normal loop counts
• Looking for off by one
Copyright © 2016 Curt Hill
Nested
• A loop in a loop
– Perhaps several deep
• Strategy:
– Minimize outer loops and test the
innermost loop like before
– Work outward until outermost loop is
tested
• Combinatorics may prevent us to
applying the simple loop strategy
Copyright © 2016 Curt Hill
Concatenated Loops
• Two loops in succession
• Independent loops can just use
simple loop strategy
• Related loops have something in
common – often the control variable
– These need a variation of the nested
loop strategy
– Verify the coupling between the two
loops is properly handled
Copyright © 2016 Curt Hill
Unstructured
• Not the problem it once was
• Characterized by either multiple
entrances or multiple exits
• Multiple entrances
– Typically a GOTO has bypassed the
loop initialization – hard to do otherwise
• Multiple exits
– Done with GOTOs, breaks, throws and
returns
Copyright © 2016 Curt Hill
Unstructured Again
• The multiple entrance, multiple exit
loop is the most insidious control
structure known
• Extremely hard to analyze and test
• Don’t do it
– Rewrite into something structured
• The multiple entrances are much
worse than multiple exits
– Breaks are most benign
– GOTOs are the worst
– Returns are somewhere in between
Copyright © 2016 Curt Hill
Finally
• Black box and white box testing
complement each other
– Use both
• Often a black box test will cover one
or more white box tests
– For time’s sake eliminate the redundant
tests
Copyright © 2016 Curt Hill