CHAPTER 9 - PART 2
Software Testing Strategies
Lesson Outlines
■ White box testing
– Data processing and calculation correctness tests
– Correctness tests and path coverage
– Correctness tests and line coverage
– McCabe’s cyclomatic complexity metrics
– Software qualification and reusability testing
– Advantages and disadvantages of white box testing
Classification According to Testing
Concept
• Two main philosophies of testing:
– One deals with internal structure of the software
– The other deals with produced outputs given inputs.
– IMHO, you definitely need both because they test the
software from two entirely different perspectives!
Definitions
• Black Box (Functionality) testing: Identifies bugs only
according to software malfunctioning as they are
revealed in its erroneous outputs.
– If outputs okay, BB Testing disregards internal paths
taken, calculations and any processing.
– This is somewhat of an oversimplification, but it is still
essentially true.
Definitions
• White Box (Structural) testing: Examines internal
paths program execution follows in order to identify
bugs.
– Often called ‘glass box testing’.
– Investigates the correctness of the code structure.
– Checks out the application in considerable detail – but
with a cost.
White Box and Black Box Testing for
classes of tests
• Black Box testing:
* We will look at these.
– * Output correctness tests
– * Documentation tests
– * Availability (reaction time) tests
– * Reliability tests
– * Stress tests (load tests and durability tests)
– * Software system security tests
– Training usability tests
– Operational usability tests
– Flexibility tests
– Testability tests
– Portability tests
– Software interoperability tests
– Equipment interoperability tests
White Box and Black Box Testing for
classes of tests
• White Box Testing:
– * Data processing and calculations correctness tests
– * Software qualification tests
– * Reusability tests
• Both white box and black box testing:
– * Maintainability tests
White Box Testing - Overview
• White box testing requires verification of every program statement / comment
in your code
– Can be very long and painstaking (‫!)دقيق‬
• Calculation Correctness Test Cases are developed to test each computation
and the path to the computation.
– Used to determine the proper path sequence of steps that compute.
• Software Qualification Checks out compliance and standards.
• Maintainability checks out internal structure for maintenance
• Reusability tests the reuse ability in a local software library.
White Box Testing: In more Detail:
Data Processing and Calculation Correctness
■Path coverage
•Path coverage is measured as percentage of all possible program paths
included in planned testing.
■Line coverage
•Line coverage is measured by the percentage of program code lines
included in planned testing.
Correctness Tests and Path Coverage
• Note: every path (if-then-else; do-while, etc.) is a ‘different’ path with
different values.
• We’d like to tests all paths.
• Total number of paths is countably infinite.
• If one calculates the number of paths in a non-trivial routine, the
number of test cases required is huge and very impractical (‫)غير عملي‬.
• Except for some special security applications, we will use alternative
strategies: line coverage.
Correctness Tests and Line Coverage
• Desirable to test every line of code at least once during testing.
• Define degree of coverage as the number of lines executed over the number
of executable lines available.
• Let’s consider a program flow chart to make our point.
– a diamond is a decision;
– a rectangle consists of imperatives and connections to conditional statements.
– Edges represent sequences of sections
• Let’s consider a program flow graph, with sections of code to replace one or
more flow chart rectangles.
Imperial Taxi Service Computations
■ ITS taxi fares for one-time passengers are calculated as follows:
1. Minimal fare: $2. This fare covers the distance traveled up to 1000 yards and
waiting time (stopping for traffic lights or traffic jams, etc.) of up to 3 minutes.
2. For every additional 250 yards or part of it: 25 cents.
3. For every additional 2 minutes of stopping or waiting or part thereof: 20 cents.
4. One suitcase: 0 change; each additional suitcase: $1.
5.
Night supplement: 25%, effective for journeys between 9pm - 6am.
■ Regular clients are entitled to a 10% discount and are not charged the night
supplement.
Taxi Service - Flow chart - Path Coverage
1
Charge the minimal fare
In planning, a flowchart
and a flow graph were
developed.
Note: five decisions
enter into planning for
path testing…
D > 1000
3
WT > 3
6
9
Twenty-four different
paths may be executed
to have full coverage.
Means we need to prepare
24 different test cases.
12
S >1
Yes
2
Distance
5
Waiting time
D ≤ 1000
4
WT ≤ 3
7
8
No.of suitcases
S≤1
11
Regular
client?
No
15
Yes
17
Print receipt.
14
Night
journey?
10
13
No
16
Program
flow graph
But, if we only insist on ‘line coverage,’
we can get full line coverage with three tests:
1 2 3 5 6 8 9 11 12 17
1 2 4 5 7 8 10 11 13 14 15 17
1 2 4 5 7 8 10 11 13 14 16 17
1
2
4
3
5
So, we have three test cases
compared with 24!
Savings goes up exponentially
with programs of greater
complexity.
7
6
8
9
10
11
13
12
15
14
16
17
Program
Flow Graph
McCabe’s cyclomatic complexity metrics
also give an upper limit to the number of
test cases needed for full line coverage.
McCabe used graph theory to arrive
at a minimum number of tests to
ensure full line coverage based
on a program flow graph.
Any path must have at least one
edge not in another path.
1
2
3
4
R2
7
5
6
8
10
R3
9
McCabe developed his cyclomatic
12
complexity based on ‘regions’ in the graph.
R1
11
13
R4
14
15
R5
16
McCabe – Maximum Number of Independent Paths for
Full Line Coverage.
for full line coverage
According to McCabe, the V(G) can be
computed in any of three ways each of
which is based on the program’s
flow graph:
V(G) = R // nbr of regions
V(G) = E – N + 2
edges – nodes
(21-17 + 2 = 6)
V(G) = P + 1
(nbr of decisions + 1)
(5 + 1 = 6)
R6
Result: max nbr of independent paths
Number of paths has been used to indicate
overall program complexity
1
2
3
R1
4
R2
7
5
6
8
9
10
R3
11
12
13
R4
14
15
R5
17
16
Advantages of White Box Testing
■ Direct determination of software correctness as expressed in the processing paths
statement by statement, including algorithms.
Were they correctly defined and coded??
Logic errors??
Must carefully design test cases!!
■ Allows performance of line coverage follow up.
We have software that identifies lines not yet executed are available.
percent execute, phantom paths…
■ Ascertains quality of coding work and its adherence to coding standards.
Disadvantages of White Box Testing
■ The vast resources utilized, much more those required for black box testing of the same
software package.
■ The inability to test software performance in terms of availability (response time),
reliability, load durability, etc.
■ Book cites that white box testing should be limited to modules of high risk and high cost
of failure – whatever that means!
■ After programs have been maintained over the years, these general testing strategies are
too simplistic.