Category-partioning and
CatGen
www.eu-etics.org
INFSOM-RI-026753
Category-partition test planning
• Originated by Ostrand and Balzer
• Lots of related publication, but quite a few
application in practice
• The original method is limited, 4D SOFT has
extended it
• Can be automated in part
• We use it frequently for different projects
and companies
INFSOM-RI-026753
Key philosophy
• The key philosophy of category-partitioning
is to make a test specification instead of
writing test cases
• The test specification is formal, can be
checked and automatically transformed into
test frames or test cases
• Test specification is very flexible and can be
easily modified
• It can be used for all type of testing
INFSOM-RI-026753
Planning functional test cases
using category-partition method
• The first step of category-partitioning is to
define the main functionalities
• Atomic, basic functionality is obtained
after gradual decomposition of the main
functionalities
• Basic functionality usually corresponds to
use cases
• Formal test specification should be done
for all the basic functionalities
INFSOM-RI-026753
Categories
• The next decomposition step is to define the basic
functionality's categories
• It is difficult to define a category. A category is a
particular characteristic of basic functionality that can
be extracted from the documents.
• If reading the related document the tester finds parts
that are not covered by any category, something is
missing.
INFSOM-RI-026753
Example
• The input for a sorting algorithm is the type, the size
and the elements of the array to be sort.
• The output is the sorted array and the min, max
element.
• Categories:
–
–
–
–
–
INFSOM-RI-026753
the size of an array,
the type of elements,
The original arrangement of the elements
The size of the elements
the place of min and max elements in the array.
Choices
Sorting out categories into choices
•
•
•
Choices are characteristics of a category
The choices reflect the often-occurred errors and the experience of
the tester
Example: in the case of a sorting algorithm the choices of the array
size category can be:
–
–
–
–
•
size = 0
size = 1
1 < size < 100
size > 100
Original arrangement:
– Sorted
– Inverse sorted
– random
INFSOM-RI-026753
Example test spec.
•
Categories and their choices are written in a formal test specification:
FUNCTION SORT
Number_of_elements NUMBER:
zero,
one,
more,
large.
Orig_Arrangement NA:
sorted,
inverse,
random.
…
FEND
– The number of test cases obtained (all choices of all categories are multiplied)
is large because there are no constraints of applicability between them
INFSOM-RI-026753
Binding among choices
The next step is to define the binding and relation
between choices
• The test engineer defines the relation between
different choices and adds this information into test
specification
• Typical relations can be:
– Two choices together are not possible
– One choice should be chosen if an other one is chosen
– A choice tests an error or an extreme case and thus should not
be combined with other choices
• As a result, a more suitable test case set is obtained
INFSOM-RI-026753
Real examples
Example:
– Find
INFSOM-RI-026753
Test frames
Generation of test frames
• After defining all the categories, choices and bindings
between them the final version of test specification is ready
• Test specification can be transformed into test frame
automatically or manually – automation is advisable
• A test frame contains the generated test cases, but it
contains choices instead of actual values.
• Each test frame correspond to a test case
INFSOM-RI-026753
category-partitioning - advantages
• Test cases are reduced and not building up (top-down
approach). It is more probable not to left out important
test cases
• It is known exactly what test cases are left out because of
the budget and other constraints. We have a clear view
about the quality
• A test case hierarchy is derived from subsequent versions
of test specification
• It can be used for almost all type of software and all
testing phase
INFSOM-RI-026753
category-partitioning - advantages
• At the same time, the method forces to walk through the
software requirements specification, design documents,
etc.
• The method is flexible. If the requirements change it will
be easy to modify the testing plan.
• Testers save time since the specification is a more
compact form. Testers also can check the number of test
cases easily
• Learning curve is minimal
INFSOM-RI-026753
Slicing and Indus
www.eu-etics.org
INFSOM-RI-026753
Program Slicing
– Analysis technique introduced by Mark Weiser (1979)
– Well known analysis that can be used to identify parts of the
program that are influenced by or do influence a given set of
program points
– Widely applied technique for
– Program Debugging: that’s how slicing was discovered
– Testing: reduce cost of regression testing after modifications (only
run those tests that needed)
– Integration: merging two programs A and B that both resulted from
modifications to Base
– Program Understanding
– Reverse Engineering: comprehending the desing by abstracting out
of the source code the design decisions
– Software Maintenance: changing source code without unwanted
side effects
– Software Quality Assurance: validate interactions between safetycritical components
INFSOM-RI-026753
Program Slicing
– Informally, a slice provides an answer to the question „What
statements potentially affect the computation of variable V at
statement S?”
– Slicing reduces program to statements relevant for partial
computation (irrelevant statements are deleted)
– A slice S includes all program statements affecting variables V at
position n in program P
– E.g. the slicing criteria S(a, 12) is a slice including all statements
affecting the value of a in line 12
INFSOM-RI-026753
Slicing Classifications
Many different variants of program slicing exist. E.g.
• Types of slices
– Full
– Data
– Control
• Direction of slicing
– Backward
– Forward
• special slice
– Executable
– Chopping
– Complete
• Levels of slices
– Intraprocedural
– Interprocedural
INFSOM-RI-026753
Forward and Backward Slicing
Forward Slicing
Contains all those statements of
P which are affected by S(V,n)
1 main( )
2{
3 int i, sum;
4 sum = 0;
5 i = 1;
6 while(i <= 10)
7 {
8
Sum = sum + 1;
9
++ i;
10 }
11 Cout<< sum;
12 Cout<< i;
13 }
Slicing Criterion: =(sum, 3)
INFSOM-RI-026753
Backward Slicing
Contains the statements of the
program P which may have some
effect in the slicing criterion S(V,n)
1 main( )
2{
3 int i, sum;
4 sum = 0;
5 i = 1;
6 while(i <= 10)
7 {
8
Sum = sum + 1;
9
++ i;
10 }
11 Cout<< sum;
12 Cout<< i;
13 }
Slicing Criterion: Cout(i, 12)
Intra- and Interprocedural Slicing
Intraprocedural Slicing
– Computes slice within one
procedure
– Assumes worse case for
functions calls
Interprocedural Slicing
Consists basically of two
steps:
1.
2.
A single slice of the procedure containing
the slicing criterion is made
Procedure calls form wihtin this
procedure are sliced using new criteria
– Compute slice over an entire
program
– Two ways for crossing
procedure boundary
–
–
INFSOM-RI-026753
Up – going from sliced
procedure into calling
procedure
Down – going from sliced
procedure into called
procedure
Indus
Indus is an effort to provide a collection of program analyses
and transformations implemented in Java to customize and
adapt Java programs. It is intended to serve as an umbrella
for
static analyses such as points-to analysis, escape
analysis, and dependence analyses,
transformations such as program slicing and program
specialization via partial evaluation, and
any software module that delivers the
analyses/transformations into a platform such as
Eclipse.
INFSOM-RI-026753
Indus modules
There are 3 modules that are part of Indus
1. Indus is a module that transforms the original Java code into an
intermediate code, then CFG, Call graph, etc.
2. StaticAnalyses module is intended to be the collection of static
analyses such as object-flow analysis, escape analysis, and
dependence analyses.
3. Java Program Slicer module contains the core implementation of
Java program slicer along with adapters that deliver the slicer in
Eclipse
INFSOM-RI-026753
Interprocedural backward slicing
•Slicing
criterion
•Slicing
criterion
converted
into
Jimple
statement
INFSOM-RI-026753
Backward slicing on more
classes I.
INFSOM-RI-026753
Coverage and JavaCov
www.eu-etics.org
INFSOM-RI-026753
Coverage



Unit test can find half of the bugs in the code.
System testing costs 3-5 times more than unit testing
Unit testing comprises of two main parts
1.
2.



Covearge analysis is necessary to decide when to stop
testing
100% coverage is advisable, but sometimes difficult to reach
Instrumentation




Test bed construction
Coverage analysis (test data adequacy criteria)
Probes (extra code) are inserted into appropriate places
Instrumented code is compiled and executed
Coverage analyser computes the coverage (executed entities
/ all entities) and displays the uncovered entities
New tests have to be generated to cover uncovered entities
INFSOM-RI-026753
Testing criteria

All-statements criterion


GJTester supports only this – not enough based on the
testing literature
All-branches criterion – every branch should be executed at
least once. In this way empty else branches are tested.


INFSOM-RI-026753
JavaCov inserts probes before and after predicates
statements
100% coverage usually can be reached
Testing criteria
•
•
•
MC/DC criterion: all the relational expressions are tested
separately. Example. If (A and B) or C we test A, B and
C separately.
When testing A, B should be true and C should be false.
In this case if A is true the so does the whole expression
while if A is false, the expression will also be false. We
should test both cases in this way. Testin B and C
happens in the same way .
This criterion has to be applied when a part of the code is
really critical. This criterion is applied for example in
avionics and other safety critical systems (Boeing,
Alstom).
INFSOM-RI-026753
JavaCov
•
•
Good news: though JavaCov is generally not free of
charge, ETICS can use it freely
JavaCov is an Eclipse plugin which
– supports all-branches criterion
– (uniquely) support MC/DC criterion
INFSOM-RI-026753