ECE 452 – Introduction to Software
Engineering
Lecture 25: Testing Strategies/OO
Testing
Verification & validation
Verification: Are we building the product right
z Validation: Are we building the right product
z
Manish Parashar
[email protected]
Department of Electrical &
Computer Engineering
Rutgers University
Testing strategy
ECE 452 - Introduction to Software Engineering
Unit testing
interface
Module
high-order
tests
requirements
local data structures
boundary conditions
independent paths
integration test
design
error handling paths
unit
test
code
Test cases
ECE 452 - Introduction to Software Engineering
Unit testing environment
interface
driver
local data structures
boundary conditions
ECE 452 - Introduction to Software Engineering
Why integration testing?
If all units work individually, why doubt that
they work together?
z Putting units together - interfacing
z
independent paths
Module
error handling paths
stub
stub
RESULTS
Test cases
ECE 452 - Introduction to Software Engineering
ECE 452 - Introduction to Software Engineering
1
Integration testing
Top down integration
incremental
construction
strategy
Big
Bang!
B
incremental
“builds”
F
D
as new modules are integrated,
some subset of tests is re-run
E
ECE 452 - Introduction to Software Engineering
Bottom-up integration
G
stubs are replaced one at
a time, “depth first”
C
regression
testing
top module is tested
with stubs
A
ECE 452 - Introduction to Software Engineering
Sandwich testing
A
B
D
G
drivers are replaced one
at a time, “depth first”
C
cluster
F
A
E
worker modules are grouped into
builds and integrated
B
validation test
z system test
z alpha and beta test
z other specialized testing
z
y performance testing
y usability testing
y ...
D
E
G
worker modules are grouped into
builds and integrated
ECE 452 - Introduction to Software Engineering
Alpha & Beta test
Alpha test
software
customer tests
developer site
customer site
Beta test
developer
reviews
developer site
ECE 452 - Introduction to Software Engineering
F
C
cluster
ECE 452 - Introduction to Software Engineering
High-order testing
top modules are tested
with stubs
customer
tests
software
customer site
ECE 452 - Introduction to Software Engineering
2
Testing strategies and OO testing
Object - oriented testing
Testing strategies & higher-order testing
z Testing object-oriented software
z
z
z
begins by evaluating the correctness and consistency
of the OOA and OOD models
testing strategy changes
y the
concept of the ‘unit’ broadens due to encapsulation
focuses on classes and their execution across a
‘thread’ or in the context of a use case
y integration
y validation
z
uses conventional black box methods
test case design draws on conventional methods, but
also encompasses special features
ECE 452 - Introduction to Software Engineering
ECE 452 - Introduction to Software Engineering
OOT strategy
Integration testing of OO software
z
z
z
classes are the smallest testable unit
inheritance defines new context for methods
y
z
z
objects have states: testing methods have to take that into
account
class testing is the equivalent of unit testing
y
y
z
behavior of inherited methods can be changed because of methods that
are called within
methods have to be tested per class
thread-based testing
x integrates
the set of classes required to respond to one input or
event
z
use-case based testing
x integrates the set of classes required to respond to one use case
x shows that every use case is executable within the system
•
with and without user interface
operations within the class are tested
the state behavior of the class is tested
OO testing concentrates on the states of the objects and their
interactions
ECE 452 - Introduction to Software Engineering
OOT - test case design (1)
Berard [BER93] proposes the following approach:
1. each test case should be uniquely identified and
should be explicitly associated with the class to
be tested
2. the purpose of the test should be stated
3. a list of testing steps should be developed for
each test and should contain [BER94]:
ECE 452 - Introduction to Software Engineering
OOT - test case design (2)
A a list of specified states for the object that is to be
tested
B a list of messages and operations that will be exercised
as a consequence of the test
C a list of exceptions that may occur as the object is
tested
D a list of external conditions (i.e., changes in the
environment external to the software that must exist in
order to properly conduct the test)
E supplementary information that will aid in
understanding or implementing the test.
ECE 452 - Introduction to Software Engineering
ECE 452 - Introduction to Software Engineering
3
Problems in testing OO software
Testing shall report errors
accessing the internal state of
objects
z Encapsulation hides internal things
z Inheritance: test cases of the superclass may
may have to be adapted
z
OOT methods: Use case based
testing
Concentrates on what the user does NOT what
the system does
z Based on use cases
z Described by
z
ya
test
y specific
ECE 452 - Introduction to Software Engineering
OOT methods: random testing
z
ECE 452 - Introduction to Software Engineering
OOT methods: partition testing (1)
z
random testing
identify a minimum test sequence
x an
operation sequence that defines the minimum life history of the
object
generate a variety of random (but valid) test sequences
xcategorize
and test operations based on their
ability to change the state of a class
x exercise other (more complex) class instance life histories
ECE 452 - Introduction to Software Engineering
z
Partition testing
attribute-based partitioning
xcategorize
and test operations based on the
attributes that they use
category-based partitioning
xcategorize
and test operations based on the
generic function each performs e.g.
•
•
•
•
Partition testing
reduces the number of test cases required to
test a class in much the same way as
equivalence partitioning for conventional
software
state-based partitioning
identify operations applicable to a class
define constraints on their use
OOT methods: partition testing (2)
user needs
ECE 452 - Introduction to Software Engineering
State diagram based testing
Use state diagram to determine a sequence of
events
z Tests should cover all states
z All transitions should be tested at least once
z Test invalid events
z
initialization
state changing
queries
termination
ECE 452 - Introduction to Software Engineering
ECE 452 - Introduction to Software Engineering
4
OOT methods: Inter-class testing (1)
z
Inter-class testing
OOT methods: Inter-class testing (2)
z
For each client class, use the list of class operators to
generate a series of random test sequences. The
operators will send messages to other server
classes.
For each message that is generated, determine the
collaborator class and the corresponding operator
in the server object.
Inter-class testing
For each operator in the server object (that has been
invoked by messages sent from the client object)
determine the messages that it transmits.
For each of the messages, determine the next level
of operators that are invoked and incorporate these
into the test sequence
ECE 452 - Introduction to Software Engineering
ECE 452 - Introduction to Software Engineering
Test case document
Class test cases
1. Usual cover information
2. Class test cases
3. Use case test descriptions
4. References
5. Appendices
z
For every class in your system
y at
least one test case for
xthe
normal behavior of every public method
xthe
exceptions of every public method
xevery state transition of the class
xinvalid
state transitions of the class
y white-box test cases for important methods
ECE 452 - Introduction to Software Engineering
Use case test descriptions
z
For every use case/sequence diagram in the
design document
test case that shows that the use case can be
executed by the system
ECE 452 - Introduction to Software Engineering
Executing tests
z
y main method of class executes the class test
y class
ya
xdescribe
z
y test
cases for all exceptions of the use case
ECE 452 - Introduction to Software Engineering
methods define (sets of) tests
Use case tests
y develop
the objects that get messages from the UI
xlist
the messages that are send from the UI to define the
test cases
Develop automatic test drivers for every class
y test
z
test classes that execute the use cases
the use cases manually with UI
System test driver calls all class test drivers
and all use case test classes
ECE 452 - Introduction to Software Engineering
5
Implementing test cases
z
Debugging: Symptoms & causes
Test driver code must
y create
the system state described in the test case
y call the method to test
y check if the expected result is produced
y print test result to file/console e.g.
Symptom may disappear when another
problem is fixed
z
Cause may be due to a combination of nonerrors
Cause may be due to a system or compiler
error
Cause may be due to assumptions that
everyone believes
Symptom may be intermittent
z
y count
ECE 452 - Introduction to Software Engineering
Symptom and cause may be separated
z
z
Class: ResourcePool
method: addAgent
test case: No. 265 - tests exception of the method
result: failed
the number of successful/failed test to produce a
summary result at the end
z
symptom
z
cause
ECE 452 - Introduction to Software Engineering
6