Black Box Testing
Complements White Box
Testing
Finds a different class of errors
Copyright © 2016 – Curt Hill
Introduction
• Black Box testing is also called
Behavioral testing
– We are interested in the behavior of the
software without regard to its internal
structure
– Almost
• The focus is on requirements of the
software
– The software is not just the final
program or system of programs
– It includes all of the components
Copyright © 2016 – Curt Hill
What kind of errors?
• Missing functions
– In the sense of purpose rather than
units
• Interface problems
• Data structure or external data
errors
• Performance errors
• Initialization and termination issues
Copyright © 2016 – Curt Hill
Categories
• There are several approaches that
help guide the design of these tests
• These include:
–
–
–
–
Graph-based
Domain Partitioning
Boundary Value Analysis
Comparison
• Lets see some more detail
Copyright © 2016 – Curt Hill
Graph-based
• A variety of techniques use a graph
• This graph usually has objects as
nodes
– This may include collections of data
and routines that are in a non-object
oriented language
• The edges/arcs are interactions
between these objects
• These are usually considered in
design and before coding
• We will look at transaction flow and
finite state Copyright © 2016 – Curt Hill
Transactions
• The notion of a transaction is one
unit of work from the system’s view
• It is usually a series of steps
–
–
–
–
–
–
Accept input
Verify
Several processing steps
Update data store
Report
Remove transaction from system
• Often this parallels or mimics an
earlier manual process
Copyright © 2016 – Curt Hill
Terminology
• The entrance of a transaction into a
system is generally an external
event
• This is often referred to as a birth
– The birth may be the entrance or follow
some type of verification
• The death is when the transaction is
complete and removed from the
system
– The transaction process is removed,
not the effects
Copyright © 2016 – Curt Hill
Transaction Flow Graph
• A different type of flow graph that
shows the processing of a
transaction
• This is a path through types of
processing
• The graph greatly condenses
processing
– Not a full flow chart of the components
• An example of an online order is
given next
Copyright © 2016 – Curt Hill
Example
cancel
User
Begin
Request
Type
Request
order order
from CPU
Accept
Order from
CPU
Process
Form
B
help
Y
B
D
Transmit
Page to
terminal
CPUAccept
Confirm
C
Accept
Input
Field
Y
Valid ?
More
Pages ?
Transmit
To CPU
More
Fields?
N
Transmit
Diagnostic
to Terminal
Copyright © 2016 – Curt Hill
User wants
Review?
C
D
N
Set up
Review
Done
Data flow diagram
•
•
•
•
Rectangle is person/department
Three sided rectangle is a data store
Arrows indicate data flow
Rounded rectangle indicates a
transformational process
Copyright © 2013 Curt Hill
Example
Copyright © 2013 Curt Hill
Commentary
• We do not necessarily see the
processing
– Nor do we see what modules may be
involved
• The transaction flow graph is similar
to a control flow graph
– Yet at a higher level
• Generating tests cases is similar
– Every node and every edge needs to be
exercised
Copyright © 2016 – Curt Hill
Finite State Testing
• As you should or will know a Finite
State Machine has a state
• Given a particular input it will move
to another state
Copyright © 2016 – Curt Hill
Finite State Diagram
• Describes an interaction with a
component
• Has one start and one or more finish
nodes
• Each box represents a state
• Each arrow a state transition
– Arrow is labeled by the input or event
that starts it
Copyright © 2013 Curt Hill
Example Windows Login
• Has one start and one end state
• Two middle states
– Entering login
– Entering password
• The Enter or Tab changes state
• All other characters return to same
state
Copyright © 2013 Curt Hill
Example Windows Login
Copyright © 2013 Curt Hill
Graph Methods
• Are these black or white?
• The transaction flow and finite state
graphs may be created before
coding
• However, the tests cannot be
effective until we know what
modules are touched
• Thus they may be a little black and
white
Copyright © 2016 – Curt Hill
Equivalence Partitioning
• The input to a system or routine is
partitioned into groups
• Samples are extracted from each
group
• We cannot usually exhaustively test,
so well chosen samples will suffice
Copyright © 2016 – Curt Hill
Simple Example
• Suppose there are two real numbers
as input
• Without knowing anything about
what is to be done with these we
could partition the input space into
four pieces based on sign of the two
• Similarly any curve in Cartesian
coordinates divides the plane into
three areas: on the line, above and
below
Copyright © 2016 – Curt Hill
Guidelines
• If an input variable defines a range
– Choose one in and two out
– One above and one below
• If an input variable requires one or
more specific discrete values
choose one good and one bad
• Both the legal and illegal values
define an equivalence class
Copyright © 2016 – Curt Hill
Boundary Value Testing
• An extension of equivalence
partitioning
• Errors are more likely at the
boundaries between classes than
the middles
– For real number input, errors are more
likely near the zero or other boundaries
• We see this in boundaries of input
values but also at boundaries of
indices of an array
Copyright © 2016 – Curt Hill
Finally
• Although black box testing is mostly
driven by requirements, there are
techniques to make this more
effective
• We have covered:
–
–
–
–
Transaction graphs
Finite state graphs
Equivalence partitioning
Its enhancement boundary value
testing
Copyright © 2016 – Curt Hill