Black Box Checking
Book: Chapter 9
Model Checking





Finite state description of a system B.
LTL formula . Translate  into an automaton P.
Check whether L(B)  L(P)=.
If so, S satisfies . Otherwise, the intersection
includes a counterexample.
Repeat for different properties.


Buchi automata (w-automata)
S - finite set of states. (B has l  n states)
S0  S - initial states. (P has m states)
S - finite alphabet.
(contains p letters)
d  S  S  S - transition relation.
F  S - accepting states.
Accepting run: passes a state in F infinitely often.
System automata: F=S, deterministic.
Example: check a
a
a
<>a
a, a
Example: check <>a
a
a
<>a
a
a
Example: check <
•>a
a, a
a
<>•
~a
a
Use automatic translation algorithms, e.g.,
[Gerth,Peled,Vardi,Wolper 95]
System
a
c
b
Every element in the product is a counter
example for the checked property.
s1
a
s2
q1
b
c
a
s3
Acceptance is
determined by
automaton P.
s1,q1
s1,q2
a
a
q2
a s ,q
2 1
b
a
c
s3,q2
a
Testing





Unknown deterministic finite state system B.
Known: n states and alphabet S.
An abstract model C of B. C satisfies all the properties
we want from B.
Check conformance of B and C.
Another version: only a bound n on the number of
states l is known.

Model Checking / Testing






Given Finite state
system B.
Transition relation of B
known.
Property represent by
automaton P.
Check if L(B)  L(P)=.
Graph theory or BDD
techniques.
Complexity: polynomial.





Unknown Finite state
system B.
Alphabet and number of
states of B or upper
bound known.
Specification given as
an abstract system C.
Check if B C.
Complexity: polynomial
if number states known.
Exponential otherwise.
Black box checking




Property represent by
automaton P.
Check if L(B)  L(P)=.
Graph theory
techniques.



Unknown Finite state
system B.
Alphabet and Upper
bound on Number of
states of B known.
Complexity:
exponential.

Combination lock automaton
Accepts only words with a specific suffix (cdab
in the example).
s1
c
s2
d
s3
a
s4
b
s5
Conformance testing
b
a
b
a
a
b
b
a
b
a
Cannot distinguish if reduced or not.
Conformance testing (cont.)
a
b
a
When the black box is nondeterministic, we
might never test some choices.
Conformance testing (cont.)
b
a
a
a
a
b
a
b
a
b
a
b
Need: bound on number of states of B.
Need reliable RESET
a
b
s1
s2
a
a
s3
b
Vasilevskii algorithm





Known automaton A has l states.
Black box automaton has up to n
states.
Check each transition. Check that there
are no "combination lock" errors.
Complexity: O(l2 n p n-l+1).
When n=l: O(l3p).
Experiments
reset
a
b
c
a
c
b
try c
try b
a
c
b
a
c
b
a
c
b
a
c
b
fail
Simpler problem: deadlock?


Nondeterministic algorithm:
guess a path of length  n from the initial
state to a deadlock state.
Linear time, logarithmic space.
Deterministic algorithm:
systematically try paths of length n, one
after the other (and use reset), until
deadlock is reached.
Exponential time, linear space.
Deadlock complexity




Nondeterministic algorithm:
Linear time, logarithmic space.
Deterministic algorithm:
Exponential (p n-1) time, linear space.
Lower bound: Exponential time (use
combination lock automata).
How does this conform with what we
know about complexity theory?
Modeling black box checking



Cannot model using Turing machines:
not all the information about B is given.
Only certain experiments are allowed.
We learn the model as we make the
experiments.
Can use the model of games of
incomplete information.
Games of incomplete
information







Two players: $-player, -player (here, deterministic).
Finitely many configurations C. Including:
Initial Ci , Winning : W+ and W- .
An equivalence relation @ on C (the $-player cannot
distinguish between equivalent states).
Labels L on moves (try a, reset, success, fail).
The $-player has the moves labeled the same from
configurations that are equivalent.
Strategy for the $-player: will lead to a configuration in
W+  W-. Cannot distinguish equivalent conf.
Nondet. strategy: ends with W+. Can distinguish.
Modeling BBC as games



Each configuration contains an automaton
and its current state (and more).
Moves of the $-player are labeled with
try a, reset... Moves of the -player with
success, fail.
c1 @ c2 when the automata in c1 and c2 would
respond in the same way to the experiments
so far.
A naive strategy for BBC





Learn first the structure of the black box.
Then apply the intersection.
Enumerate automata with n states (without
repeating isomorphic automata).
For a current automata and new automata,
construct a distinguishing sequence. Only one
of them survives.
Complexity: O((n+1)p (n+1)/n!)
On-the-fly strategy





Systematically (as in the deadlock case), find
two sequences v1 and v2 of length <=m n.
Applying v1 to P brings us to a state t that is
accepting.
Applying v2 to P brings us back to t.
Apply v1 (v2 )n+1 to B. If this succeeds,
there is a cycle in the intersection labeled
with v2, with t as the P (accepting)
component.
Complexity: O(n2p2mnm).
Learning an automaton




Use Angluin’s algorithm for learning an
automaton.
The learning algorithm queries whether
some strings are in the automaton B.
It can also conjecture an automaton Mi
and asks for a counterexample.
It then generates an automaton with
more states Mi+1 and so forth.
A strategy based on learning





Start the learning algorithm.
Queries are just experiments to B.
For a conjectured automaton Mi ,
check if Mi  P = 
If so, we check conformance of Mi with B
(Vasilevskii algorithm).
If nonempty, it contains some v1 (v2)w . We
test B with v1 (v2)n+1. If this succeeds: error,
otherwise, this is a counterexample for Mi .
Black Box Checking Strategy
discrepancy
Model
Incremental
learning
Model
Checking
no counterexample
black box
testing
conformance established
No error
found
false negative
Path
counterexample
Comparing
counterexample
System
actual error
Report
error
Complexity



l - real size of B.
n - an upper bound of size of B.
p - size of alphabet.
Lower bound: reachability is similar to
deadlock.
 O(l 3 p l + l 2mn) if there is an error.
 O(l 3 p l + l 2 n p n-l+1+ l 2mn) if there is no
error.
If n is not known, check while time allows.

Some experiments




Basic system written in SML (by Alex
Groce, CMU).
Experiment with black box using Unix
I/O.
Allows model-free model checking of C
code with inter-process communication.
Compiling tested code in SML with BBC
program as one process.
Conclusions




Black box checking is a combination of
testing and model checking.
If a tight bound on size of B is given:
learn B first, then do model checking.
Tight lower bound on complexity, up to
polynomial factor.
Use of games of incomplete information
to model testing problems.