CTL Model-checking for Systems
with Unspecified Components
Gaoyan Xie and Zhe Dang
School of Electrical Engineering and Computer Science,
Washington State University, Pullman, WA 99164, USA
October 29, 2004
SAVCBS’04
Presented by: Gaoyan Xie
Motivation

Challenges to the quality assurance of
component-based systems:



Externally obtained components may be new
sources of system failures
Safely upgrading a deployed component is
extremely difficult
Example:

The Ariane 5 disaster was caused by reusing
certain software module from Ariane 4 without
sufficiently testing it in the new environment
SAVCBS’04
Two Dimensions of The Possible
Solutions

Methodologies



Compositional approachesconventional, used most often
 Bottom-up, from components to systems
Decompositional approachesnot many can be automated
 Top-down, from systems to components
Techniques


Software Testing
 A trial and error technique
 Integration/system testing is often needed at the system level
Model-checking:
 A formal and usually complete technique
 The assume-guarantee paradigm is often used for system-level
verification
SAVCBS’04
Testing a Component in Isolation

Before it is released



By component developers
By third-party labscomponent certification
Difficulty:


A component may target a wide scope of deployment
environments, which may not be tried out
When it is integrated into a system


By system developers
Expensive:


A component is usually built with multiple sets of functionalities
Infeasible:

The state space of a component’s interface may be too large to be
thoroughly tested
SAVCBS’04
Testing A Component-based
System (CBS)

Integration/system testing

Difficulties:





Components within a CBS may run concurrently
Even inside a component may have multi-threads running
concurrently, and
Concurrency is notoriously difficult to test
Locating the source of a bug is not easy!
Feasibility:

A component may be used for dynamic upgrading a running
system, which may not be stopped for testing
SAVCBS’04
Model-checking


Checks whether a finite-state system is a model of
a given temporal formula
Requires a complete and formal specification of
the finite-state system


Not always possible: design details or source codes of
components in a CBS are generally unavailable
The assume-guarantee paradigm is often used for
system-level verification


A compositional approach
Requires an assumption for the environment of a
module
SAVCBS’04
The Research Problem

A different perspective


How to ensure that a component functions
correctly in a specific deployment environment
(which is called a host system)
A model-checking problem

Given a host system M with an unspecified
component X and a CTL formula f, check
whether f is a model of (M, X):
(M, X)= f
SAVCBS’04
Basic Ideas

Goal


Seek a definite answer (no approximation) to the
model-checking problem, i.e., (M, X)= f
Solution


Automatically deduce a sufficient and necessary
condition over the unspecified component
Check the condition through testing the
unspecified component
SAVCBS’04
The System Model

A system Sys= (M, X) consists of a host system M
and an unspecified component X


The host system M is a well-specified finite state
transition system
The unspecified component X can be viewed as a
deterministic Mealy machine




Internal detail of X is unknown
Interface (input/output symbols) and an upper bound for the
number of states in X are known
An implementation of X is available for testing
M and X run concurrently and communicate with each
other by synchronizing over X’s input/output symbols
SAVCBS’04
An Example

Check: starting from state s0, is state s3 reachable?

(M, Comm), s0= E[true U s3]
msg?
s0
msg?
send/no
s4
ack/yes
s1
msg?
send/yes
s2
ack/yes
data
s3
ack
Host system M
SAVCBS’04
Comm
yes
no
An Examplecontd.

Check: starting from state s0, is state s3 reachable?

M, s0= E[true U s3]
s0
s1
s4
s3
s2
Transition system M
SAVCBS’04
An Examplecontd.

Check: starting from state s0, is state s3 reachable?


M, s0= E[true U s3]
Search the graph to see whether there is a path between s0
and s3

s0s1s4s3 is such a witness path
s0
s1
s4
s3
s2
Transition system M
SAVCBS’04
An Examplecontd.

Check: starting from state s0, is state s3 reachable?


(M, Comm), s0= E[true U s3]
Search the graph to see whether there is a path between s0
and s3

Is s0s1s4s3 still a witness path?
msg?
s0
msg?
send/no
s4
ack/yes
s1
msg?
send/yes
s2
Test Comm to see whether the communication
trace along the path can be accepted by Comm:
Will “send/no, ack/yes” be a successful
test case for Comm?
ack/yes
send
s3
ack
Host system M
SAVCBS’04
Comm
yes
no
An Examplecontd.

Infinite number of paths could be witness path:


s0s1s2s1s4s3, s0s1s2s3, s0s1s2s1s2…s3,…
Infinite number of communication traces need to be tested:

“send/yes send/no ack/yes”, “send/yes ack/yes”, “send/yes
send/yes…ack/yes”, …
msg?
s0
msg?
send/no
s4
ack/yes
s1
msg?
send/yes
s2
ack/yes
send
s3
ack
Host system M
SAVCBS’04
Comm
yes
no
An Examplecontd.


Use an annotated directed graph Gh (called witness graph) to represent
all those communication traces
The original model-checking problem is:
 True, if one of the communication traces in Gh is a successful test
case for Comm
 False, if none of the communication traces in Gh is a successful test
case for Comm
msg?
s0
msg?
send/no
s4
ack/yes
s1
msg?
send/yes
s2
s1
send/yes
send/no
ack/yes
s3
ack/yes
s4
Host system M
ack/yes
s3
Witness graph Gh
SAVCBS’04
s2
An Examplecontd.

How to handle communication traces with infinite
length?

A result from traditional black-box testing:


This result implies that


An equivalent internal structure of a black-box can be
recovered through testing if only an upper bound for its
number of states is known
Only communication traces with bounded length need to be
tested
The bound can be calculated from the number of states
in M and the upper bound for the number of states in
Comm
SAVCBS’04
Our Algorithm

The original CTL model-checking algorithm for
checking M= f:


Goes through a series of stages.
A subformula h of f is processed at each stage:




h is added to the labels of a state s if h is true at s
Returns true if s0 is labeled with f, or false otherwise
Our algorithm adapts the original algorithm to
handle systems with unspecified components.
It follows a similar structure, except that during
each stage for subformula h, the labeling of a state
s is far more complicated
SAVCBS’04
Our Algorithm—contd.

A state s could be labeled with






1  if h is true at s and the truth has nothing to do with communications
A witness graph  if h is an CTL operator and the truth of h at s depends
on communication traces in the graph
A logical combination of witness graphs  if h is a logic combination of
other formulas and the truth of h at s depends on communication traces in
the graph
For each CTL operator in h, a witness graph is constructed
and associated with an ID (which begins from 2)
A state is labeled with an ID expression (constructed from
IDs and logic connectives , )
A labeling function Lh is returned upon the completion of
processing h
SAVCBS’04
Our Algorithm—contd.

The algorithm outline:
Procedure CheckCTL(M, X, s0, f)
Lf := ProcessCTL(M, f);
If s0 is labeled by Lf Then
If Lf (s0) = 1 Then
Return true;
Else
Return TestWG(X, reset, s0, Lf (s0));
Endif
Else
Return false;
Endif
End Procedure


SAVCBS’04
ProcessCTL is the lableling process, it
recursively handles the five cases of f:
, , EX, EU, and EG by calling the
procedures Negation, Union,
ProcessEX, ProcessEU, and
ProcessEG respectively
The maximal length of communication
traces to be tested is bounded by
(k·n·m2), where k is the number of
CTL operators in the formula f, m is
the upper bound for the number of
states in the unspecified component X,
and n is the number of states in the
host system M
Another Example
Check: starting from the initial state s0, whenever the system reaches
state s2, it would eventually reach s3; i.e., check
(M, X), s0= AG(s2  AF s3 ) to be true
Taking a negation of the original problem, it’s equivalent to checking
(M, X), s0= E[true U (s2  EG s3 )] to be false


msg?
s0
msg?
send/no
s4
ack/yes
s1
msg?
send/yes
s2
ack/yes
send
s3
ack
Host system M
SAVCBS’04
Comm
yes
no
Check (M, X), s0=E[true U(s2EGs3)]
Step 1. Atomic subformula s2 is processed by ProcessCTL, which
returns a labeling function L1 = {(s2, 1)}

msg?
s0
msg?
send/no
s4
ack/yes
s1
msg?
send/yes
s2
ack/yes
send
s3
ack
Host system M
SAVCBS’04
Comm
yes
no
Check (M, X), s0=E[true U(s2EGs3)]
Step 1. Atomic subformula s2 is processed by ProcessCTL, which
returns a labeling function L1 = {(s2, 1)}
Step 2. Atomic subformula s3 is processed by ProcessCTL, which
returns a labeling function L2 = {(s3, 1)}


msg?
s0
msg?
send/no
s4
ack/yes
s1
msg?
send/yes
s2
ack/yes
send
s3
ack
Host system M
SAVCBS’04
Comm
yes
no
Check (M, X), s0=E[true U(s2EGs3)]
Step 1. Atomic subformula s2 is processed by ProcessCTL, which
returns a labeling function L1 = {(s2, 1)}
Step 2. Atomic subformula s3 is processed by ProcessCTL, which
returns a labeling function L2 = {(s3, 1)}
Step 3. Subformula s3 is processed by Negation, which returns a
labeling function L3 = {(s0, 1), (s1, 1), (s2, 1), (s4, 1)}



msg?
s0
msg?
send/no
s4
ack/yes
s1
msg?
send/yes
s2
ack/yes
send
s3
ack
Host system M
SAVCBS’04
Comm
yes
no
Check (M, X), s0=E[true U(s2EGs3)]
Step 4. Subformula EG s3 is processed by ProcessEG, which
constructs a witness graph G1 = <N, E, L3> with an ID 2 and returns a
labeling function L4 = {(s0, 2), (s1, 2), (s2, 2)}

msg?
s0
msg?
send/no
s4
ack/yes
s1
send/yes
msg?
s2
s0
s1
send/yes
ack/yes
Witness graph G1 with ID 2
s3
The host system M
SAVCBS’04
s2
Check (M, X), s0=E[true U(s2EGs3)]
Step 4. Subformula EG s3 is processed by ProcessEG, which
constructs a witness graph G1 = <N, E, L3> with an ID 2 and returns a
labeling function L4 = {(s0, 2), (s1, 2), (s2, 2)}
Step 5. Subformula s2  EG s3 is processed by Negation and Union,
which finally return a labeling function L5 = {(s2, 2)}


msg?
s0
msg?
send/no
s4
ack/yes
s1
send/yes
msg?
s2
s0
s1
send/yes
ack/yes
Witness graph G1 with ID 2
s3
The host system M
SAVCBS’04
s2
Check (M, X), s0=E[true U(s2EGs3)]
Step 6. The formula E[true U (s2  EG s3)] is processed by procedure
ProcessEU, which constructs a witness graph G2 = <N, E, Ltrue, L5>
with an ID 3 and returns a labeling function Lf = {(s0, 3), (s1, 3), (s2, 3),
(s3, 3), (s4, 3)}
Lf (s0)  1, this indicates that the truth of the original problem depends
on communications. Then we shall see whether Lf (s0) can be evaluated
to be true


msg?
s0
msg?
send/no
s4
ack/yes
s1
send/yes
msg?
s3
The host system M
s2
s0
s1
send/yes
send/no
ack/yes
ack/yes
s4
ack/yes
s3
Witness graph G2 with ID 3
SAVCBS’04
s2
Evaluate ID Expression Lf (s0)
Evaluating Lf (s0) is a testing process with test-cases generated from the
two witness graphs (details can be seen in the paper)
Essentially, we are testing whether some communication trace (of
bounded length) with m consecutive symbol pairs “send yes” is a
successful test-case for the unspecified component X.


msg?
s0
msg?
send/no
s4
ack/yes
s1
send/yes
msg?
s2
Errata: on page 6, in the paragraph
above “Note.”, “… two consecutive
symbol pairs …” should be “… m
consecutive symbol pairs… ”
ack/yes
send
s3
ack
The host system M
SAVCBS’04
Comm
yes
no
Summary

A decompositional approach


A hybrid approach


Reduces a system verification problem to a testing problem over the
unspecified component
Combines model-checking with software testing techniques
Advantages





Stronger confidence about the reliability of the system
Customization of testing over a component with respect to particular
system requirement
Reuse of intermediate verification results
Automatic carrying out of the whole process
Complete and sound algorithm with an appropriate bound
SAVCBS’04
Related Work





Specification-based testing [Heitmeyer et. al. 1999], generate test-cases
from counter-examples produced by a model-checker
Black-box checking [Peled et. al. 1999], test a single black-box against a
given LTL formula
Automatic assume-guarantee [Giannakopoulou et. al. 2004], automatic
generation of assumptions of a component, uses a less expressive
formalism (LTS) for properties
Automatic deduction of model-checking conditions [Fisler et. al.
2001], has false negative
[Xie & Dang 2004]
 LTL algorithms for systems with only one, finite-state unspecified
component (FATES’04)
 An automata-theoretic and decompositional approach to testing a
system of concurrent black-boxes (submitted)
SAVCBS’04
Ongoing Work


Decompositional LTL model-checking
Less-restricted system model




Asynchronous communications
Multiple unspecified components
Unspecified components with infinite state
space
Case studies

CORBA applications
SAVCBS’04