Model checking for probabilistic
real-time systems
Marta Kwiatkowska
School of Computer Science
www.cs.bham.ac.uk/~mzk
Joint work with
Gethin Norman, Roberto Segala, Jeremy
Sproston, Dave Parker and Fuzhi Wang
Overview
• Motivation
– Why probability, examples of real world protocols
• Probabilistic timed automata
– Syntax, semantics
– The logic PTCTL
• Model checking for probabilistic timed automata
– The digital clocks approach
– The symbolic zone-based approach
– Probabilistic reachability, expectation and PTCTL properties
• Experimental results
– Prototype implementations
– FireWire root contention case study
Motivation
• In distributed environment, probability helps
– As a symmetry breaker
– In gossip-based routing and multicasting
• In distributed environment, clocks and real-time used
– To measure delays and time-outs
• Distributed computation implies non-determinism
– To model parallel execution
• Need modelling formalisms and verification methods
capable of dealing with
– Quantitative probability, timing and non-determinism
Real-world protocol examples
• Protocols featuring (discrete) probability, real-time and
nondeterminism
– Randomised back-off schemes
• Ethernet, WiFi (802.11), Zigbee (802.15.4)
– Random choice of waiting time
• Bluetooth, device discovery phase
– Random choice of a timing delay
• Root contention in IEEE 1394 FireWire
– Random choice over a set of possible addresses
• IPv4 dynamic configuration (link-local addressing)
– Random choice of a destination
• Crowds anonymity, gossip-based routing
• Continuous probability distribution needed to model network
traffic, random delays…
www.cs.bham.ac.uk/~dxp/prism
Aims
• Adopt the probabilistic timed automata model [KNSS02],
combining
– Probability (discrete or continuous – here discrete only)
– Real-time, non-determinism
• Develop foundations of efficient model checking for
probabilistic timed automata, and in particular
– The digitisation techniques
– Fully symbolic techniques
• Enhance the probabilistic symbolic model checker PRISM
– Allow the modelling of real-valued clocks
– www.cs.bham.ac.uk/~dxp/prism
Background: semantics of executions
• Markov decision processes (MDPs)
–
Probability & nondeterminism coexist
• State-to-state transition:
–
–
init
s0
Nondeterministic choice over
probability distributions μ from state
Probabilistic choice of target state
according to the chosen distribution μ
d
1
a
1
0.02
try
s1
1
fail
s3
b
0.98
c
succ
s2
1
e
• Formally, (S,s0,Steps,Act,L):
–
–
–
–
S finite set of states, s0 initial state
Steps maps states s to sets of probability distributions μ over S
Act labelling of steps with actions
L: S ! 2AP atomic propositions
• Unfold into infinite paths s0a0μ0s1a1 μ1s2… s.t. μi(si,si+1) > 0, all i
Probability space and adversaries
• Nondeterminism is resolved by adversaries (schedulers)
mapping finite path s0a0μ0s1a1μ1…sn to a distribution from sn
– There may be no unique probability of certain behaviours
– Obtain the minimum and maximum probabilities or expected cost
• Probability space induced on Paths by adversary (policy) A
• Intuitively, for a fixed adversary A 2 Adv:
– Sample space = infinite paths PathAs from s
– Event = set of paths
s0 … sn
– Basic event = cone
• Formally, probability space (PathAs, Ω, PrA)
– Assign probability P(.) to finite paths ω = s0a0μ0s1a1μ1s2a2…sn
– Define PrA (C(ω)) = P(ω), for cones C(ω):
C(ω) = { π 2 PathAs | ω is prefix of π}
Time, clocks and zones
• Dense real-time, t 2 R¸0
• Clocks take values from time domain R¸0
–
–
–
–
Increase at the same rate as real time
Assume finite set X of clocks, maximum const in guards kmax
If n clocks, then v,v’ 2 Rn¸0 are clock valuations
v+t is time increment, v[X:=0] clock reset of all clocks in X 2 X
• Zones of X, for x,y 2 X, d 2 N, ~ stands for <, ≤, ≥ or >
ζ ::= x ~ d j x-y ~ d j ζ Æ ζ j ζ Ç ζ j : ζ
– Consider only in canonical form
– Closed, diagonal-free if do not feature x < d, x > d, x-y ~ d
– Convex, or non-convex (cf [Tri’98])
Probabilistic Timed Automata: syntax
• Features:
– Clocks, x, real-valued
– Can be reset,
e.g. {x:=0}
– Invariants, e.g. x·8
– Probabilistic transitions,
guarded e.g. x¸4, x=8
• Formally, (Loc,s0,Inv,prob,Act,L):
–
–
–
–
send
1
true
x¸4
{x:=0} x=8
wait
waited
x≤8
0.01
transmit
0.99
ok
true
Loc finite set of locations
s0 initial location
Inv maps locations s to invariant clock constraints
prob probabilistic edge relation that yields the probability of
moving from s to s’ if enabled at s, resetting specified clocks
– Act action labelling of transitions μ (probab. or point distr.)
– L: S ! 2AP atomic propositions
Probabilistic Timed Automata: semantics
• Assume n clocks,
– t,t’ 2 R¸0, v,v’ 2
Rn
send
¸0
1
{x:=0} x=8
wait
waited
x≤8
true
• Markov decision process:
x¸4
– (S,s0,Steps,Act[R¸0,L)
– S, states (s,v), where
0.01
transmit
0.99
• s location, v clock valuation, v ² Inv(s)
– Steps, distributions μ, time points t:
time elapse (s,v) ! σ, μ (s,v+t), μ point distribution
if σ=t 2 R¸0 and Inv(s) satisfied by v+t and v+t’ for all 0 · t’ · t
discrete transition (s,v) ! σ, μ (s’,v’)
if σ=a 2 Act and 9 μ 2 prob enabled at (s,v) and probability of
moving to (s’,v’) resetting clocks in X is induced from prob
– s0,L, initial state and state labelling induced from PTA
ok
true
PTAs: costs and rewards
• Add cost function
– rate, r, of cost accumulation,
proportional to time t
– cost, c, assigned to event
– e.g.
send
1
{x:=0} x=8
wait
waited
x≤8
true
x¸4
r=1
c(transmit)=2
c(waited)=0
• Formally, annotate (Loc,s0,Inv,prob,Act,L) with
0.01
transmit
0.99
ok
true
cost function (r,c):
– r is in R¸0 : rate at which cost is accumulated as time passes
– c maps from (discrete and time elapse) events to R¸0 : eventcost function assigning a cost to each event
• Soluble via stochastic shortest path [CY’90,BT’91,dAl’97]
• Generalisation of uniformly priced timed automata
• Special case: time=cost, with r=1 and c(.)=0
Paths and adversaries
• Unfold into finite/infinite paths s0 σ 0 μ0 s1 σ 1 μ1 s2 σ 2 μ2 s3…
– divergent if for any t 2 R 9 j 2 N s.t. D(j) > t
where D(j) is duration up to j-th state
• Adversary A is a function from finite paths to timedistribution pairs
– s.t. time divergent, i.e. for each state s, the probability of
divergent paths under A is 1
Require non-zenoness,
i.e. there must exist a
divergent adversary
• Probability measure PrAs on paths π 2 PathAs generalises to
this case (basic sets determined by intervals)
The logic PTCTL
• Probabilistic Timed CTL for PTAs
– Based on TCTL [AD94]
– Add probabilistic operator P » p(¢) of PCTL, and p is a
probability threshold
• Syntax (without expectation operator)
φ ::= a | ζ | φ Ç φ | :φ | z.[φ] | P» p(φ U φ)
where z ranges over formula clocks, ζ are clock constraints (zones)
over formula and system clocks
• Example: z.[ P¸ 0.98 (} delivered Æ z < 5)]
“under any scheduling, with probability ¸ 0.85 the message
is correctly delivered within 5 ms”
• Semantics derived from PCTL and TCTL
PTCTL semantics
• Let s 2 S, E be a formula clock valuation
• The probabilistic operator is from PCTL
– “ under any scheduling, the probability bound is true at s,E ”
s,E ² P» p(φ1 U φ2) , PrA { π 2 PathAs j π,E ² φ1 U φ2 } » p
for all A 2 Adv
• Semantics of remaining formulas is standard:
s,E ² a
s,E ² :φ
s,E ² φ1 Ç φ2
π,E ² φ1 U φ2
,
,
,
,
a 2 L(s)
s,E 3 φ
s,E ² φ1 or s,E ² φ2
π = s0L and 9 i 2 N, t · D(i+1)-D(i) s.t.
π(i)+t,E+D(i)+t ² φ2
if t’<t, then π(i)+t’,E+D(i)+t’ ² φ1 Ç φ2
if j<i and t’<D(i+1)-D(i),then π(j)+t’,E+D(j)+t’ ² φ1 Ç φ2
Model checking for PTAs: regions
• Region equivalence
– finite partition of TA
state space
– e.g. x=y=0,
x=0 Æ 0<y<1
– time abstract region
graph
y
(kmax,kmax)
• Quotient preserves
satisfaction
– clock constraints
– TCTL formulas
(0,0)
x
• Idea: Generalise region automaton to MDP over regions
Model checking: regions
• Main result [KNSS’99,’02]
–
–
–
–
–
Can adapt the region graph construction [ACD’93] to PTAs
Path divergence also adapts
Obtain time-abstract, finite-state MDP over regions
Full PTCTL is preserved via region quotient
Can translate PTCTL to PCTL, map H(.), such that
φ satisfied in PTA iff H(φ) in the induced MDP over regions
– Can model check the MDP using standard methods
• Problem: prohibitive complexity (exponential in number of
clocks and size of largest constant), cannot handle expected
costs
• Not implemented
Model checking for PTAs: digital clocks
• ε-digitisation [HMP92]
– restrict to closed,
diagonal-free TAs
– Time domain N, with
integer-valued clocks
– Define time increment
by min{v(x)+t, kx +1)
– Integer-valued time
elapse
y
(0,0)
• Finiteness of state space immediate
• Preserves a subset of properties, cf reachability
(kmax,kmax)
x
Model checking: digital clocks
• Main result [KNS’02,03]: digitisation preserves
– minimum/maximum reachability probability
– minimum/maximum expected cost reachability
• Digitally-clocked PTAs (and variables representing cost) can
be represented straightforwardly in the PRISM input
language, and so can apply model checking directly on MDPs
• Restriction to closed, diagonal-free not important for many
case studies
• Subset of PTCTL only, but expected costs possible
• Problem: inefficiency for some models, as large constants
give rise to very large state spaces
Model checking for PTAs: symbolic
• Zones
– usually convex
conjunctions of atomic
constraints,
– e.g. 0<x<2 Æ 0<y<1
– algebra of operations on
zones: conjunctions, pre,
post
– time abstract zone graph
y
(0,0)
• Explore the zones, constructing the graph
– Forwards, using post (UPPAAL) – on the fly
– Backwards, using pre (KRONOS)
(kmax,kmax)
x
Model checking: forwards
• Main result [KNSS’99,’02][DKN’02][WK’05]
–
–
–
–
–
Can adapt the forwards zone graph construction to PTAs
Obtain time-abstract, finite-state MDP over zones
Bound on maximum probabilities of reachability only
Can model check the MDP using standard methods
Loss of on-the fly, must construct MDP first
• Implementations:
– KRONOS pre-processor into PRISM input language, outputs
time-abstract MDP [DKN’02]
– Explicit, using Difference Bound Matrices (DBMs), to PRISM
input language [WK’05]
– Symbolic, using Difference Decision Diagrams (DDDs), via
MTBDD-coded PTA syntax directly to PRISM engine [WK’05]
• Problem: even maximum probabilities are bounds
Model checking: backwards
• Main result [KNS01, KNSW04]
– Can adapt the backwards TA approach to PTAs
– Can calculate both minimum and maximum probabilities
– Must compute conjunctions of zones to preserve probabilistic
branching
– Obtain time-abstract, finite-state MDP over zones
– Full PTCTL is preserved via quotient
– Loss of on-the fly, must construct MDP first
• Experimental implementation
– Implemented in Java, using Difference Bound Matrices (DBMs)
– Explicit, into PRISM input language
• Problem: need to consider non-convex zones (unions of
zones, i.e. lists of DBMs)
Symbolic model checking
• Symbolic states
–
pairs (l, ζ), where
ƒ
ƒ
l ∈ Loc (location)
ζ is a zone over PTA clocks and formula clocks
• Operations on (sets) of symbolic states
–
–
–
z.U – clock reset
U ∧ V – conjunction
U ∨ V – disjunction
–
tpre(U,V) – timed predecessor, all states from which one can
reach state in V by letting time elapse, without leaving U
–
dpre(e,V) – discrete predecessor, all states from which one can
reach state in V, by taking the edge e
PTCTL model checking for PTAs
Algorithm PTCTLModelCheck
Input: PTA, PTCTL property θ
Output: set of symbolic states « θ ¬ such that
«a¬
«ζ¬
«:φ¬
«φÇψ¬
:= { (l,inv(l)) : l 2 Loc and l 2 L(a) };
:= { (l,inv(l) Æ ζ ) : l 2 Loc };
:= { (l,inv(l) Æ : Ç(l,ζ) 2 « φ ¬ ζ ) : l 2 Loc };
:= « φ ¬ Ç « ψ ¬;
« z.[φ] ¬
:= { (l , [z:=0]ζ) : (l,ζ ) 2 « φ ¬ };
« P» p[φ U ψ] ¬ := Until( « φ ¬, « ψ ¬, » p );
Model checking Until
• Sufficient to compute maximum/minimum probability
• This is possible since
{s,ξ╞ P ~ p(φ1 U φ2 )} =
{s,ξ | pmax(s,ξ , φ1 U φ2 )~p} if ~ ∈ {≤, <}
{s,ξ | pmin (s,ξ , φ1 U φ2 )~p} if ~ ∈ {≥, >}
where for any PTCTL formula ϕ, fixed s,E:
pmax(s,ξ , ϕ) = supA ∈ Adv PrA { π ∈ PathAs | π,ξ╞ ϕ}
pmin(s,ξ , ϕ) = infA ∈ Adv PrA { π ∈ PathAs | π,ξ╞ ϕ}
Quantitative maximum probabilities
Maximum probability of reaching l4
(l1,y≥x) (l1,y=x) (l1,x≥y)
l1
true
½
½
l3
true
l2
true
y≥x
x≥y
l4
true
½
(l2,y≥x)
½
½ ½
(l3,x≥y)
(l4,true)
Start with set of target symbolic states
Backwards exploration: dpre(tpre(.,.))
Continue exploration
Predecessors from the same probabilistic transition: take
conjunction of zones to preserve probabilistic branching
Minimum probabilities
• Problem: restriction to divergent adversaries
– the minimum probability of reaching z > 1, for z a clock, equals:
• 1 under divergent adversaries
• 0 under all adversaries, e.g. consider any adversary which lets time
converge to a value < 1
• Solution: based on the on classical approach for TAs
• Simple case: AFφ (= true ∀U φ):
– find state satisfying the dual formula EG¬φ
– set of states satisfying EGφ is the greatest fixpoint of
φ ∧ z.( X ∃U z>c )
• 1 iteration: satisfy φ
• 2 iterations: can satisfy φ until c time units have passed, …
• k+1 iterations: can satisfy φ until k⋅c time units have passed
– where c is any constant greater than 0
Minimum probabilities: qualitative
• Corresponding dual for PTAs:
pmin(s,ξ, Fφ) = 1 - pmax(s,ξ, G¬φ)
• Qualitative case:
– {s,ξ | pmax(s,ξ, Gφ)=1 } equals the greatest fixpoint of
φ ∧ z. ¬ P <1(X U (X ∨ z>c))
s,ξ╞ ¬ P <1(X U (X ∨ z>c)) if and only if
from s,ξ max probability of remaining in X until c time units elapse is 1
•
•
•
•
1 iteration: satisfy φ
2 iterations: can satisfy φ until c time units have passed with probability 1
…
k+1 iterations: can satisfy φ until k⋅c time units have passed with probability 1
Minimum probabilities : quantitative
• Quantitative case reduces to computing:
– The maximum probability of an until formula (which we have
seen)
– The set of states {s,ξ | pmax(s,ξ, Gφ)=1 }
– Based on the following result:
pmax(s,ξ, Gφ) = pmax(s,ξ, φ U {s,ξ | pmax(s,ξ, Gφ)=1 } )
• In the general case (φ1 U
φ2) use the dual of until: release
pmax(s,ξ, φ1 R φ2) = pmax(s,ξ , φ2 U {s,ξ | pmax(s,ξ, φ1 R φ2)=1 } )
– {s,ξ | pmax(s,ξ, φ1Rφ2)=1 } again computed via a greatest fixpoint
Experimental results: FireWire
• IEEE 1394 FireWire root contention
– Based on model of Stoelinga, without probabilities
– Randomised leader election protocol
– If contention occurs, the node selects a short or long waiting
time at random and then repeats this process
• Model checking
– Probability of leader being elected within a time bound
– Expected time, number of rounds and power consumption to
leader election
– Use forwards/backwards algorithms
– Compare with the digital clocks approach
• Also IEEE 802.3 CSMA/CD (Carrier Sense, Multiple Access
with Collision Detection)
FireWire: expected time
• Study effect of using a biased coin on maximum time to
•
elect leader
Performance improves if ‘fast’ is favoured
FireWire: expected time
• Minimum expected time the same
• Both abstract and full model agree
FireWire: expected number of rounds
• Expected number of rounds to leader election
• Varies depending on coin bias
FireWire: number of states
time bound
backwards
forwards
digital clocks
2
states size (KB) states
1,219
7.24
825
size (KB)
18.9
states
80,980
size (KB)
554
4
4,844
30.6
2,329
35.2
434,364
730
6
10,981
55.0
3,833
51.9
1,093,658
860
8
-
-
6,841
74.1
1,915,291
875
10
-
-
9,661
90.1
2,746,691
875
20
-
-
35,041
204
6,903,691
890
FireWire: computation time
time
bound
backwards
forwards
digital clocks
construct.
m/c
construct.
m/c
2
544+33.0
0.10
0.4+0.6
0.38
10.2
7.8
4
26,992+753
0.34
0.9+2.0
0.80
38.3
43
6
618,493+4,388
1.3
1.6+3.7
1.4
85.8
145
8
-
-
2.9+10
1.6
145
228
10
-
-
4.2+20
2.5
205
335
20
-
-
18+226
5.1
549
469
construct. m/c
Challenges for future
• Exploiting structure
•
•
•
•
•
–
–
–
–
Partial order reduction, in progress
Abstraction
Symmetry reduction
Compositionality
Parametric probabilistic verification?
Approximation methods
Combination with simulation
Statistical testing and model checking
Continuous PTAs
– Efficient model checking methods?
• More expressive specifications
– Probabilistic LTL/PCTL*/mu-calculus?
• Real software, not models!
Continuous PTAs
• Allow clock reset according
•
to cont. probability
distribution
Region graph no longer works
[Alur]
– Set x to random[0,1], y to 0
– When x < 1, reset y to
random[0,1]
– Consider transitions x=1, y=1
– If y < 0.5, x = 1 first,
else don’t know (error)
(Rmax,Rmax)
y
1
(0,0)
1
x
• Can approximately model check by subdividing region graph
• Problem: prohibitive complexity!!!
For more information…
• Rutten, M. Kwiatkowska, G. Norman and D. Parker, Mathematical Techniques
•
•
•
•
for Analyzing Concurrent and Probabilistic Systems. Volume 23 of CRM
Monograph Series. AMS (2004). P. Panangaden and F. van Breugel (eds.).
M. Kwiatkowska, G. Norman, D. Parker and J. Sproston. Performance Analysis
of Probabilistic Timed Automata using Digital Clocks. Formal Methods in
System Design, Springer. To appear, 2006. Earlier version appeared in
FORMATS’03.
M. Kwiatkowska, G. Norman, J. Sproston and F. Wang. Symbolic Model
Checking for Probabilistic Timed Automata. In Proc. FORMATS/FTRTFT'04,
volume 3253 of LNCS, pages 293-308, Springer (2004).
M. Kwiatkowska, G. Norman, R. Segala and J. Sproston. Automatic
Verification of Real-time Systems with Discrete Probability Distributions.
Theoretical Computer Science, 282, pages 101-150 (2002).
M. Kwiatkowska, G. Norman and J. Sproston. Symbolic computation of
maximal probabilistic reachability. In Proc. CONCUR'01, volume 2154 of
LNCS, pages 169-183, Springer (2001).
www.cs.bham.ac.uk/~dxp/prism/
• Case studies, statistics, group publications
• Download, version 3.0 (3400+ downloads)