The Algorithmics
of
Probabilistic Automata
Weak Bisimulation
Andrea Turrini
Saarland University
based on a joint work with
Holger Hermanns
Probabilistic Automata (PA)
s
coin
coin
f
u
retry
flip
8/16
8/16
beepf
Andrea Turrini
Saarland University
flip
12/16
tf
hf
sms
flip
sms
flashf
tu
sms
flashu
4/16
hu
sms
beepu
n
sms
vibrate
Probabilistic Automata (PA)
PA = (Q, q0, E, H, D)
Transition relation
D  Q  (E  H )
 Disc(Q)
Internal (hidden) actions
External actions: E  H = 
Initial state: q0  Q
States
Andrea Turrini
Saarland University
Weak bisimulation
Weak bisimulation between A1 and A2
 s, t, a, m  r
Equiv. Rel. R  Q  Q
+ Q , such that
Q = Q1 
2
t
p
s0
R
t
s1
b
t
1-p
s2
b
s3
s
t1
b
s4
Andrea Turrini
Saarland University
t3
a
a
m Rr

m
R
r
[JL91]
C  Q/R . m (C) = r (C)
Usual decision procedure
Bisimulation(A1, A2)
N
FindSplit(W)
+
W := {Q1 
Q2}
+
N for each (s, a, m)  D1 
D2
(C, a) := FindSplit(W)
N for each t  [s]W
r such
if there does
not
exist
a
while C   do
W := Refine(W, a, C)
(C, a) := FindSplit(W)
that t
r and m W r
return ([s]W, a)
return (, a)
return W
+ D |, | Q 
+ Q |)
N = max (| D1 
2
1
2
Andrea Turrini
Saarland University
Weak transition
s
coin
coin
f
u
retry
flip
8/16
hf
sms
beepf
flip
8/16
flip
12/16
tf
sms
flashf
tu
sms
flashu
4/16
hu
sms
beepu
n
sms
vibrate
{beep: 5/16, flash: 7/16, vibrate: 4/16} {beep: 2/16, flash: 2/16, vibrate: 12/16}
Andrea Turrini
Saarland University
How many weak transitions?
t
t
t
t
t
t
t
t
t
t
C00: bbbbb
C01: bbbbt
C02: bbbtb
C30: ttttb
C31: ttttt
Andrea Turrini
Saarland University
Weak transition as Flow
s
coin
coin
f
u
retry
flip
8/16
hf
sms
beepf
flip
8/16
flip
12/16
tf
sms
flashf
tu
sms
flashu
4/16
hu
sms
beepu
{beep: 5/16, flash: 7/16, vibrate: 4/16}
Andrea Turrini
Saarland University
n
sms
vibrate
Weak transition vs. Flow Problem
s
coin
f
coin
16/16
16/16
hf
sms
beepf
flip
8/16
flip
12/16
tf
sms
16/16
u
retry
flip
8/16
16/16
tu
sms
16/16
flashf
4/16
hu
sms
16/16
flashu
Saarland University
n
sms
16/16
beepu
{beep: 0/16, flash: 16/16, vibrate: 0/16}
Andrea Turrini
16/16
16/16
vibrate
Flow as Linear Programming Problem
min S fi
under constraints
0 ≤ fi ≤ ci (< )
f0 + f 8 = f1 + f4
f1 = f 2 + f3
f2 = f 9
f3 = f10
f4 = f 5 + f6 + f7
f5 = f8 + f11
f6 = f12
f7 = f13
s
beep
flash
vibrate
f0 = 1
f9 + f12 = 5/16
f10 + f11 = 7/16
f13 = 4/16
Andrea Turrini
Saarland University
f0
f1 8/16
1
s
f4
8/16
f
4/16
f2
u
0
4/16
f3
f8
f5
8/16 8/16
12/16
tf
hf
tu
4/16
4/16
f9
f10
Polynomial!
beepf
f2 = 8/16(f2 + f3)
f3 = 8/16(f2 + f3)
f5 = 12/16(f5 + f6)
f6 = 4/16(f5 + f6)
flashf
flashu
f6
1/16
f7
4/16
hu
3/16
f11
4/16
3/16
n
4/16
1/16
f12
beepu
{beep: 5/16, flash: 7/16, vibrate: 4/16}
f13
vibrate
✓
Complexity:
#variables  O(|Q||D|)
#constraints  O(|Q||D|)
Usual decision procedure
Bisimulation(A1, A2)
N
FindSplit(W)
+
W := {Q1 
Q2}
+
N for each (s, a, m)  D1 
D2
(C, a) := FindSplit(W)
N for each t  [s]W
not exist r such
a
Polynomial if there does
while C   do
that t
W := Refine(W, a, C)
(C, a) := FindSplit(W)
return W
Saarland University
return ([s]W, a)
return (, a)
Polynomial!


N = max (| D1
Andrea Turrini
r and m W r
+
D2 |, | Q1
+
Q2 |)
From bisimulation to minimal automaton
A = (Q, q0,
E, H, D)
zeroconf-nt: |Q| = 670, |D| = 827
Quotienting
QA = (W, [q0]W, E, H, DW)
zeroconf-nt: |W| = 41, |DW| = 55
Transition reduction
mA = (W, [q0]W, E, H, DW)
Andrea Turrini
Saarland University
zeroconf-nt: |W| = 41, |DW| = 52
Adding costs to PA
c
s
tr
h0
1-p
p
r2
h1
1-p
1
1
tr
p
r2
…
1-p
tr
p
r2
d
hn-1
1-p
C(csh0) = 1 x 1
C(h0trh1) = r2 x p
C(h0trh0trh1) = (r2 + r2) x (1-p) x p
C(h0trh0trh0trh1) = (r2 + r2 + r2) x (1-p) x (1-p) x p
C((h0tr)ih1) = (i + 1) x r2 x (1-p)i x p
Overall cost: 1 + nr2/p + 1
Andrea Turrini
Saarland University
tr
p
r2
r2 / p
hn
Adding costs to PA
c
s
tr
h0
1-p
1
p
1
tr
p
r2
tr
…
1-p
p
r2
d
tr
hn-1
p
1-p
hn
r2
Polynomial!
0
c
1
h1
1-p
r2
1
1
r2
1 p
h0
1-p
r2
(1-p) / p
r2
1 p
h1
1-p
r2
(1-p) / p
r2
1 p
…
1-p
r2
(1-p) / p
r2
1 p
hn-1
1-p
r2
(1-p) / p
hn
1
1
c
1
Minimum cost equals optimal LP solution
Andrea Turrini
Saarland University
0
Nondeterminism and Cost PA
c
s
h0
1-p
t5
p
52 o
q1
1-p
1
t5
h1
p
1-p
0
52 o
t4
p
42
d
1
q2
1-p
t5
h2
p
1-p
0
52 o
t4
p
42
q3
1-p
t5
h3
p
1-p
0
p
q4
1-p
2 + 89/p  cost  2 + 100/p
Andrea Turrini
Saarland University
0 o
52
t4
42
h4
t4
p
42
q5
Comparing Cost PA
c
s
t5
h0
p
1-p
52 o
1
t5
h1
p
1-p
0
q1
52 o
t4
1-p
0
52 o
t4
0
p
h4
0 o
52
t4
t4
q4
p
1-p
42
t5
h3
1-p
q3
p
1-p
42
p
1-p
q2
p
t5
h2
 2 + 89/p
d
1
p
1-p
42
q5
42
w
s
k0
1-p
t4
k1
p
42
1-p
Andrea Turrini
Saarland University
1
t4
p
42
1
k2
1-p
t4
p
42
k3
1-p
2 + 80/p
d
t4
p
42
k4
1-p
t4
p
42
k5
Minor Cost Weak Bisimulation
Andrea Turrini
Saarland University
Comparing Cost PA
c
s
t5
h0
p
1-p
52 o
1
t5
h1
p
1-p
0
q1
52 o
t4
1-p
0
52 o
t4
0
p
h4
0 o
52
t4
t4
q4
p
1-p
42
t5
h3
1-p
q3
p
1-p
42
p
1-p
q2
p
t5
h2
 2 + 89/p
d
1
p
1-p
42
q5
42
w
s
k0
1-p
t4
k1
p
42
1-p
Andrea Turrini
Saarland University
1
t4
p
42
1
k2
1-p
t4
p
42
k3
1-p
2 + 80/p
d
t4
p
42
k4
1-p
t4
p
42
k5
Conclusion
• Weak transitions as LP problems
• Polynomial decision procedure for PA
weak probabilistic bisimulation
• Minimal automaton
• Cost decorated PA
• Prototypical implementation
• Extensions of LP approach
– Specific weak transitions, hyper-transitions,
restricted transitions, equivalence matching, …
– MDP multi-objective reachability
– Interval multi-objective reachability
– Multi-objective reachability in interval models
• Base for other models: Markov Automata
Andrea Turrini
Saarland University
Decision procedure in practice
Andrea Turrini
Saarland University