A Simple Algorithm for Solving
Qualitative Probabilistic Parity Games
Sven Schewe
University of Liverpool
joint work with Ernst Moritz Hahn, Andrea Turrini & Lijun Zhang
motivation
reachability games & attractors
parity games
Probabilistic Parity Games
R1
zone1
zone2
zone3
zone4
Good abstraction for systems with
choices under our control
choices under environment control
randomised choices
Suitable for linear time properties
R0
hhR0 iiP≥1 [GF(zone1 ∧ Fzone2 )]
Our goal is to win the game with probability 1.
1 / 15
motivation
reachability games & attractors
parity games
Reachability Games
a
d
c
b
e
f
g
Reachability Game R = hV0 , V1 , E , F i
V0 , and V1 are disjoint finite sets of game positions
E ⊆ (V0 ∪ V1 ) × (V0 ∪ V1 ) is a set of edges, and
F ⊆ (V0 ∪ V1 ) is a set of final / accepting game positions
Played by placing a pebble on the arena
– on V0 player 0 chooses a successor, on V1 player 1
⇒ infinite play π
S = (V0 ∪ V1 ) r F
π∈
/ S ω ; player 0 wins, π ∈ S ω ; player 1 wins
2 / 15
motivation
reachability games & attractors
parity games
Solving Reachability Games
arena
F
Algorithm – for R = hV0 , V1 , E , F i
start with the final states F
set W0 to 0-attractor(F )
set W1 to V r W0
σ-attractor(X ): fix-point of X union
for player σ: positions in Vσ that can reach X
for player 1 − σ: positions in V1−σ that can only reach X
3 / 15
motivation
reachability games & attractors
parity games
Solving Reachability Games
arena
W0
Algorithm – for R = hV0 , V1 , E , F i
start with the final states F
set W0 to 0-attractor(F )
set W1 to V r W0
σ-attractor(X ): fix-point of X union
for player σ: positions in Vσ that can reach X
for player 1 − σ: positions in V1−σ that can only reach X
3 / 15
motivation
reachability games & attractors
parity games
Solving Reachability Games
arena
W0
W1
Algorithm – for R = hV0 , V1 , E , F i
start with the final states F
set W0 to 0-attractor(F )
set W1 to V r W0
σ-attractor(X ): fix-point of X union
for player σ: positions in Vσ that can reach X
for player 1 − σ: positions in V1−σ that can only reach X
3 / 15
motivation
reachability games & attractors
parity games
Parity Games
3
2
1
2
3
1
4
Parity Game P = hV0 , V1 , E , αi
V0 , and V1 are disjoint finite sets of game positions
E ⊆ (V0 ∪ V1 ) × (V0 ∪ V1 ) is a set of edges, and
α : (V0 ∪ V1 ) → N is a priority function
Played by placing a pebble on the arena
– on V0 player 0 chooses a successor, on V1 player 1
⇒ infinite play, lowest priority occurring infinite often
even ; player 0 wins, odd ; player 1 wins
4 / 15
motivation
reachability games & attractors
parity games
State of the Art
# priorities
3
4
5
6
7
8
McNaughton
O(m n2 )
O(m n3 )
O(m n4 )
O(m n5 )
O(m n6 )
O(m n7 )
Browne & al.
O(m n3 )
O(m n3 )
O(m n4 )
O(m n4 )
O(m n5 )
O(m n5 )
Jurdziński
O(m n2 )
O(m n2 )
O(m n3 )
O(m n3 )
O(m n4 )
O(m n4 )
w.o. strategy / [GW15]
O(m n)
O(m n2 )
1 12
Big Steps [S07]
O(m n)
O(m n )
O(m n2 )
[CHL15]
O(n2.5 )
O(n3 )
O(n3 3 )
1
O(m n3 )
2 13
O(m n )
3
O(n3 4 )
3
O(m n2 4 )
1
O(n4 16 )
1
O(m n3 16 )
9
O(n4 20 )
but if you don’t tell the games that they are hard
McNaughton is by far the fastest
5 / 15
motivation
reachability games & attractors
parity games
McNaughton’s Algorithm
arena
α−1 (c)
McNaughton’s Algorithm – for P = hV0 , V1 , E , αi
set c to the minimal priority, σ to c modulo 2, and σ to 1 − σ
set A to σ-attractor α−1 (c)
set (U0 , U1 ) to McNaughton(P r A)
set Wσ to σ-attractor(Uσ ), and set Wσ to ∅
set (U0 , U1 ) to McNaughton(P r Wσ )
˙ 0 , W1 ∪U
˙ 1)
return (W0 ∪U
6 / 15
motivation
reachability games & attractors
parity games
McNaughton’s Algorithm
arena
A
McNaughton’s Algorithm – for P = hV0 , V1 , E , αi
set c to the minimal priority, σ to c modulo 2, and σ to 1 − σ
set A to σ-attractor α−1 (c)
set (U0 , U1 ) to McNaughton(P r A)
set Wσ to σ-attractor(Uσ ), and set Wσ to ∅
set (U0 , U1 ) to McNaughton(P r Wσ )
˙ 0 , W1 ∪U
˙ 1)
return (W0 ∪U
6 / 15
motivation
reachability games & attractors
parity games
McNaughton’s Algorithm
arena
Uσ
A
Uσ
McNaughton’s Algorithm – for P = hV0 , V1 , E , αi
set c to the minimal priority, σ to c modulo 2, and σ to 1 − σ
set A to σ-attractor α−1 (c)
set (U0 , U1 ) to McNaughton(P r A)
set Wσ to σ-attractor(Uσ ), and set Wσ to ∅
set (U0 , U1 ) to McNaughton(P r Wσ )
˙ 0 , W1 ∪U
˙ 1)
return (W0 ∪U
6 / 15
motivation
reachability games & attractors
parity games
McNaughton’s Algorithm
arena
Wσ
McNaughton’s Algorithm – for P = hV0 , V1 , E , αi
set c to the minimal priority, σ to c modulo 2, and σ to 1 − σ
set A to σ-attractor α−1 (c)
set (U0 , U1 ) to McNaughton(P r A)
set Wσ to σ-attractor(Uσ ), and set Wσ to ∅
set (U0 , U1 ) to McNaughton(P r Wσ )
˙ 0 , W1 ∪U
˙ 1)
return (W0 ∪U
6 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Probabilistic Parity Games
3
2
1
2
3
1
4
Probabilistic Parity Game P = hV0 , V1 , VR , E , αi
V0 , V1 , and VR are disjoint finite sets of game positions
E ⊆ (V0 ∪ V1 ∪ VR ) × (V0 ∪ V1 ∪ VR ) is a set of edges, and
α : (V0 ∪ V1 ∪ VR ) → N is a priority function
Played by placing a pebble on the arena
– on V0 player 0 chooses a successor, on V1 player 1
– on VR a successor is chosen randomly
⇒ infinite play, lowest priority occurring infinite often
even ; player 0 wins, odd ; player 1 wins
7 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Removing the Random Player: Gadget Construction
random vertex with priority 0
0
0
0
0
8 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Removing the Random Player: Gadget Construction
random vertex with priority 1
1
1
1
1
0
1
8 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Removing the Random Player: Gadget Construction
random vertex with priority 2
2
2
2
2
0
1
2
8 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Removing the Random Player: Gadget Construction
random vertex with priority 3
3
3
3
3
0
1
3
2
3
8 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Removing the Random Player: Gadget Construction
random vertex with priority 4
4
4
4
4
0
1
4
2
3
4
8 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Different Attractors
Weak attractors
reach goal almost surely
Strong attractors
reach goal with positive probability
⇒ treat probabilistic nodes as “yours”
Note: weak attractor(G ) ⊆ strong attractor(G )
9 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Probabilistic McNaughton
arena
α−1 (c)
Prob-McNaughton’s Algorithm – for P = hV0 , V1 , VR , E , αi
set c to the minimal priority and σ to c modulo 2
treat random nodes as player σ vertices
set A to σ-attractor α−1 (c)
set (U0 , U1 ) to Prob-McNaughton(P r A)
set Wσ to σ-attractor(Uσ ), and set Wσ to ∅
set (U0 , U1 ) to Prob-McNaughton(P r Wσ )
˙ 0 , W1 ∪U
˙ 1)
return (W0 ∪U
10 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Probabilistic McNaughton
arena
A
Prob-McNaughton’s Algorithm – for P = hV0 , V1 , VR , E , αi
set c to the minimal priority and σ to c modulo 2
treat random nodes as player σ vertices
set A to σ-attractor α−1 (c)
set (U0 , U1 ) to Prob-McNaughton(P r A)
set Wσ to σ-attractor(Uσ ), and set Wσ to ∅
set (U0 , U1 ) to Prob-McNaughton(P r Wσ )
˙ 0 , W1 ∪U
˙ 1)
return (W0 ∪U
10 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Probabilistic McNaughton
arena
Uσ
A
Uσ
Prob-McNaughton’s Algorithm – for P = hV0 , V1 , VR , E , αi
set c to the minimal priority and σ to c modulo 2
treat random nodes as player σ vertices
set A to σ-attractor α−1 (c)
set (U0 , U1 ) to Prob-McNaughton(P r A)
set Wσ to σ-attractor(Uσ ), and set Wσ to ∅
set (U0 , U1 ) to Prob-McNaughton(P r Wσ )
˙ 0 , W1 ∪U
˙ 1)
return (W0 ∪U
10 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Probabilistic McNaughton – σ = 0, σ = 1
arena
Wσ
Prob-McNaughton’s Algorithm – for P = hV0 , V1 , VR , E , αi
set c to the minimal priority and σ to c modulo 2
treat random nodes as player σ vertices
set A to σ-attractor α−1 (c)
set (U0 , U1 ) to Prob-McNaughton(P r A)
set Wσ to σ-attractor(Uσ ), and set Wσ to ∅
set (U0 , U1 ) to Prob-McNaughton(P r Wσ )
˙ 0 , W1 ∪U
˙ 1)
return (W0 ∪U
10 / 15
probabilistic parity games
gadgets
attractors
probabilistic McNaughton
Probabilistic McNaughton – σ = 1, σ = 0
arena
Wσ
Prob-McNaughton’s Algorithm – for P = hV0 , V1 , VR , E , αi
use weak attractor
(instead of the strong attractor)
co-game of the weak attractor might have sub-stochastic
vertices
they are turned into “good” vertices, by
giving them priority 0 or
adding an accepting sink as successor
⇒ visiting them infinitely often wins
10 / 15
Experimental Results
property
Reachability
hhR0 iiP≥1
[ F zone1
∧ F zone2 ]
Repeated
Reachability
hhR0 iiP≥1
[ GF zone1
∧ GF zone2
∧ GF zone3
∧ GF zone4 ]
Repeated
Ordered
Reachability
hhR0 iiP≥1
[GF(zone1 ∧
F zone2 )]
Reach-avoid
hhR0 iiP≥1
[ ¬zone1 U zone2
∧ ¬zone4 U zone2
∧ ¬zone4 U zone1
∧ Fzone4 ]
game construction
vertices
priorities
tprod
gadget construction
vertices
tgadg
solving time
tgMcN
tgJur
tpMcN
n
sat
12
16
20
24
28
32
36
40
true
true
true
true
true
true
-
1 324 704
4 418 592
11 050 656
23 211 552
43 334 304
74 294 304
-
2
2
2
2
2
2
-
2
9
22
51
102
197
–MO–
–MO–
5 351 584
17 996 832
45 166 752
95 045 152
177 634 464
-
3
13
28
58
115
–MO–
-
0
3
8
18
35
-
4
24
84
219
–MO–
-
0
1
3
7
14
24
-
12
16
20
24
28
32
36
40
true
true
true
true
true
true
-
1 324 704
4 418 592
11 050 656
23 211 552
43 334 304
74 294 304
-
2
2
2
2
2
2
-
2
9
20
44
88
180
–MO–
–MO–
6 010 528
20 085 792
50 273 952
105 643 552
197 278 368
-
3
15
29
59
121
–MO–
-
1
3
9
21
38
-
5
25
85
227
–MO–
-
0
1
3
8
15
27
-
12
16
20
24
28
32
36
40
true
true
true
true
true
true
true
true
693 048
2 264 184
5 611 320
11 729 784
21 836 088
37 367 928
59 984 184
91 564 920
5
5
5
5
5
5
5
5
0
3
6
14
27
54
65
121
4 009 976
13 206 072
32 844 792
68 787 512
128 198 136
219 543 096
-
2
9
17
62
69
135
–MO–
–MO–
0
2
6
13
24
–MO–
-
3
16
50
129
282
–MO–
-
0
0
1
4
7
13
21
36
12
16
20
24
28
32
36
40
true
true
true
true
true
true
-
1 616 400
5 452 848
13 703 184
28 855 728
53 951 760
92 585 520
-
4
4
4
4
4
4
-
2
14
35
79
171
323
–MO–
–MO–
7 656 720
25 820 208
64 877 328
136 606 128
255 402 000
-
4
15
33
116
135
–MO–
-
1
6
19
40
–MO–
-
–TO–
–TO–
–TO–
–TO–
–MO–
-
0
1
5
11
21
38
-
11 / 15
Summary
solving probabilistic parity games is as fast
as solving parity games
solving probabilistic parity games is as simple
as solving parity games
solving probabilistic parity games is as beautiful
as solving parity games
the same class of algorithms works best
12 / 15
Summary
Solving probabilistic parity games:
fast
simple
beautiful
13 / 15
14 / 15
‘Comparison’ to GIST
vertices
1,000
10,000
50,000
edges
5,000
50,000
250,000
best
0.63
39.38
2063.40
average
1.17
51.43
2513.18
worst
1.59
62.61
2711.23
15 / 15