A Simple Algorithm for Solving Qualitative
Probabilistic Parity Games
Ernst Moritz Hahn1 , Sven Schewe2 , Andrea Turrini1 , and Lijun Zhang1
1
State Key Laboratory of Computer Science, Institute of Software, CAS, China
2
University Of Liverpool, UK
Abstract. In this paper, we develop an approach to find strategies that
guarantee a property in systems that contain controllable, uncontrollable, and random vertices, resulting in probabilistic games. Such games
are a reasonable abstraction of systems that comprise partial control over
the system (reflected by controllable transitions), hostile nondeterminism (abstraction of the unknown, such as the behaviour of an attacker
or a potentially hostile environment), and probabilistic transitions for
the abstraction of unknown behaviour neutral to our goals. We exploit
a simple and only mildly adjusted algorithm from the analysis of nonprobabilistic systems, and use it to show that the qualitative analysis of
probabilistic games inherits the much celebrated sub-exponential complexity from 2-player games. The simple structure of the exploited algorithm allows us to offer tool support for finding the desired strategy, if
it exists, for the given systems and properties. Our experimental evaluation shows that our technique is powerful enough to construct simple
strategies that guarantee the specified probabilistic temporal properties.
1
Introduction
The automated synthesis of reactive protocols (strategies, policies) from temporal specifications has recently attracted considerable attention in numerous
applications. Such a scenario can present both nondeterministic and probabilistic behaviours, so the resulting model can be seen as a Markov Decision Process
(MDP) [31]. MDPs can also be viewed as 1 12 -player games, where the full player
decides which action to perform when resolving nondeterministic choices, and the
1
2 -player (or: random player) resolves the probabilistic choices.
This game can be enriched with a second player, e.g. an environment player,
who controls some of the nondeterministic choices. Usually, the second player
acts in a hostile manner: he tries to prevent the first (full) player from reaching
her goal. The resulting game is known as a 2 12 -player game. Examples of 2 12 player games are security and communication protocols [24, 28, 29, 34, 35] and
robots playing in pursuit-evasion games [20].
A particular application in software engineering is the development of probabilistic reactive protocols and interfaces for probabilistic components. Such an
interface would restrict the interactions a component offers to its environment.
This technically corresponds to choosing a strategy for the component in a game
with its environment, where the goal of the component is to satisfy its specification, while the goal of the environment is to violate it.
The contribution of this paper is to provide an efficient algorithm to synthesise strategies for the controllable player in 2 12 -player games that are equipped
with a parity winning condition. Parity conditions are very general winning conditions that can be used to represent all ω-regular properties. In particular, parity objectives contain temporal logics such as LTL (linear temporal logic [30]),
which are useful in specifying system and program properties. Because of this,
we can handle LTL with a probabilistic semantics, i.e. the qualitative fragment
of PLTL [1], in our synthesis framework.
We focus on computing the regions of the game, in which one of the players
is almost sure winning [9]. The algorithm from [9] is based on translating the
stochastic parity game into a simple parity game [8] (a parity game without random vertices / a 2-player game) using a so called “gadget” construction, albeit
to the cost of a blow-up by a factor linear in the number of priorities. For a small
number of priorities, or colours, the complexity of this algorithm is better than
the algorithm we suggest here, because solving the blown-up simple parity game
with [21] or [32] provides better bounds. This advantage is, however, purely theoretical: even for non-stochastic parity games on which these algorithms can be
applied directly without requiring a costly transformation, they do not perform
well in practice [14], such that a nested fixed-point algorithm [13, 27] would be
the natural choice for analysing the blown-up game.
Our algorithm is an adaptation of the classical nested fixed-point algorithm of
McNaughton [13,27] for the analysis of non-probabilistic systems. In particular, it
does not involve a translation of the game. Thus, we avoid the practical problems
existing algorithms with good complexity have [14]. The simple structure of
the exploited algorithm also allows us to offer tool support by implementing a
protocol synthesiser.
Present algorithms with the best theoretical complexity bounds for solving
2-player parity games with a low or fixed number of priorities are described
in [21, 32]. However, the ranking based approach from [21] does not perform
very well in practice [14], and the hybrid approach from [32] will inherit these
practical draw-backs.
The direct algorithm we describe has exponential worst-case complexity. In
the paper, we exploit the simple structure of our algorithm to lift the subexponential algorithm for 2-players games of Jurdiński, Paterson, and Zwick [22]
to the qualitative analysis of 2 12 -player games.
Related Work. There is a rich body of literature of algorithms for parity games
in a two player setting [13, 21, 22, 27, 32, 36], and a few for multi player games
with concurrent moves [3–5], in which players make simultaneous choices in every
move. All of these algorithms share an nO(n) running time.
Some experiments have suggested that the algorithm proposed in [13, 27,
36] performs best among them, in particular because it can be implemented
symbolically. In this paper, we are considering non-concurrent games. We have
adjusted McNaughton’s algorithm [13, 27, 36] for solving these 2 12 -player games.
R1
zone1
zone2
zone3
zone4
n
4
R0
4
n
Fig. 1. Robot arena.
Such games have also been considered in the literature, for instance in [2, 6, 9,
15, 16]. Like 2-player games, they have pure memoryless optimal strategies.
Algorithms for solving probabilistic games have been implemented among
others in PRISM-games [11]. This tool can, however, only handle a restricted
class of properties, namely PCTL [19], which does not support nested temporal
operators. As mentioned above, our approach can handle properties specified in
the logics PLTL [1], as it translates such properties to parity objectives.
2
A motivating example
In a two-dimensional battlefield (cf. Figure 1), there are two robots, R0 and R1 ,
and four marked zones, zone1 , . . . , zone4 . The battlefield is surrounded by a solid
wall and has a square tiled floor of n fields in breadth and width. Each tile can be
occupied by at most one robot. The robots act in strict alternation. When it is
the turn of a robot, this robot can move as follows: decide a direction and move
one field forward; decide a direction and attempt to move two fields forward. In
the latter case, the robot moves two fields forward with a probability of 50%,
but only one field forward with a probability of 50%. If the robot would run into
a wall or into the other robot, it stops at the field before the obstacle.
We assume that we are in control of R0 but cannot control the behaviour
of R1 . Our goal is to fulfil a certain objective depending on the zones with
probability 1, such as repeatedly visiting all zones infinitely often, visiting the
zones in a specific order, performing such visits without entering other zones in
the meanwhile, and so on. We have to ensure that these objectives are fulfilled
under any possible behaviour of R1 . As an example, we can specify that the
robot eventually reaches zone1 and zone2 , which can be specified using the LTL
formula F zone1 ∧ F zone2 .
3
3.1
Preliminaries
Markov parity games
We now introduce the formal definition of the Markov parity games that model
the 2 12 -player games together with the other concepts and notations we use in
the remainder of the paper.
Definition 1. A finite Markov Parity Game, MPG for short, is a tuple P =
(V0 , V1 , Vr , E, pri), where
– V = V0 ∪ V1 ∪ Vr is the set of vertices, where V0 , V1 , and Vr are three finite
disjoint sets of vertices owned by the three players: Player 0, Player 1, and
Player random, respectively;
– E ⊆ V × V is a set of edges such that (V, E) is a sinkless directed graph,
i.e. for each v ∈ V there exists v 0 ∈ V such that (v, v 0 ) ∈ E; and
– pri : V → N is the priority function mapping each vertex to a natural number.
We call the image of pri the set of priorities (or: colours), denoted by C.
Note that, since the set of vertices V is finite, C is finite as well. For an MPG
P = (V0 , V1 , Vr , E, pri), we call the tuple A = (V0 , V1 , Vr , E) the arena of P. For
ease of notation, we sometimes use games when we refer to their arenas only. We
also use the common intersection and subtraction operations on directed graphs
for arenas and games: given an MPG P with arena A = (V0 , V1 , Vr , E),
– P ∩ V 0 denotes the Markov parity game P 0 we obtain when we restrict the
arena A to A ∩ V 0 = (V0 ∩ V 0 , V1 ∩ V 0 , Vr ∩ V 0 , E ∩ (V 0 × V 0 )).
– P \ V 0 denotes the Markov parity game P 0 with arena A \ V 0 = A ∩ (V \ V 0 ),
where V = V0 ∪ V1 ∪ Vr .
Note that the result of such an intersection may or may not be sinkless. While
we use these operations freely in intermediate constructions, we make sure that,
wherever they are treated as games, they have no sinks (cf. Lemma 4).
Plays. One can view the dynamics of a parity game as a board game, played
by moving a pebble over the game arena. When the pebble is on a vertex v, the
next vertex v 0 is chosen by the player owning the vertex v, that is, by Player 0
if v ∈ V0 , by Player 1 if v ∈ V1 , and by Player random if v ∈ Vr . The respective
player chooses a proper successor v 0 of v, i.e. a vertex v 0 with (v, v 0 ) ∈ E, and
pushes the pebble forward to v 0 . This way, they together construct an infinite
play. If V0 = ∅ or V1 = ∅, we arrive at the model of Markov decision processes
(MDPs).
A play is an infinite sequence π = v0 v1 v2 v3 . . . such that (vi , vi+1 ) ∈ E for all
i ∈ N. Each play is evaluated by the lowest priority that occurs infinitely often.
Player 0 wins a play π = v0 v1 v2 v3 . . . if the lowest priority that occurs infinitely
often in the sequence pri(v0 )pri(v1 )pri(v2 )pri(v3 ) . . ., pri(π) = lim inf i→∞ pri(vi ) is
even, while Player 1 wins if pri(π) is odd. Below, we formalise winning strategies
and winning regions for a given MPG:
Definition 2. For a given MPG P, we say
– For σ ∈ {0, 1}, a strategy fσ of Player σ is a mapping fσ : Vσ → V from the
vertices of Player σ to their successor states, i.e. for each v ∈ Vσ , (v, fσ (v)) ∈
E. A play π = v0 v1 v2 v3 . . . is called fσ -conform if, for all i ∈ N, if vi ∈ Vσ ,
then vi+1 = fσ (vi ). A strategy fσ for the player σ defines an MDP, namely
the MDP where each vertex v ∈ Vσ has exactly one successor, fσ (v).
– Given a vertex v ∈ V , a strategy f0 for Player 0 is called v-winning if,
starting from v, Player 0 wins almost surely in the MDP defined by f0 ; a
strategy f1 for Player 1 is called v-winning if, starting from v, Player 1 wins
with non-zero probability in the MDP defined by f1 , that is, Player 1 does
not lose almost surely in the MDP defined by f1 .
– For σ ∈ {0, 1}, a vertex v in V is v-winning for Player σ if Player σ has a
v-winning strategy fσ . We call the set of v-winning vertices for Player σ the
winning region of Player σ, denoted Wσ .
As we concentrate on finding almost surely winning/losing regions, it suffices
to assume that there is an ε > 0, such that all choices of Player random are
made with probability of at least ε. We therefore omit the probabilities in our
model.
4
McNaughton’s algorithm and memoryless determinacy
In this section, we adjust the classic algorithm for solving 2-player parity games
to Markov parity games. The classic algorithm dates back to McNaughton [27]
and was first published in this form by Emerson and Lei [13] and Zielonka [36].
The algorithm is the algorithmic version of a simple proof of the memoryless
determinacy for parity games. The proof uses an inductive argument over the
number of vertices. As an induction basis, games with only one game vertex are
clearly memoryless determined: there is only one strategy, and it is memoryless.
The game is won by Player 0 if the priority of this vertex is even and by Player 1 if
the priority of this vertex is odd. We provide an algorithm for determining where
Player 0 wins almost surely, i.e. its winning region W0 . Similarly, we compute the
winning region W1 , in which Player 1 wins with non-zero probability. Adjusting
it to finding sure winning sets for Player 1 is straight forward.
Lemma 1. If, for a game (V0 , V1 , Vr , E, pri), the image of pri consists only of
even priorities, then Player 0 wins (surely) from all vertices, and W0 = V .
Procedure prob−McNaughton(P): (P = (V0 , V1 , Vr , E, pri))
if V = ∅ then return (∅, ∅) (note that V = V0 ∪ V1 ∪ Vr )
set c to the minimal priority occurring in P
if the image of pri is even then return (V, ∅)
if the image of pri is odd then return (∅, V )
if c is even then
(a) set W1 to ∅
(b) repeat
i. set P 0 to P \ satr0 (pri−1 (c), P)
ii. set (W00 , W10 ) to prob−McNaughton(P 0 )
iii. if W10 = ∅ then
A. set W0 to V \ W1
B. return (W0 , W1 )
iv. set W1 to W1 ∪ satr1 (W10 , P)
v. set P to P \ satr1 (W10 , P)
6. set W0 to ∅
7. repeat
(a) set P 0 to P \ satr1 (pri−1 (c), P)
(b) set (W00 , W10 ) to prob−McNaughton(P 0 )
(c) if W00 = ∅ then
i. set W1 to V \ W0
ii. return (W0 , W1 )
(d) set W0 to W0 ∪ watr0 (W00 , P)
(e) set P to P \ watr0 (W00 , P)
adding a sink vertex reachable from the random vertices adjacent to W0 (cf.
Lemma 7); this sink is a technicality and does not count when we check W00 = ∅
in line 7.c
1.
2.
3.
4.
5.
Fig. 2. The algorithm prob−McNaughton(P) returns the ordered pair (W0 , W1 ) of
winning regions of Player 0 and Player 1, respectively. V and pri denote the states and
the priority function of the parity game P.
If, for a game (V0 , V1 , Vr , E, pri), the image of pri consists only of odd priorities, then Player 1 wins (surely) from all vertices, and W1 = V .
For general parity games P with lowest priority c, our adjustment of the
McNaughton’s algorithm, the procedure ‘prob−McNaughton’ shown in Figure 2,
first determines the set pri−1 (c) of vertices with priority c, i.e. the vertices with
minimal priority. If c is even, then Player 0 wins on all plays, where c occurs
infinitely often. The algorithm then constructs the region, from which Player 1
cannot almost surely avoid to reach a vertex with priority c. This is obtained
using attractors.
4.1
Attractors
Attractors are usually defined for classical 2-player games. For an arena A =
(V0 , V1 , E), a set T ⊆ V of target vertices, and a player σ ∈ {0, 1}, the σ-
attractor of T is the set of game vertices from which Player σ can force the
pebble into the set T of target vertices. The σ-attractor A of a set T can be
defined as the least fixed point of sets that contain T and that contain a vertex
v of Player σ if it contains some successor of v and a vertex v of Player 1 − σ if it
contains all the successors of v. Equivalently,
S the σ-attractor σ−Attractor (T, A)
of T in the arena A can be defined as A = j∈N Aj where
A0 = T,
Aj+1 = Aj ∪ { v ∈ Vσ | ∃v 0 ∈ Aj . (v, v 0 ) ∈ E }
∪ { v ∈ V1−σ | ∀(v, v 0 ) ∈ E. v 0 ∈ Aj }.
This definition also provides a memoryless strategy for Player σ to move the
pebble to T from all vertices in A: for a vertex v ∈ A, there is a minimal i ∈ N
such that v ∈ Ai . For i > 0 (i.e. for v ∈
/ T ) and v ∈ Vσ , v has a successor
in Ai−1 , and Player σ can simply choose such a successor. (For v ∈ V1−σ , all
successors are in Ai−1 .) Likewise, A itself provides a memoryless strategy to keep
the pebble out of A (and hence out of T ) for Player 1 − σ when starting from a
vertex v ∈
/ A, namely to never enter A.
For 2 21 -player games, we distinguish two different types of attractors: strong
attractors, where the random player co-operates with the player σ who wants to
push the pebble into the target set (denoted satrσ (T, P) when Player σ tries to
move the pebble to T in the arena of P); and weak attractors, where the target
needs to be reached almost surely. (The ‘strong’ and ‘weak’ in their names are
inspired by satrσ (T, P) ⊇ watrσ (T, P), which makes the first attractor stronger.)
Note that the principle of the attractor construction is not affected by the cooperation for the strong attractor: we simply treat the random vertices as vertices
of player σ and apply the normal attractor construction from 2-player games.
Lemma 2. For an arena A and a set T of target states, the strong σ-attractor
of T can be constructed in time linear in the edges of A.
The construction of weak attractors is more complex since it requires solving
a singly nested fixed-point. Consider an MPG P, a set T ⊆ V , and the player
σ; the weak σ-attractor watrσ (T, P) of the set T is defined as follows:
S0 = satrσ (T, P),
Cj = satr1−σ (V \ Sj , P \ T ),
Sj+1 = satrσ (T, P \ Cj ),
\
watrσ (T, P) =
Sj .
j∈N
As a singly nested fixed point, constructing weak attractors can be reduced to
solving Büchi games, which can be done in O(n2 ) time [7]. (We have implemented
the classic O(m · n) iterated fixed point algorithm.)
Lemma 3. For an arena A and a set T of target states, the weak σ-attractor of
T can be constructed in time quadratic in the states of A.
Note that the co-games of attractors are proper games: they have no sinks.
Lemma 4. For an arena A and a set T of target states, A0 = A \ satrσ (T, A)
and A00 = A \ watrσ (T, A) are arenas for σ ∈ {0, 1}.
For strong attractors, this is because every vertex in (Vσ ∪ Vr ) \ satrσ (T, A)
has the same successors in A0 and A, while every vertex in V1−σ \ satrσ (T, A) has
some successor in A0 . For weak attractors, every vertex in Vσ \ watrσ (T, A) has
the same successors in A00 and A, while every vertex in (V1−σ ∪ Vr ) \ watrσ (T, A)
has some successor in A0 .
4.2
Traps and paradises
After constructing A = satrcmin mod 2 (pri−1 (cmin ), P), the co-game P 0 = P \ A
of P is solved.
A σ-trap Tσ ⊆ V is a set of vertices Player σ cannot force to leave, not even
with some probability greater than 0 (she is trapped there). A set Tσ is a σ-trap
if it is the co-set of a strong σ-attractor, i.e. if it satisfies V \Tσ = satrσ (V \Tσ , P).
The co-game P 0 is smaller than P: compared to P, it has less vertices. By induction hypothesis, it is therefore memoryless determined. By induction over the
size of the game, P 0 can therefore be solved by a recursive call of the algorithm.
The following picture shows how the first part of the algorithm works. We first
determine the minimal priority in the game (line 2 of prob−McNaughton). Let
c := cmin denote the minimal priority, and select the target to be T = pri−1 (c),
shown as the solid green area in the top right corner. We then construct the
respective strong attractor A = satrσ (T, P) with σ = c mod 2, shown as T plus
the NE hatched part around T , and consider the co-game P 0 (lines 5.b.i and 7.a
of prob−McNaughton).
0
We then solve P 0 , resulting in the winning regions W1−σ
, shown in solid red,
0
and Wσ , the green SW hatched part below (lines 5.b.ii and 7.b of the procedure
prob−McNaughton).
arena
′
W1−σ
pri−1 (c)
A
Wσ′
Fig. 3. First part of the algorithm; c is the minimal priority and σ = c mod 2
We fix the target set T . We call a subset P0 ⊆ W0 of the winning region of
Player 0 a 0-paradise if it is a 1-trap and Player 0 has a memoryless strategy f ,
which is v-winning for all v ∈ P0 in the game P ∩ P0 , such that P0 cannot be
left in any f -conform play.
We call a subset P1 ⊆ W1 of the winning region of Player 1 a 1-paradise if
Player 1 has a memoryless strategy f , which is v-winning for all vertices v ∈ P1 ,
such that the probability measure of the plays π with odd pri(π) that never leave
P1 is non-zero for all vertices in P1 . (Note that it suffices for this property to
define f on P1 .) In particular, all vertices in v ∈ P1 are v-winning for Player 1.
0
Returning to the picture, W1−σ
—the solid red area—is a (1 − σ)-paradise for
the game P.
Lemma 5. For a parity game P with a σ-trap Tσ and a (1 − σ)-paradise W1−σ
of P 0 = P ∩ Tσ , W1−σ is a (1 − σ)-paradise for P.
The lemma is obvious: Player 1−σ can simply use the same winning strategy
f for P as for P 0 on Tσ : as Tσ is a σ-trap, neither Player σ nor Player random
have additional moves in P, and every f -conform play that starts in W1−σ in P
0
is also an f -conform play in P 0 . The winning region W1−σ
of P 0 is therefore a
(1 − σ)-paradise in P.
This property can be easily extended to paradises using attractor operations
(strong attractor for player odd and weak attractor for player even):
Lemma 6. The strong 1-attractor A = satr1 (P1 , P) of a 1-paradise P1 for a
parity game P is a 1-paradise for P, and the weak 0-attractor A = watr0 (P0 , P)
of a 0-paradise P0 for a parity game P is a 0-paradise for P. A winning strategy
for the respective player σ on A can be composed of the winning strategy for
Player σ on Pσ and an attractor strategy on A \ Pσ .
As a result, for a given 0-paradise P0 for Player 0 in a parity game P, we
can reduce solving P to computing the weak 0-attractor A = watr0 (P0 , P) of P0 ,
and solving P \ A.
Lemma 7. Let P be a parity game, P0 be a 0-paradise with weak 0-attractor
A = watr0 (P0 , P), and W00 and W10 be the winning regions of Player 0 and
Player 1, respectively, on P 0 , which is obtained from the game P 00 = P \ A by
adding a random vertex s, a sink with a self-loop and even priority, which is
reachable from the random vertices that have (in the parity game P) a successor
in A. Then
– W1 = W10 is the winning region of Player 1 on P, and he can win by following
his winning strategy from P 0 on his winning region, and
– W0 = W00 ∪ A \ {s} is the winning region of Player 0 and she can win by
following her winning strategy for A on A and her winning strategy from P 0
on W00 .
Proof. Player 0 wins with her strategy from every vertex in A by a composition
of the attractor strategy on A \ P0 and her winning strategy on P0 by Lemma 6.
Let g0 be a winning strategy for Player 0 on W00 in P 0 . Consider a probability
distribution on P 0 that summarises the probabilities to transfer to A in each
situation to the same probability to transfer to s, and a fixed counter strategy f
of her opponent. As the likelihood of winning is 1 after transferring to A (with
the strategy from above by Lemma 6) in P and in P 0 (as all plays that reach s
are winning), the chance of winning is equal in both cases. Vertices in W00 \ {s}
are therefore winning for the composed strategy.
Similarly, let g1 be a winning strategy for Player 1 on W10 in P 0 . We consider
a probability distribution on P 0 that summarises the probabilities to transfer to
A in each situation to the same probability to transfer to s, and a fixed counter
strategy f of his opponent. Estimating the chance of winning to 0 after reaching
A results in the same likelihood of winning in P and P 0 . As this is larger than 0,
Player 1 wins with an extension of the same strategy on his winning region. t
u
For our implementation, we have, instead of adding a sink with even priority,
reduced the priority of all random vertices having s as successor to 0 (it could
be set to any other even priority not greater than any priority of the vertices in
P \A). This change is safe to implement: both players have the same memoryless
strategies as in the game with the additional edges. Assume the players play
according to fixed memoryless strategies f (Player 0) and g (Player 1), such
that only stochastic decisions are left open. Player 0 will win almost surely
from a vertex v if, and only if, for all leaf SCCs reachable from v the minimal
priority among all priorities of the states in this leaf SCC is even. The leaf SCCs
reachable from a vertex v other than the singleton leaf SCC {s} from Lemma 7
are exactly those leaf SCCs reachable after re-prioritising, that do not contain a
vertex whose priority is changed to 0. Thus, the same leaf SCCs with minimal
odd priority are reachable. Consequently, the same vertices in A are winning for
Player 0 (and for Player 1) in both constructions.
Corollary 1. Let P be a parity game, P0 be a 0-paradise with weak 0-attractor
A = watr0 (P0 , P), and W00 and W10 be the winning regions of Player 0 and
Player 1, respectively, on P 0 , which is obtained from the game P 00 = P \ A by
changing the priority of the random vertices that have (in the parity game P) a
successor in A to 0. Then
– W1 = W10 is the winning region of Player 1 on P, and he can win by following
his winning strategy from P 0 on his winning region, and
– W0 = W00 ∪ A \ {s} is the winning region of Player 0 and she can win by
following her winning strategy for A on A and her winning strategy from P 0
on W00 .
Note that this change of priority is, like adding the sink s, introduced recursively on a level in the call tree. While it is inherited in calls from there,
the changes introduced in a level of the call tree (in line 7.e) are revoked when
returning the values (in line 7.c.ii).
For a given 1-paradise P1 for Player 1 in a parity game P, we can reduce
the qualitative analysis of a parity game P to computing the strong 1-attractor
A = satr1 (P1 , P) of P1 , and solving P \ A.
Lemma 8. Let P be a parity game, P1 be a 1-paradise with winning strategy f1
and strong 1-attractor A = satr1 (P1 , P), and W00 and W10 be the winning regions
of Player 0 and Player 1, respectively, on P 0 = P \ A. Then
– W1 = W10 ∪ A is the winning region of Player 1 on P, and he can win by
following his winning strategy for A on A and his winning strategy g1 from
P 0 on W10 , and
– W0 = W00 is the winning region of Player 0, and she can win by following
her winning strategy g0 from P 0 on W0 .
Proof. Player 0 wins with her winning strategy, g0 , on her complete winning
region W0 of P \ A, since Player 1 has no additional choices in W0 in P. Consequently, the set of g0 -conform plays in P that start in W0 coincides with the set
of g0 -conform plays (with the same probability distribution on them) in P \ A
that start in W0 .
Similarly, Player 1 wins with his strategy from every vertex in A, by a composition of his attractor strategy on A \ P1 and his winning strategy f1 on P1
by Lemma 6.
Let g1 be a winning strategy for Player 1 in P 0 . Then every g1 -conform play
in P that starts in a vertex in W10 either eventually reaches A, and is then almost
surely followed by a tail (remainder of the play) in P that starts in A, which is
winning for Player 1 with a likelihood strictly greater than 0 by Lemma 6; or it
stays for ever in the sub-game P 0 . But these plays are also won by Player 1 with
a non-zero likelihood.
t
u
0
We now distinguish two cases: firstly, if W1−σ
is non-empty, we can reduce
solving P to constructing the weak or strong, respectively, (1−σ)-attractor U1−σ
0
of W1−σ
, and solving the co-game P 00 = P \ U1−σ by Lemma 7 or 8, respectively.
The co-game P 00 is simpler than P: compared to P, it contains less vertices
(though not necessarily less priorities). By induction over the size of the game,
P 00 can therefore be solved by a recursive call of the algorithm.
0
The figure below aligns this to our algorithm. We have seen that W1−σ
from
the previous picture, shown again in solid red, is a (1 − σ)-paradise, and so is its
(weak or strong) (1−σ)-attractor U1−σ (the complete red area, full and hatched).
It is constructed in lines 5.b.iv and 7.d, respectively, of prob−McNaughton.
arena
′
W1−σ
U1−σ
Fig. 4. Attractor of the player 1 − σ where c is the minimal priority and 1 − σ = c
mod 2
0
is empty, we can compose the winning strategy for Player σ
Secondly, if W1−σ
0
on P with his attractor strategy for pri−1 (c) to a winning strategy on P.
Lemma 9. Let P be a parity game with minimal priority c, σ = c mod 2 be the
player who wins if c occurs infinitely often, A be the strong σ-attractor of pri−1 (c),
and f be an attractor strategy for Player σ on her vertices on A \ pri−1 (c). If
Player σ has a winning strategy f 0 for every vertex in P 0 = P \ A, then f and
f 0 can be composed to a winning strategy for Player σ for every vertex in P.
Proof. Let g be a strategy for Player σ that agrees with f and f 0 on their respective domain. We distinguish two types of g-conform plays: those that eventually
stay in P 0 , and those that visit A infinitely often. The latter plays almost surely
contain infinitely many vertices with priority c and are therefore almost surely
winning for Player σ. Games that eventually stay in P 0 consist of a finite prefix,
followed by an f 0 -conform play in P 0 . The lowest priority occurring infinitely
often is therefore almost surely even for σ = 0 and odd with a likelihood strictly
greater than 0 for σ = 1, respectively.
t
u
Theorem 1. For each parity game P = (V0 , V1 , Vr , E, pri), the game vertices
are partitioned into a winning region W0 of Player 0 and a winning region W1
of Player 1. Moreover, Player 0 and Player 1 have memoryless strategies that
are v-winning for every vertex v in their respective winning region.
In the following proof, we do not count the sink that is added by Lemma 7.
Proof. Games with a single vertex are trivially won by the player winning on
the priority of this vertex (induction basis).
For the induction step, assume that the memoryless determinacy holds for
games with up to n vertices. For a parity game with n + 1 vertices, we can then
select the lowest priority cmin , set σ to cmin mod 2 to identify the Player σ who
wins if cmin occurs infinitely often (note that cmin is the dominating priority in
this case), and set A = satrσ (pri−1 (cmin ), P).
Then P 0 = P \ A is a—possibly empty—parity game with strictly less states
and priorities. (Note that, by the attractor construction, every vertex in P 0 has
a successor, and the co-set of A is a σ-trap.)
By induction hypothesis, P 0 vertices are partitioned into winning regions of
the two players, and both players have memoryless winning strategies on their
winning regions.
We can now distinguish three cases:
1. The winning region of Player 1 − σ on P 0 is empty. In this case, Player σ
wins memoryless by Lemma 9.
2. σ = 0 and the winning region of Player 1 is non-empty.
Then W100 = satr1 (W10 , P) is a 1-paradise for P by Lemmas 5 and 6. We can
therefore solve the remainder of the game, P \ W100 , individually and use the
respective winning regions and (by induction) memoryless winning strategies
of the players by Lemma 8.
3. σ = 1 and the winning region of Player 0 is non-empty.
Then W000 = watr0 (W10 , P) is a 0-paradise for P by Lemmas 5 and 6. We can
therefore solve the remainder of the game, P \ W000 , individually and use the
respective winning regions and (by induction) memoryless winning strategies
of the players by Lemma 7.
In case (1) we are done, in (2), (3) we reduced the problem to solving games
with less states. By induction, memoryless determinacy extends to the complete
game.
t
u
The worst case running time of this extension to McNaughton’s algorithm
[13,27,36] (cf. Procedure prob−McNaughton of Figure 2) occurs if U1−σ is always
small and exactly one vertex with minimal priority c belongs to U1−σ . For parity
games with c priorities, n vertices, and m edges, the adjusted algorithm based
c−1 steps when the highest priority
on McNaughton’s requires O m · nc + 1
of the game is even, like McNaughton’s algorithm itself [13, 36]: the cost of the
attractor constructions is always dominated by the cost of the recursive calls,
and the complexity analysis is the same.
When the highest priority is odd, the higher complexity for the weak attracc+1 c complexity, and to an O m · nc + 1
tor construction leads to a O nc + 1
complexity of our implementation, due to the cost of constructing weak attractors of O(n2 ) and O(m · n) in our implementation, respectively. (The only point
where this complexity is not dominated by other steps is when only the highest
and the second highest priority are left. When the second highest priority is
even, only strong attractors are used.)
The presented algorithm has a theoretical drawback: its running time is exponential in the number of priorities, which in turn can be as high as the number
of states. For parity games without random vertices, the development of a deterministic subexponential algorithm [22] was considered a breakthrough, which
also led to better bounds for games with a small number of priorities [32].
√
Theorem 2. Parity games with n vertices can be solved in time nO(
n)
.
The proof is completely analogously to the one from [22]. It uses the fact
that the proofs do not rely on the way intermediate paradises are constructed.
It is therefore
√ possible to try out all small sets of vertices (i.e. all sets of vertices
up to size n) and check if they are paradises for the player who loses
√ on the
minimal priority. This can be done using brute force and takes nO( n) time.
Afterwards, one builds their union, computes the attractor for the set obtained
this way, and performs a recursive call to decide the remaining game.
The progress obtained in one step, consisting of trying√out all small sets of
vertices, followed by a recursive call, is therefore at least n – unless the call
returns an
case we are finished. This provides a call tree of
√ empty set, in which
√
O( n)
size nO( n)√, with cost
n
on
each node of the call tree, such that the total
√
√
cost is nO( n) · nO( n) = nO( n) .
5
5.1
Implementation and experimental results
Overview of the algorithm
We have written a prototypical implementation for the approach of this paper.
In Figure 5, we sketch how we analyse properties of formal models. Our tool
Probabilistic
multi-player game
Coalition
building
State space
exploration
Probabilistic
two-player game
Model
Formula
Product
building
Product
parity game
Automaton
construction
Parity
automaton
Decision
algorithm
Büchi
automaton
Determinisation
Yes/No
Fig. 5. Overview of the algorithm
reads a model specification in the input language of the probabilistic model
checker PRISM-games [11]. This input language is an extension of probabilistic
guarded-commands language of the probabilistic model checker PRISM [25]. We
extend the recent probabilistic model checker IscasMC [17,18] that also supports
this language. Each state of the model semantics is assigned to a specific player.
When being in a state, a player can choose from several guarded commands.
In each state, the player controlling this state can choose to execute one of
the commands the guard of which is valid in the state. Afterwards, one of the
possible effects of the command is chosen according to the probability of this
effect. The next state is decided according to this effect. The models specified
in this way may contain more than two players, and they do not contain any
winning condition.
The property specification we use in our tool is also inspired by PRISMgames [11]. Its general form is hhcoalitioniiP≥1 [φ]. Here, coalition is a subset of
the players from the model specification, and P≥1 [φ] is a qualitative probabilistic
LTL formula. This formula requires that the coalition of the players of coalition
(by subsuming all of them into a single player) can enforce that φ holds with
probability 1 under all possible behaviours of the players not in coalition. Variants like hhcoalitioniiP>0 [φ] requiring only positive probability can be computed
by minor adaptation of our algorithm.
To obtain the actual parity game to apply our algorithm on we proceed as
follows. We first construct the explicit-state semantics of the model originally
given in the high-level input language of PRISM-games. This semantics is thus
a probabilistic multi-player game without a winning condition. Then, in this
semantics we subsume the players of coalition to the single even player and
the remaining players to the odd player, so as to obtain a probabilistic twoplayer game, still without a winning condition. We then transform φ into a
generalised Büchi automaton using the tool SPOT [12]. Afterwards, we transform
the Büchi into a deterministic parity automaton, using the algorithm from [33].
Simultaneously, we build a product of the model with this parity automaton,
so as to fix the winning condition according to the property we want to decide.
This on-the-fly approach allows us to only construct the states of the parity
automaton which will actually be needed to construct the product. A state of
this product is thus a tuple of a state from the two-player model and a state of
the parity automaton.
The algorithm from [33] produces automata whose priorities are on the transitions (rather than on states). The product we produce is however a state-labelled
parity game as in Definition 1: when building the product of the model with the
automaton, the current state fixes the successor transition of the deterministic
parity automaton. Therefore, we can label the product state with the priority
of this successor transition. The first component of the successor states consists
of a successor state of the model state. The second one consists of the single
successor state of the parity automaton.
Finally, we can apply the algorithm discussed on the resulting MPG and
decide whether all initial states of the product are winning, which means deciding
whether the specification holds for all initial states of the model. If it does, we
also obtain a winning strategy.
As stated before, the mutually optimal strategies are memoryless. However,
this only holds for the game we are computing strategies for. If we map back
these strategies to the original PRISM-games model, indeed memory might be
required. This is because we have to remember the current state of the parity
automaton with which the model had been composed to obtain the product on
which we finally apply our algorithm.
5.2
Robots
Reconsider our robot example introduced in Section 2. We have applied our tool
to construct the state-space of several instantiations of this model which we have
modelled in the input language of PRISM-games. The machine we used is a 3.6
GHz Intel Core i7-4790 with 16 GB 1600 MHz DDR3 RAM of which 12 GB
assigned to the tool; the timeout has been set to 30 minutes. In Table 1 we
provide for each size “n” of the battlefield we consider, the number of vertices of
the corresponding multiplayer game (“vertices”) and the time in seconds required
to construct it (“tconstr ”).
We remark that the constructed game en- Table 1. Arena construction
codes the behaviour of the robots on the batn
vertices
tconstr
tlefield without winning conditions, i.e. without
considering the formula to be checked; as seen,
12
370 656
1
the state-space contains millions of vertices for
16 1 175 040
3
larger battlefield sizes n (it grows with O(n4 )).
20 2 872 800
6
Note that we cannot compare to PRISM-games
24 5 961 600
13
because it does not support general PLTL for28 11 049 696
25
mulas, and we are not aware of other tools to
32 18 855 936
47
compare with.
36 30 209 760
69
We have applied our tool on a number of
40 46 051 200
112
properties that require the robot R0 to visit the
different zones in a certain order. In Table 2 we report the performance measurements for these properties. In the column “property” we state the PLTL
formula we consider, column “n” states the width of the battlefield instance,
Table 2. Robots analysis: different reachability properties
property
Reachability
hhR0 iiP≥1
[ F zone1
∧ F zone2 ]
Repeated
Reachability
hhR0 iiP≥1
[ GF zone1
∧ GF zone2
∧ GF zone3
∧ GF zone4 ]
Repeated
Ordered
Reachability
hhR0 iiP≥1
[GF(zone1 ∧
F zone2 )]
Reach-avoid
hhR0 iiP≥1
[ ¬zone1 U zone2
∧ ¬zone4 U zone2
∧ ¬zone4 U zone1
∧ Fzone4 ]
game construction
vertices
colours
tprod
gadget construction
vertices
tgadg
solving time
tgMcN
tgJur tpMcN
n
sat
12
16
20
24
28
32
36
40
true
true
true
true
true
true
-
1 324 704
4 418 592
11 050 656
23 211 552
43 334 304
74 294 304
-
2
2
2
2
2
2
-
2
9
22
51
102
197
–MO–
–MO–
5 351 584
17 996 832
45 166 752
95 045 152
177 634 464
-
3
13
28
58
115
–MO–
-
0
3
8
18
35
-
4
24
84
219
–MO–
-
0
1
3
7
14
24
-
12
16
20
24
28
32
36
40
true
true
true
true
true
true
-
1 324 704
4 418 592
11 050 656
23 211 552
43 334 304
74 294 304
-
2
2
2
2
2
2
-
2
9
20
44
88
180
–MO–
–MO–
6 010 528
20 085 792
50 273 952
105 643 552
197 278 368
-
3
15
29
59
121
–MO–
-
1
3
9
21
38
-
5
25
85
227
–MO–
-
0
1
3
8
15
27
-
12
16
20
24
28
32
36
40
true
true
true
true
true
true
true
true
693 048
2 264 184
5 611 320
11 729 784
21 836 088
37 367 928
59 984 184
91 564 920
5
5
5
5
5
5
5
5
0
3
6
14
27
54
65
121
4 009 976
13 206 072
32 844 792
68 787 512
128 198 136
219 543 096
-
2
9
17
62
69
135
–MO–
–MO–
0
2
6
13
24
–MO–
-
3
16
50
129
282
–MO–
-
0
0
1
4
7
13
21
36
12
16
20
24
28
32
36
40
true
true
true
true
true
true
-
1 616 400
5 452 848
13 703 184
28 855 728
53 951 760
92 585 520
-
4
4
4
4
4
4
-
2
14
35
79
171
323
–MO–
–MO–
7 656 720
25 820 208
64 877 328
136 606 128
255 402 000
-
4
15
33
116
135
–MO–
-
1
6
19
40
–MO–
-
–TO–
–TO–
–TO–
–TO–
–MO–
-
0
1
5
11
21
38
-
and “sat” shows whether the formula is satisfied. For the “game construction”
part, we present the number of “vertices” of the resulting MPG, the number of
“colours”, and the time “tprod ” required to generate the MPG. The construction
of [8] to turn a stochastic parity game into a non-stochastic parity game replaces
each stochastic node by a so-called “gadget”, which consists of a combination
of player odd and even nodes. Applying this construction thus leads to a game
which can be solved using existing methods, although at the cost of increasing
the number of vertices and the time to perform the transformation. The “gadget construction” part of the table shows the total number of “vertices” after
applying this construction and the time “tgadg ” spent to do so.
Finally, the “solving time” part shows the time spent by the specific solvers:
tgMcN and tgJur correspond to the gadget construction solved by using the classical non-stochastic McNaughton and Jurdziński approach, respectively, while
tpMcN refers to our prob−McNaughton algorithm proposed in Figure 2. Note
that these times represent only the actual time spent by the solver on the final
MPG. Entries marked by “–TO–” and “–MO–” mean that the corresponding
phase has failed by time-out or has gone out of memory; entries marked by “-”
mean that the phase has not been executed due to a failure in a previous phase.
The results show that our approach can be used to solve games with several
million vertices, even though it is currently only implemented as an explicitstate solver. In particular, the part of the process that consumes the largest
share of time is not the solution algorithm itself, but the preprocessing steps:
most of the time was spent on constructing the product of the battlefield and the
parity automaton. The largest amount of time spent in the solver was 38 seconds
for a parity game with more than 90 million vertices. Despite the exponential
complexity of the algorithm, our prototype performs quite well on the large
state-spaces of this model. One reason is that the maximal number of different
priorities seen was just 5, and the implementation was often able to use the
lines 3 and 4 of the algorithm prob−McNaughton in Figure 2 to terminate the
construction quickly. Indeed, we did not see more than 5 recursive calls of the
algorithm. It is worthwhile to remark that, even if all properties are satisfied,
not all vertices are winning for the robot R0 : for instance, for the reach-avoid
property, around 1/3 of the vertices are winning for the robot R1 ; R0 is anyway
able to avoid such vertices and win with probability 1.
In comparison, the solution methods based on the gadget construction have
an additional phase that takes quite some time and memory to be completed,
so this affects their performances as they have to work on much larger games.
While the classical non-stochastic McNaughton algorithm has a reasonable performance on these games, it still consumes much more resources than our approach prob−McNaughton. Jurdziński’s approach turns out to be really slow in
practice, confirming the previous results of [14]. As the results in the table show,
our approach really outperforms the methods based on gadget construction: for
instance for the reach-avoid property for n = 24, our approach takes only 11
seconds instead of 40 (plus 79 for the gadget construction) taken by the classical
McNaughton and the time-out of 30 minutes by the Jurdziński algorithm.
5.3
Two investors
As a further small case study, which provides similar results regarding the performance of the Jurdziński theoretical better algorithm, we consider an example
originally from [26] in the version of [10]. In this version of the model, there
are two investors, who are able to make investments in futures. Buying a future
means to reserve an amount of company shares at a certain point of time and
will be delivered at a later point of time to the market price the share has then.
In this version of the model, there are three players: investor 1, investor 2, and
the market. We considered a number of properties for which we provide results
in Table 3; the meaning of the formulas is as follows:
1. Investor 1 is able to ensure (against the market and the other investor) that
the share has eventually a value of at least 5 without ever stopping to invest.
2. She can ensure that the share repeatedly has a value of 5.
3. She can guarantee a permanent value of at least 5.
4. She can ensure that the probability of this event is non-zero.
5. If all players collaborate, this event is certain.
The parity game constructed to decide these properties contains less than 1.5
million vertices. Therefore, the time to decide the properties is also almost negligible: all experiments have taken a total of one or two seconds to complete, except
for the Jurdziński approach that went time-out on all except one property.
Table 3. Two investors analysis.
property
sat
game construction
vertices colours tprod
gadget construction
vertices
tgadg
solving time
tgMcN
tgJur tpMcN
1.
hhinvestor1 iiP≥1 [
G¬done1 ∧ Fv ≥ 5]
true
410 531
4
0
1 001 645
1
0
–TO–
0
2.
hhinvestor1 iiP≥1 [
G¬done1 ∧ GFv ≥ 5]
false
410 531
4
0
1 265 755
1
0
–TO–
0
3.
hhinvestor1 iiP≥1 [
G¬done1 ∧ FGv ≥ 5]
false
413 171
5
0
1 486 783
1
0
–TO–
0
4.
hhinvestor1 iiP>0 [
G¬done1 ∧ FGv ≥ 5]
true
413 171
5
0
1 486 783
1
0
0
0
5.
hhinvestor1 , investor2 ,
marketiiP≥1 [
G¬done1 ∧ FGv ≥ 5]
false
413 171
5
0
1 486 783
1
0
–TO–
0
6
Conclusions and Future Work
We have introduced a simple and effective algorithm for solving Markov games
with a parity winning condition and implemented the algorithm as an explicitstate prototype as an extension of the model checker IscasMC [17, 18]. The algorithm has proven to be capable of handling rather large examples, obtaining
strategies that almost surely obtain their goal or demonstrating that such strategies do not exist. This is a very encouraging result for the automated construction of simple probabilistic reactive protocols, as they are already used in leader
election problems.
The construction of such protocols is already difficult for deterministic systems, and new protocols (as well as the old ones) had been discovered when
they had been recently synthesised [23]. Some complicated programming problems like mutual exclusion, leader election, and variations thereof, have complete
specifications. Yet, they are very difficult to implement, e.g. due to problems arising through context switches. While such problems have proven to be difficult
to implement for human developers due to parallelism and nondeterminism in
traditional systems, allowing for randomness does—while potentially simplifying the algorithm—add another layer of difficulty for the human developer. We
believe that this establishes the need for synthesis techniques, and in this light it
is a very good news that the solution we have developed in this paper shows po-
tential. In future work, we will extend this efficient technique to the quantitative
analysis of systems.
Acknowledgement
This work is supported by the National Natural Science Foundation of China
(Grants No. 61472473, 61532019, 61550110249, 61550110506), by the National
973 Program (No. 2014CB340701), by the CDZ project CAP (GZ 1023), by the
Chinese Academy of Sciences Fellowship for International Young Scientists, by
by the CAS/SAFEA International Partnership Program for Creative Research
Teams, and by the Engineering and Physical Sciences Research Council (EPSRC)
through grant EP/M027287/1 (Energy Efficient Control).
References
1. A. Bianco and L. de Alfaro. Model checking of probabilistic and nondeterministic
systems. In FSTTCS, volume 1026 of LNCS, pages 499–513, 1995.
2. K. Chatterjee. Stochastic ω-Regular Games. PhD thesis, University of California
at Berkeley, 2007.
3. K. Chatterjee. Qualitative concurrent parity games: Bounded rationality. In CONCUR, volume 8704 of LNCS, pages 544–559, 2014.
4. K. Chatterjee, L. de Alfaro, and T. A. Henzinger. Strategy improvement for concurrent reachability games. In QEST, pages 291–300. IEEE Computer Society,
2006.
5. K. Chatterjee, L. de Alfaro, and T. A. Henzinger. Qualitative concurrent parity
games. ACM Trans. Comput. Log., 12(4):28, 2011.
6. K. Chatterjee, L. Doyen, H. Gimbert, and Y. Oualhadj. Perfect-information
stochastic mean-payoff parity games. In FOSSACS, volume 8412 of LNCS, pages
210–225, 2014.
7. K. Chatterjee and M. Henzinger. An O(n 2 ) time algorithm for alternating Büchi
games. In SODA, pages 1386–1399. SIAM, 2012.
8. K. Chatterjee, M. Jurdziński, and T. A. Henzinger. Simple stochastic parity games.
In CSL, volume 2803 of LNCS, pages 100–113, 2003.
9. K. Chatterjee, M. Jurdziński, and T. A. Henzinger. Quantitative stochastic parity
games. In SODA ’04, pages 121–130, 2004.
10. T. Chen, V. Forejt, M. Z. Kwiatkowska, D. Parker, and A. Simaitis. Automatic
verification of competitive stochastic systems. Formal Methods in System Design,
43(1):61–92, 2013.
11. T. Chen, V. Forejt, M. Z. Kwiatkowska, D. Parker, and A. Simaitis. PRISMgames: A model checker for stochastic multi-player games. In TACAS, volume
7795 of LNCS, pages 185–191, 2013.
12. A. Duret-Lutz. LTL translation improvements in Spot. In VECoS, pages 72–83,
2011.
13. E. A. Emerson and C. Lei. Efficient model checking in fragments of the propositional µ-calculus. In Proc. LICS, pages 267–278. IEEE Computer Society Press,
1986.
14. O. Friedmann and M. Lange. Solving parity games in practice. In ATVA, volume
5799 of LNCS, pages 182–196, 2009.
15. T. Gawlitza and H. Seidl. Games through nested fixpoints. In CAV, volume 5643
of LNCS, pages 291–305, 2009.
16. H. Gimbert and W. Zielonka. Perfect information stochastic priority games. In
ICALP, volume 4596 of LNCS, pages 850–861, 2007.
17. E. M. Hahn, G. Li, S. Schewe, A. Turrini, and L. Zhang. Lazy probabilistic model
checking without determinisation. In CONCUR, volume 42 of LIPIcs, pages 354–
367, 2015.
18. E. M. Hahn, Y. Li, S. Schewe, A. Turrini, and L. Zhang. IscasMC: A web-based
probabilistic model checker. In FM, volume 8442 of LNCS, pages 312–317, 2014.
19. H. Hansson and B. Jonsson. A logic for reasoning about time and reliability. FAC,
6(5):512–535, 1994.
20. J. P. Hespanha, H. J. Kim, and S. Sastry. Multiple-agent probabilistic pursuitevasion games. In CDC, pages 2432–2437, 1999.
21. M. Jurdziński. Small progress measures for solving parity games. In STACS,
volume 1770 of LNCS, pages 290–301. Springer, 2000.
22. M. Jurdziński, M. Paterson, and U. Zwick. A deterministic subexponential algorithm for solving parity games. SIAM J. Comput., 38(4):1519–1532, 2008.
23. G. Katz and D. Peled. Genetic programming and model checking: Synthesizing
new mutual exclusion algorithms. In ATVA, volume 5311 of LNCS, pages 33–47.
Springer, 2008.
24. S. Kremer and J. Raskin. A game-based verification of non-repudiation and fair
exchange protocols. Journal of Computer Security, 11(3):399–430, 2003.
25. M. Z. Kwiatkowska, G. Norman, and D. Parker. PRISM 4.0: Verification of probabilistic real-time systems. In CAV, volume 6806 of LNCS, pages 585–591. Springer,
2011.
26. A. McIver and C. Morgan. Results on the quantitative mu-calculus qMu. ACM
Transactions on Computational Logic, 8(1), 2007.
27. R. McNaughton. Infinite games played on finite graphs. Ann. Pure Appl. Logic,
65(2):149–184, 1993.
28. T. Park and K. G. Shin. Lisp: a lightweight security protocol for wireless sensor
networks. ACM Transactions on Embedded Computing Systems, 3:634–660, 2004.
29. A. Perrig, R. Szewczyk, V. Wen, D. Culler, and J. D. Tygar. Spins: Security
protocols for sensor networks. In Wireless Networks, pages 189–199, 2001.
30. A. Pnueli. The temporal logic of programs. In Proc. FOCS, pages 46–57. IEEE
Computer Society Press, 1977.
31. M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, 1994.
32. S. Schewe. Solving parity games in big steps. In FSTTCS, volume 4805 of LNCS,
pages 449–460. Springer, 2007.
33. S. Schewe and T. Varghese. Tight bounds for the determinisation and complementation of generalised Büchi automata. In ATVA, volume 7561 of LNCS, pages
42–56. Springer, 2012.
34. W. van der Hoek and M. Wooldridge. Model checking cooperation, knowledge,
and time - a case study. Research in Economics, 57(3):235–265, 2003.
35. L. Wu, K. Su, and Q. Chen. Model checking temporal logics of knowledge and its
application in security verification. In CIS, volume 3801 of LNAI, pages 349–354,
2005.
36. W. Zielonka. Infinite games on finitely coloured graphs with applications to automata on infinite trees. LNCS, 200(1-2):135–183, 1998.