2
SIEGMUND DUALITY FOR MARKOV CHAINS ON
PARTIALLY ORDERED STATE SPACES
3
PAWEŁ LOREK
4
Mathematical Institute
University of Wrocław
1
5
Abstract. For Markov chains on finite partially ordered state space we
show that the Siegmund dual exists if and only if the chain is Möbius
monotone, in which case we give formula for the transitions of this dual.
This is an extension of Siegmund’s result for totally ordered state spaces
in which case the existence of the dual is equivalent to usual stochastic monotonicity (which is - for total ordering - equivalent to Möbius
monotonicity). Exploiting the relation between stationary distribution
of ergodic chain and absorption probabilities of its Siegmund dual we
show three applications: calculating absorption probabilities of chain
with two absorbing states knowing stationary distribution of other chain;
calculating stationary distribution of ergodic chain knowing absorption
probabilities of other chain; providing stable simulation scheme for stationary distribution of a chain provided we can simulate its Siegmund
dual. These are accompanied by concrete examples: some nonsymmetric
game; some extension of birth and death chain and nonstandard tandem
network of two servers respectively.
We also show that one can construct a Strong Stationary Dual chain
by performing appropriate Doob transform of the Siegmund dual of
time-reversed chain.
Keywords: Markov chains; Siegmund duality; Strong Stationary Duality; Möbius monotonicity; partial ordering; Doob h-transform.
6
1. Introduction
7
Siegmund [19] introduced a notion of duality (today called Siegmund duality) for Markov chains with discrete time and general state space with total
ordering. This duality is intended to relate the stationary distribution of a
process with the absorption probability of its dual. In case of total ordering,
Siegmund showed that dual chain exists if and only if the original chain is
stochastically monotone (w.r.t. total ordering). The aim of this note is a
generalization of this result to state spaces which are only partially ordered.
Thomas Liggett in his famous book [12] writes (about Siegmund’s result for
total ordering):
8
9
10
11
12
13
14
15
E-mail address: [email protected].
Work supported by NCN Research Grant DEC-2013/10/E/ST1/00359.
1
16
“This result depends heavily on the fact that the state space of
the processes is totally ordered. In the particle system context,
the state spaces are only partially ordered, and unfortunately the
analogous result fails. [...] having a (reasonable) dual is a much
more special property then being monotone, when the state space
is not totally ordered.“
17
As the main result we show that the existence of the Siegmund dual is
equivalent to Möbius monotonicity of the chain (introduced in Lorek and
Szekli [15]) w.r.t. fixed partial ordering. In this case we give transitions of
the dual.
It turns out that the usual stochastic monotonicity of a chain does not
imply existence of Siegmund dual for general partial orderings (it does for
linear ordering, since then stochastic and Möbius monotonicities are equivalent). In general, the required Möbius monotonicity is not stronger, but
different than stochastic monotonicity.
There is a one-to-one correspondence (the relation is given in (2.3)) between the stationary distribution of ergodic chain and absorption probabilities of its Siegmund dual. We will show three potential applications of this
relation. First application: given chain with two absorbing states (gambler’s
ruin-like) we can find ergodic chain for which the original one is a Siegmund
dual. Then finding stationary distribution of this chain answers the question
about the absorption probabilities. As an example for this application we
present some nonsymmetric game, which turns out to be a Siegmund dual to
a nonsymmetric random walk on cube considered in Lorek and Szekli [15].
Second application: given ergodic chain, the task is to find its stationary distribution. We can calculate its Siegmund dual, then finding the absorption
probabilities of the dual answers the question about the stationary distribution of the ergodic chain. As an example we present ergodic chain for
which Siegmund dual corresponds to ”Gambler’s Ruin with Catastrophes
and Windfalls“ considered in Hunter [11] for which absorption probabilities
are calculated therein explicitly. Third application: This is similar to the
second one, we are given an ergodic chain, the task is again to find its stationary distribution. Calculating Siegmund dual can be relatively easy, but
finding such probabilities is often a challenging task. However if we are able
to simulate the dual, then we can have stable simulation scheme for the stationary distribution of the ergodic chain via simply estimating absorption
probabilities. In contrast to most of Monte Carlo Markov Chains (MCMC)
we do not require a priori any knowledge about, e.g., rate of convergence,
mean absorption time etc. Besides, MCMC are designed to provide a sample from stationary distribution, whereas stable simulation scheme lets us
estimate the stationary distribution at the specific state. As the example we
present (including calculations of Siegmund dual) some tandem network of
two servers. This is some modification of the standard tandem (for which
2
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
stationary distribution is known): when one of the servers is empty, the rates
of the other are changed. It turns out that then the stationary distribution
is not known (it is surely not of product form as for standard tandem). Foley and McDonald [7] considered similar modified tandem (if the server is
idle it helps the other) showing large deviations and rough asymptotics for
the stationary distribution. In particular they showed that such changes on
borders imply dramatic changes in the stationary distribution.
Siegmund dualities appear in various contexts. Original paper Siegmund
[19] deals with stochastically monotone chains w.r.t a total ordering. Many
authors focused on birth and death chains, e.g., Cox and Rösler [2], Diaconis
and Fill [4] (where connections to Strong Stationary Duality is also given),
Dette et al. [3] (where the Siegmund duality is related to Wall duality)
or Huillet [9] (where a specific birth and death chain, Moran model, was
considered). The observation, that such duality holds for some random walks
on {0, 1, . . .} goes back to Lindley [13]. The duality was also studied in a
financial context, where the probability that the steady-state queue length
exceeds level k equals the probability that a dual risk process starting at level
k is ruined in a finite time. There were some approaches to extend this to
nonlinear state spaces. Błaszczyszyn and Sigman [1] considered Rd -valued
Markov processes (their Siegmund dual was set-valued). Recently, Huillet
and Martínez [10] considered dualities related to Möbius matrix for other
nonlinear state spaces, namely for Markov chains on partitions and sets. In
Lorek and Szekli [15] we consider general partial orderings and show that
Möbius monotonicity of a time-reversed chain implies existence of a Strong
Stationary Dual (SSD) chain on the same state space (more details in Section
5).
Recently, in Fill and Lyzinski [6] and in Miclo [17] dualities for onedimensional diffusions were studied. Further research include studying existence of Siegmund dual in d-dimensional diffusions, e.g., the ones studied
in Harrison and Williams [8], as well as developing the theory for Möbius
monotone processes for continuous-time and general state space chains.
The rest of the note is organized as follows. In Section 2 we describe
Siegmund duality and relations between the stationary distribution of a chain
and the absorption probability of its dual. In Section 3 we shortly recall
Möbius monotonicity. In Section 4 we present our main result (Theorem
4.1) on equivalence of Möbius monotonicity and Siegmund duality for general
partial orderings. Section 5 gives a connection to Strong Stationary Duality.
Finally, in Section 6 we present three applications: in Section 6.1 we give
winning/losing probabilities for some nonsymmetric game, in Section 6.2 we
give stationary distribution of some extension of birth and death chain and
in Section 6.3 we present stable simulation scheme for some nonstandard
tandem queue system.
3
96
2. Siegmund duality
97
Let X ∼ (ν, PX ) be a discrete-time Markov chain with initial distribution ν, transition matrix PX and finite state space E = {e1 , . . . , eM } partially ordered by . Throughout the paper we assume it is ergodic with
the stationary distribution π. We assume, that there exist unique minimal state, say e1 ,Pand unique maximal state, say eM . P
For A ⊆ E define PX (e, A) := e0 ∈A PX (e, e0 ) and similarly π(A) := e∈A π(e). Define also {e}↑ := {e0 ∈ E : e e0 }, {e}↓ := {e0 ∈ E : e0 e} and
δ(e, e0 ) = 1(e, e0 ). Recall that Markov chain X is stochastically monotone,
if ∀(e1 e2 )∀(D − down set)PX (e1 , D) ≥ P(e2 , D), where D is a down set
if (e e0 , e0 ∈ D) =⇒ e ∈ D.
We say that Markov chain Z with transition matrix PZ is the Siegmund
dual of X if
98
99
100
101
102
103
104
105
106
107
108
(2.1)
109
110
111
112
113
Siegmund [19] showed that for total ordering (then, let us denote elements of
E as numbers {1, 2, . . . , M }) such dual exists if and only if X is stochastically
monotone. The main thing then is to show that (2.1) holds for one-step
transitions. Since the main part of the proof is one line long, we include it
here. We want to have PX (i, {j}↓ ) = PZ (j, {i}↑ ). We can calculate
(2.2)
114
115
116
117
118
119
120
121
∀(ei , ej ∈ E) ∀(n ≥ 0) PnX (ei , {ej }↓ ) = PnZ (ej , {ei }↑ ).
PZ (j, i) = PZ (j, {i}↑ ) − PZ (j, {i + 1}↑ )
= PX (i, {j}↓ ) − PX (i + 1, {j}↓ ).
The latter is nonnegative if and only if X is stochastically monotone. Note,
that
P (2.2) does not have to define transition matrix, since we may have
i PZ (j, i) < 1. Siegmund [19] adds then
P one extra absorbing state, say
M + 1, and defines PZ (j, M + 1) = 1 − i PZ (j, i). In a similar way, for
general partial ordering, if we are able to find a subprobability kernel fulfilling
(2.1) for all ei , ej ∈ E, we may be forced to add one extra absorbing state,
say eM +1 . Note that (2.1) implies that eM is an absorbing state, thus Z has
two absorbing states. Taking limits as n → ∞ on both sides of (2.1) we have
122
(2.3)
π({ej }↓ ) = limn→∞ PnZ (ej , {ei }↑ ) = P (Z is absorbed in eM |Z0 = ej ).
124
This way the stationary distribution of X is related to the absorption of its
Siegmund dual Z.
125
3. Möbius monotonicity
126
Let C(ei , ej ) = 1(ei ej ). We can always rearrange states so that
ei ej implies i ≤ j. Then the matrix C is 0-1 valued, upper-triangular,
thus invertible. The inverse C−1 is often denoted as µ and is called Möbius
function. The famous Möbius inversion formula (see, e.g., Rota [18]) states:
4
123
127
128
129
130
(3.1)
Let F̄ (e) =
X
f (e0 ),
then f (e) =
e0 e
131
132
133
134
135
136
137
138
X
µ(e, e0 )F̄ (e0 ).
e0 e
We say that function F̄ : E → RM is Möbius monotone if the resulting f (e)
in (3.1) is nonnegative for all e ∈ E.
For all states e2 ∈ E denote: F̄e2 (e0 ) = PX (e0 , {e2 }↓ ). We say that chain
X is Möbius monotone, if functions F̄e2 are Möbius monotone for all e2 ∈ E.
Equivalently, the definition can be stated in matrix form:
Definition 3.1. Markov chain X with transition matrix PX is Möbius
monotone w.r.t partial order if C−1 PX C ≥ 0 (i.e., each entry nonnegative). In terms of transition probabilities, it means that
X
∀(ei , ej ∈ E)
µ(ei , e)PX (e, {ej }↓ ) ≥ 0.
eei
Remark 3.2. In a similar way to (3.1), we have the following version of
Möbius inversion formula:
X
X
Let F (e) =
f (e0 ), then f (e) =
µ(e0 , e)F (e0 ).
e0 e
139
140
e0 e
This way,X
once π({ej }↓ ) is calculated for all ej ∈ E in (2.3), we can calculate
π(ej ) =
µ(e, ej )π({e}↓ ).
eej
141
For more details on Möbius monotonicity see Lorek and Szekli [15].
142
4. Main result
143
In this section we will show the existence of the Siegmund dual on state
space E∗ = E∪{eM +1 }, where we add one extra absorbing state eM +1 (coffin
state). At each step, the process can be possibly killed, i.e., absorbed in this
state.
144
145
146
147
148
149
150
151
Theorem 4.1. Let X ∼ (ν, PX ) be a Markov chain on E = {e1 , . . . , eM }
with partial ordering . Assume e1 is a unique minimal state and eM is
a unique maximal state. Then, there exists the Siegmund dual Z of X on
E∗ = E ∪ {eM +1 } if and only if X is Möbius monotone. The transitions of
the dual are as follows:
PZ (ej , ei )
=
X
µ(ei , e0 )PX (e0 , {ej }↓ ), ei , ej ∈ E,
e0 ei
(4.1)
PZ (eM +1 , ej ) = δ(eM +1 , ej ),
X
PZ (ek , eM +1 ) = 1 −
PZ (ek , e),
e∈E
5
ej ∈ E∗ ,
ek ∈ E.
152
153
154
155
156
Proof. Note that the state space of dual is enriched with one extra absorbing
state called eM +1 . We need equation (2.1) to hold for all states from E. One
can think that eM +1 is incomparable to all other states in E∗ , what we
assume, since it simplifies some notation.
First, we show that (2.1) holds for n = 1, what can be rewritten as
X
PZ (ej , e).
PX (ei , {ej }↓ ) =
eei
157
Using Möbius inversion formula (3.1) we have
X
(4.2)
PZ (ej , ei ) =
µ(ei , e)PX (e, {ej }↓ ),
eei
what is nonnegative if and only if X is Möbius monotone. For
P partial order
with unique minimal element e1 the Möbius function fulfills: ei ∈E µ(ei , e) =
1(e = e1 ) (to see this consider first row of C−1 after applying first elementary row operation of Gauss-Jordan elimination). This implies that rhs of
(4.2) is not greater than 1, since
X
X X
PZ (ej , ei ) =
µ(ei , e)PX (e, {ej }↓ )
ei ∈E
ei ∈E eei
158
(∗)
=
XX
µ(ei , e)PX (e, {ej }↓ )
ei ∈E e
=
X
e
159
160
161
162
163
164
165
166
167
X
µ(ei , e) = PX (e1 , {ej }↓ ) ≤ 1,
ei ∈E
where in (∗) we used the fact that µ(ei , e) = 0 for ei e (see, e.g., proof of
Proposition 1 in Rota [18]).
Note that submatrix PZ with column and row corresponding to coffin
state eM +1 excluded, can be written as (C−1 PX C)T , i.e. we have
PZ (ej , ei ) = (C−1 PX C)T (ej , ei ) for ej , ei ∈ E. Chapman-Kolmogorov
equations allow to extend the transition probabilities to PnZ (ej , ei ) for all
ej , ei ∈ E∗ and n ≥ 0. Then, the dual chain is defined. To see that (2.1)
holds for ej , ei ∈ E note that since eM +1 is absorbing state, we must have
PnZ = (C−1 PnX C)T , i.e.
(4.3)
168
PX (e, {ej }↓ )
C(PnZ )T = PnX C,
which is (2.1) written in a matrix form.
169
170
Remark 4.2. Note that the Siegmund dual chain will have two absorbing
states. Beside the extra state eM +1 , the state which is maximal w.r.t is
6
171
172
also absorbing. Indeed
PZ (eM , ei ) =
=
X
0 e
eX
i
µ(ei , e0 )PX (e0 , {eM }↓ )
µ(ei , e0 ) = 1(ei = eM ).
e0 ei
173
174
175
176
Remark 4.3. The assumption on existence of the
relaxed, as the following example shows:

 4 1 1 
1
6
6
6
 4 1 1 



PX = 
 6 6 6 , C =  0
1
1
4
0
6
6
6
minimal state cannot be
0 1


1 1 
.
0 1
In other words, we have three states, say e1 , e2 , e3 . The state e3 is a maximal
one and states e1 , e2 are incomparable. Then, we can calculate
 1 1 1 
2
2

(C−1 PX C)T = 
 0
0
0
6
1
6
0
1

.

177
Thus, PZ given in (4.2) does not define a subprobability kernel.
178
Remark 4.4. For given X we can distinguish two Siegmund dual chains. The
one defined in (2.1) can be called Siegmund↓ dual, say Z ↓ . The other, say
Z ↑ , called Siegmund↑ dual, is the one fulfilling
179
180
∀(ei , ej ∈ E) ∀(n ≥ 0) PnX (ei , {ej }↑ ) = PnZ (ej , {ei }↓ ).
181
182
183
184
185
The monotonicity defined in Lorek and Szekli [15] was actually called ↓ Möbius monotonicity, whereas ↑ -Möbius monotonicity was defined as (CT )−1 PCT ≥
0. In a similar way one can have a version of Theorem 4.1, showing that the
Siegmund↑ dual exists if and only if X is ↑ -Möbius monotone. We skip the
details, noting that these monotonicities are not equivalent.
186
5. Siegmund duality and Strong Stationary Duality
187
In this Section we point out a connection to Strong Stationary Duality.
Diaconis and Fill [4] show that their Strong Stationary Dual chain can be
constructed in three steps, where one step involves calculating the Siegmund
dual w.r.t. to total ordering. It turns out that Strong Stationary Dual chain
given in Lorek and Szekli [15] can be constructed in similar three steps, where
one involves calculating the Siegmund dual w.r.t. fixed partial ordering.
Recall that X ∼ (ν, P) is an ergodic Markov chain on finite state space
E = {e1 , . . . , eM } with the stationary distribution π. Let E∗ = {e∗1 , . . . , e∗N }
be, possibly different, state space of absorbing Markov chain X ∗ ∼ (ν ∗ , P∗ )
whose unique absorbing state is denoted by e∗N . Define Λ, a matrix of size
7
188
189
190
191
192
193
194
195
196
197
198
N ×M , to be a link if it is a stochastic matrix with the property: Λ(e∗N , e) =
π(e). We say that X ∗ is a Strong Stationary Dual (SSD) of X with link Λ if
ν = ν ∗ Λ and ΛP = P∗ Λ.
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
Diaconis and Fill [4] prove that then the absorption time T ∗ of X ∗ is so
called Strong Stationary Time for X. This is such a random variable T that
XT has distribution π and T is independent from XT . The main application
is in studying the rate of convergence of an ergodic chain to its stationary
distribution, since for such random variable we always have: dT V (νPk , π) ≤
sep(ν, Pk , π) ≤ P (T > k), where dT V stands for total variation distance,
and sep stands for separation ”distance“. For details see Diaconis and Fill
[4].
In general, there is no recipe on how to find SSD, i.e., a triple E∗ , P∗ , Λ. In
Diaconis and Fill [4] authors give a recipe for a dual on the same state space
←
−
E∗ = E provided that time reversed chain X is stochastically monotone with
respect to total ordering. For given P let h be its harmonic function, i.e.,
such a nonnegative function h that Ph = h. By bold h denote the vector
h = (h(e1 ), . . . , h(eM )). Then, Ph defined as Ph (e, e0 ) = P(e, e0 )h(e0 )/h(e)
on {e : h(e) > 0} is a transition matrix and is often called Doob h-transform.
The SSD, in case when time reversed chain is stochastically monotone, is then
given by authors in three steps (Theorem 5.5 in Diaconis and Fill [4]):
←
−
i) Calculate time reversal P of P.
←
−
←
−
ii) Calculate the Siegmund dual ( P )Z of P .
←
−
←
−
iii) Calculate the Doob H-transform P∗ = (( P )Z )H of ( P )Z , where
H = πC with π = (π(e1 ), . . . , π(eM )).
In above procedure the Siegmund dual must be calculated w.r.t. total ordering, and then we have C(e, e0 ) = 1(e ≤ e0 ).
In Lorek and Szekli [15] we give a recipe for SSD on the same state space
E∗ = E for chains, whose time-reversal is Möbius monotone with respect to
some partial ordering expressed by matrix C(e, e0 ) = 1(e e0 ). In matrix
form, the transitions are given by:
←
−
P∗ = diag(H)−1 (C−1 P C)T diag(H),
P
where H = πC (i.e., H = (H(e1 ), . . . , H(eM )) with H(e) = e0 :e0 e π(e))
and diag(H) denotes diagonal matrix with vector H on the diagonal. Thus,
←
−
P∗ is a Doob H-transform of (C−1 P C)T what is exactly the Siegmund dual
←
−
of P from Theorem 4.1. In other words, SSD given in Lorek and Szekli [15]
results from exactly the same steps i)–iii) as in Theorem 5.5 in Diaconis and
Fill [4].
Remark 5.1. For non-linear ordering Möbius monotonicity and stochastic
monotonicity are, in general, different. In particular, we can have a chain
which is not stochastically monotone, but for which we can construct both,
8
241
Siegmund dual and Strong Stationary Dual, since it can be Möbius monotone. According to our knowledge Falin [5] was the first who observed that
these are two different notions of monotonicity (however, connections to
daulities were not given there). In a subsequent paper Lorek and Markowski
[14] we give more details on relations between Möbius, realizable, weak and
usual (strong) stochastic monotonicities in chains on partially ordered state
spaces.
242
6. Applications
235
236
237
238
239
240
Let us recall the relation between the stationary distribution of X and
absorption probabilities of its Siegmund dual Z (i.e., (2.3))
π({ej }↓ ) = P (Z is absorbed in eM |Z0 = ej ).
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
This relation can have three potential applications:
A1) Suppose we are given Markov chain Z with two absorbing states
(winning and losing). To calculate the probability of being absorbed
in one of them we can find an ergodic chain X for which Z is its
Siegmund dual. Then the task reduces to calculating the stationary
distribution of X.
A2) Suppose we are given ergodic Markov chain X, the task is to find its
stationary distribution. Then we can calculate its Siegmund dual Z
and if we are able to find absorption probabilities, then we have the
stationary distribution of X
A3) Similarly to A2): We are given ergodic chain X. Assume that the
approach given in b) is infeasible, we can find Siegmund dual but
we are unable to calculate its absorption probabilities. However,
we can have so-called stable simulation scheme for estimating the
stationary distribution. Say we want to estimate π(e). Then we
calculate Siegmund dual Z of X and we simulate it several times
always starting the chain at e and simply estimate the winning/losing
probability. Note that (in contrast to most Monte Carlo Markov
Chains methods) we do not need any knowledge about, e.g. the rate
of convergence or time to absorption, we simply run the chain Z until
it is absorbed.
In this Section we provide three examples for A1, A2 and A3 types of applications. In the first two we exploit some knowledge from literature on
the stationary distribution (A1) or absorption probabilities (A2) to solve the
problem, whereas for application A3 we present all the calculations needed
for stable simulation scheme for some nonstandard tandem network of two
stations.
6.1. Application A1: Some nonsymmetric game: explicit formula
for probabilities of winning and losing. Consider the following game.
The states of the game are (0, W ON, LOST ). The state 0 means we did
not yet win nor lose, the names of other states speak for themselves. We
9
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
start at state 0. At each step we can win with probability p > 0, lose with
probability q > 0 or nothing can happen with probability 1 − (p + q). Of
course, eventually we will end up in W ON or LOST state with probabilities
p/(p + q) and q/(p + q) respectively.
Now consider the following generalization. We are given d games G1 , . . . , Gd
Pd
and parameters pi , qi , i = 1, . . . , d such that
i=1 (pi + qi ) ≤ 1. We win
the whole game if we win all the games G1 , . . . , Gd and we lose the whole
game if we lose at least one game Gi , i ∈ {1, . . . , d}. If we win game Gi
we will not play it anymore. The generic state is either eL := LOST or
e = (e(1), . . . , e(d)), e(i) ∈ {0, 1}, i = 1, . . . , d, where e(i) = 1 means that
we have already won the game Gi . Denote also eW := (1, . . . , 1). At each
step we can win new game Gi with probability pi (provided we have not
won it already) or lose it with probability qi (in which case we automatically
lose P
the whole game), or we can do nothing with the remaining probability
1 − i:e(i)=0 (pi + qi ). The described dynamics is a Markov chain, say Z on
state space E∗ = {0, 1}d ∪ {eL } with the following transitions:

pi
if e0 = e + si ,



X



qi
if e0 = eL , e 6= eL ,



i:e(i)=0
X
PZ (e, e0 ) =

1
−
(pi + qi ) if e0 = e 6= eL ,




i:e(i)=0



 1
if e0 = e = e ,
L
290
291
292
293
294
295
296
297
298
where si = (0, . . . , 0, 1, 0, . . . , 0) with 1 at the i-th coordinate. The chain has
two absorbing states: eL and eW (we lose or we win). Eventually, the chain
will be absorbed in one of them. A natural question arises: What is the
probability of winning the whole game starting with arbitrary set of already
won games e0 ∈ E? In other words, we want to calculate the following
probabilities:
ρ(e0 ) = P (τeM < τe0 | Z0 = e0 ),
where τe := inf{n ≥ 0 : Zn = e}. To answer the question consider first
another chain, the nonsymmetric random walk X on E = {0, 1}d , defined
for the same parameters pi , qi , i = 1, . . . , d, with transitions:

if e0 = e + si ,

 qi


 p
if e0 = e − si ,
i
0
PX (e, e ) =
X
X


1−
qi −
pi if e0 = e.



i:e(i)=0
i:e(i)=1
Under mild assumption that at least for one state e we have that PX (e, e) >
0, the chain is ergodic with the stationary distribution:
Y
Y
pi
qi
.
π(e) =
p i + qi
pi + qi
i:e(i)=0
i:e(i)=1
10
299
300
301
302
P
Let |e| = di=1 e(i) (called a level of e). We consider coordinate-wise ordering, i.e. e e0 if e(i) ≤ e0 (i) for all i = 1, . . . , d. The state e1 = (0, . . . , 0)
is a unique minimal and eM = (1, . . . , 1) is a unique maximal one. Then,
(E, ) is a Boolean lattice with the following Möbius function:
0
µ(e, e0 ) = (−1)|e |−|e| 1(e e0 ).
The chain was considered in Lorek and Szekli [15], where we calculated its
Strong Stationary Dual w.r.t to above partial order. The chain is Möbius
P
monotone if and only if di=1 (pi + qi ) ≤ 1. ”In between the lines“ we also
calculated its Siegmund dual (the reader is referred to Lorek and Szekli [15]
for calculations). It turns out, that PZ is the Siegmund dual of PX and
thus, via (2.3), we have
X Y
Y
qi
pi
ρ(e0 ) =
.
p i + qi
p i + qi
0
ee i:e(i)=1
303
i:e(i)=0
For example, if pi = p and qi = q for all i = 1, . . . , d, then we have
X
1
ρ(e0 ) =
q |e| pd−|e|
(p + q)d
ee0
d−|e0 |
|e0 | 0 X
1
|e | k d−k
p
=
q p
=
.
k
p+q
(p + q)d
k=0
304
305
306
307
308
309
310
311
312
313
314
315
316
Note that then of course the probability ρ(e0 ) depends only on level |e0 |. In
0
particular, for p = q we have that ρ(e0 ) = 2d−|e | .
Remark 6.1. Matrix PZ can be written as upper-triangular and thus
P we can
read the eigenvalues from the diagonal. These are equal λA = 1 − i∈A (pi +
qi ) for all A ⊆ {1, . . . , d}. Because of relation (4.3), these are also the
eigenvalues of PX .
6.2. Application A2: Finding stationary distribution of some extension of birth and death chain. Consider classical birth and death
chain on state space E = {1, 2, . . . , N } with constant birth rate q > 0 and
death rate p > 0. Assume p 6= q and p + q < 1. Let X be the following
modification of this birth and death chain: in addition, there is extra probability 1 − p − q of going from any state to maximal state N . Formally, the
transitions of X are given by
(6.1)

q
if j = i + 1, i = 1, . . . , N − 2,






1−p
if (i = N − 1, j = N ) or (i = j = N ),


p
if (j = i − 1, i = 2, . . . , N ) or (i = j = 1),
PX (i, j) =




1 − p − q if i = 1, . . . , N − 2, j = N,




0
otherwise.
11
317
318
319
Theorem 6.2. Consider Markov chain X on E = {1, . . . , N } with transitions given in (6.1). Assume that p 6= q and p + q < 1. Then, the stationary
distribution of the chain is given by


b k2 c−1 b k−1
c
2
X
X
1
k
k
−
1

γj −
γj 
2p
2j + 1
2j + 1
j=0
j=0
(6.2) π(k) =
2(2p)N −k+2 ,
b N 2−1 c X
N
γj
2j + 1
j=0
320
where γ = (1 − 4pq).
321
Proof. First, we will calculate Siegmund dual of X. Consider linear ordering
:=≤ on E. The transitions of Siegmund dual are then calculated using in
(2.2), i.e., PZ (j, i) = PX (i, {j}↓ ) − PX (i + 1, {j}↓ ). Considering all the cases
we have
322
323
324
PZ (j, j − 1)
= PX (j − 1, {j}↓ ) − PX (j, {j}↓ )
= q + p − p = q, j = 2, . . . , N,
PZ (j, j + 1)
= PX (j + 1, {j}↓ ) − PX (j + 2, {j}↓ )
= p − 0 = p, j = 1, . . . , N − 1,
PZ (N − 1, N ) = PX (N, {(N − 1)}↓ ) = p,
= PX (N, {N }↓ ) = 1.
P
For every j = 1, . . . , N − 1 we have that i PZ (j, i) < 1, more precisely
PZ (N, N )
325
N
X
i=1
N
X
PZ (1, i) = p < 1,
P(j, i)
= p + q < 1, j = 2, . . . , N − 1.
i=1
326
327
Thus we add one extra state, call it 0 obtaining transition matrix of Siegmund
dual Z on E∗ = {0, 1, . . . , N }:

p
if j = i + 1, i = 1, . . . , N − 1,






q
if j = i − 1, i = 2, . . . , N − 1,




 1−p
if i = 2, j = 1,
(6.3)
PZ (i, j) =

1 − p − q if i = 2, . . . , N − 1, j = 0,






1
if (i = j = 0) or (i = j = N ),




0
otherwise.
12
These are exactly the transitions corresponding to Gambler’s Ruin with
Catastrophes considered in Hunter [11]. The ruin probability is given therein
in eq. (2.6), the winning probability reads (with ρ(0) = 0)
k k
√
√
1 + 1 − 4pq
1 − 1 − 4pq
−
2p
2p
ρ(k) = N N .
√
√
1 + 1 − 4pq
1 − 1 − 4pq
−
2p
2p
For linear ordering the relation (2.3) reads π({k}↓ ) = ρ(k), thus
π(k) = ρ(k) − ρ(k − 1).
Above relation and some elementary calculations using binomial expansion
k−1
(1 +
√
x)k − (1 −
√
b 2 c
√ X
k
k
x) = 2 x
xj
2j + 1
j=0
328
yield (6.2) and thus complete the proof.
329
6.3. Application A3: Stable simulation scheme for some nonstandard tandem network of two servers. In this Section we will present
some two-node closed tandem network with the unknown stationary distribution (which is surely not of product form). Studying the absorption probability in the resulting Siegmund dual chain is a rather complicated task.
We will present stable simulation scheme and will estimate the stationary
distribution for some concrete parameters. It is also worth noting that the
structure of the resulting Siegmund dual is interesting, e.g., once it hits some
barriers, it will not leave them, see details below.
Consider a tandem network of two stations, each with finite buffer of size
N . Denote the chain by X ≡ (X)n≥0 with state space E = {0, . . . , N }2 ,
where state (x, y) denotes that there are x customers at server 1 and y
customers at server 2.
When x > 0 and y < N the system operates as the following classical
tandem: with probability λ1 there is arrival to station 1 (if x < N ); with
probability λ2 there is arrival to station 2 (if y < N ); with probability µ1
customers traverse from server 1 to server 2; with probability µ2 the customer
from server 2 leaves the network; with the remaining probability nothing
happens. Without loss of generality we assume that λ1 + µ1 + λ2 + µ2 = 1.
When x = 0 then the probability of arrival to station 2 is λ2 + µ1 (if y < N )
and when y = N there is a departure from server 1 with probability µ1 (if x >
0). This can be seen as a modification of standard Gordon–Newell network.
Similar modification (of Jackson network, i.e., with countable state space)
where considered in Foley and McDonald [7], where the rough asymptotics
(and large deviations) were derived, showing that the modification (only on
borders) changes the stationary distribution dramatically.
13
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
Transitions of the chain under consideration are as follows:
0
0
(6.4) PX ((x, y), (x , y )) =



































































356
357
λ1
if x0 = x + 1, y 0 = y,
λ2
if y 0 = y + 1, x0 = x > 0,
λ2 + µ1 if y 0 = y + 1, x0 = x = 0,
µ2
if y 0 = y − 1, x0 = x,
µ1
if x0 = x − 1, y 0 = y + 1,
µ1
if x0 = x − 1, y 0 = y = N,
µ2
if x0 = x < N, y 0 = y = 0,
µ2 + λ1 if x0 = x = N, y 0 = y = 0,
µ1 + λ2 if x0 = x = 0, y 0 = y = N,
λ1
if x = x0 = N, 0 < y 0 = y < N,
λ2
if 0 < x0 = x < N, y 0 = y = N,
λ1 + λ2 if x0 = x = y 0 = y = N.
Consider coordinate-wise ordering, i.e., (x, y) (x0 , y 0 ) ⇐⇒ x ≤ x0 and
y ≤ y 0 . Then, the Möbius function is following:
(6.5)

1
if






−1 if
µ((x, y), (x0 , y 0 )) =






0
otherwise.
358
359
(x0 = x and y 0 = y) or
(x0 = x + 1 and y 0 = y + 1),
(x0 = x + 1 and y 0 = y) or
(x0 = x and y 0 = y + 1),
We will calculate transition matrix PZ of the Siegmund dual Z directly from
Theorem 4.1:
PZ ((x1 , y1 ), (x2 , y2 )) =
X
µ((x2 , y2 ), (x, y))PY ((x, y), (x1 , y1 )↓ ),
(x,y)(x2 ,y2 )
360
where PY ((x, y), (x1 , y1 )↓ ) :=
361
function µ we have
X
PY ((x, y), (x0 , y 0 )). With our Möbius
(x0 ,y 0 )(x1 ,y1 )
PZ ((x1 , y1 ), (x2 , y2 ))
14
362
(6.6) = PY ((x2 , y2 ), (x1 , y1 )↓ ) − PY ((x2 + 1, y2 ), (x1 , y1 )↓ )
−PY ((x2 , y2 + 1), (x1 , y1 )↓ ) + PY ((x2 + 1, y2 + 1), (x1 , y1 )↓ ),
363
364
365
366
where we should understand that for example, for y2 = N the corresponding
terms, in this case PY ((x2 , y2 +1), (x1 , y1 )↓ ) and PY ((x2 +1, y2 +1), (x1 , y1 )↓ )
are equal to 0. Let us start with the case: x1 , x2 , y1 , y2 > 1 and x1 , x2 , y1 , y2 <
N . Then, using (6.6) we have for example:
PZ ((x1 , y1 ), (x1 − 1, y1 ))
367
= PY ((x1 − 1, y1 ), (x1 , y1 )↓ ) − PY ((x1 , y1 ), (x1 , y1 )↓ )
−PY ((x1 − 1, y1 + 1), (x1 , y1 )↓ ) + PY ((x1 , y1 + 1), (x1 , y1 )↓ )
= λ1 + µ2 − µ2 − µ2 + µ2 = λ1 .
368
369
We have also PZ ((x1 , y1 ), (x1 − k, y1 )) = 0 for k = 1, . . . , x1 − 1. Considering
some possible transitions to border of the state space:
PZ ((x1 , y1 ), (0, y1 − 1))
370
= PX ((0, y1 − 1), (x1 , y1 )↓ ) − PX ((1, y1 − 1), (x1 , y1 )↓ )
−PX ((0, y1 ), (x1 , y1 )↓ ) + PX ((1, y1 ), (x1 , y1 )↓ )
= 1 − 1 − (λ1 + µ2 ) + (µ2 + λ1 ) = 0.
371
372
373
374
We will skip the rest of (lengthy) calculations, but considering case-bycase, X turns out to be Möbius monotone. We add one extra (coffin) state,
let us name it (+∞, +∞). The transitions of the Siegmund dual are as
follows:
15
375
376
PZ ((x, y), (x0 , y 0 )) =






























(6.7)





























where η(x, y) =
(x2 ,y2
377
378
379
380
381
382
383
384
385
386
387
λ1
if N − 1 > x0 = x − 1, y 0 = y,
λ2
if N − 1 > y 0 = y − 1, x0 = x < N,
λ2 + µ1
if N − 1 > y 0 = y − 1, x0 = x = N,
µ1
if x0 = x + 1, N − 1 > y 0 = y − 1,
µ2
if x0 = x, y 0 = y + 1,
λ1 + λ2 + µ1 if x = 0, y = 0, x0 = +∞, y 0 = +∞, ,
λ2 + µ1
if x > 0, y = 0, x0 = +∞, y 0 = +∞,
λ1
if x = 0, y > 0, x0 = +∞, y 0 = +∞,
1
if x0 = x = y 0 = y = N
or x0 = x = y 0 = y = +∞,
1 − η(x, y)
X
)∈E∗ :(x
if x0 = x, y 0 = y.
PX ((x, y), (x2 , y2 )). Note that outside
2 ,y2 )6=(x,y)
the borders the transitions look like some reversed network (however, note
that this is not the usual time reversal). The behavior on the borders is
different. First, the chain can go to (+∞, +∞) only from states of form
(0, y) or (x, 0). Second, once the process is on ”upper“ ((x, N )) or ”right“
((N, y)) border, it behaves like a usual, absorbing birth and death chain with
probabilities of being killed only possible on (0, N ) and (N, 0) respectively.
On ”upper“ border, birth rate is µ1 and death rate is λ1 , whereas on ”right“
border the birth rate is µ2 and death rate is µ1 + λ2 .
The similar behaviour (i.e., not leaving the borders) we also noticed in
Strong Stationary Dual for two-node network representing two independent
servers, see Lorek and Szekli [16] for details.
Stable simulation scheme. To estimate π({(x, y)}↓ ) we simply estimate
winning probability in Siegmund dual Z by running the chain K times independently, each time starting from (x, y). Denote the output of i-th run as
Yi ∈ {0, 1}, where 0 means that the chain was absorbed in +∞ and 1 means
that the chain was absorbed in (N, N ). Thus, we can estimate the winning
probability by
K
1 X
Yi .
ρ̂(x, y) =
K
i=1
By relation relation (2.3) we thus estimated stationary distribution: π̂({(x, y)}↓ ) =
ρ̂(x, y). If we wish to estimate π(x, y), then we can use Remark 3.2 to get
16
(for x > 0, y > 0)
π̂(x, y) =
=
X
µ((x0 , y 0 ), (x, y))π̂({(x0 , y 0 }↓ )
(x0 ,y 0 )(x,y)
π̂({(x, y)}↓ )
− π̂({(x − 1, y)}↓ )
− π̂({(x, y − 1)}↓ ) + π̂({(x − 1, y − 1)}↓ ),
388
389
390
391
392
393
394
thus we need separately to estimate π({(x, y)}↓ ), π({(x − 1, y)}↓ ), π({(x, y −
1)}↓ ) and π({(x − 1, y − 1)}↓ ).
Numerical results. We estimated the stationary distribution of the tandem
for N = 5 (thus the state space is E = {0, . . . , 5}2 with |E| = 36) with
3
1
6
parameters λ1 = 16
, λ2 = 16
, µ1 = µ2 = 16
. The simulations were performed
in The Julia Language. For each (x, y) ∈ E we performed independently
K = 1 mln iterations. Results are presented in Fig. 1.
y\x
0
1
2
3
4
5
0
0.029739
0.028980
0.022189
0.012979
0.008949
0.006902
1
0.048597
0.034663
0.021324
0.013645
0.006901
0.002193
2
0.067564
0.04128
0.020593
0.010742
0.005404
0.003982
3
0.087365
0.046606
0.024072
0.009349
0.005434
0.001226
4
0.11586
0.051627
0.020465
0.010822
0.003319
0.000404
5
0.159256
0.050266
0.018471
0.006062
0.001679
0.001091
Figure 1. Estimated π̂(x, y), x, y ∈ {0, . . . , N }, N = 5, λ1 =
1
6
3
16 , λ2 = 16 , µ1 = µ2 = 16
398
Performing the simulations using 2 mln iterations for each (x, y) ∈ E
yields in results agreeing with above estimation on the first three digits after
comma. The L2 distance between π̂ estimated using 1 mln iterations and π̂
estimated using 2 mln iterations was less then 2 · 10−5 .
399
Acknowledgments
400
403
Author thanks Bartłomiej Błaszczyszyn and Zbigniew Palmowski for helpful discussions. Partial support by the project RARE-318984, a Marie Curie
IRSES Fellowship within the 7th European Community Framework Programme, is kindly acknowledged.
404
References
395
396
397
401
402
405
406
407
408
409
410
411
[1] B. Błaszczyszyn and K. Sigman. Risk and duality in multidimensions. Stochastic
processes and their applications, 83:331–356, 1999.
[2] J. T. Cox and U. Rösler. A duality relation for entrance and exit laws for Markov
processes. Stochastic processes and their applications, 16:141—-156, 1984.
[3] H. Dette, J. A. Fill, J. Pitman, and W. J. Studden. Wall and Siegmund Duality
Relations for Birth and Death Chains with Reflecting Barrier. Journal of Theoretical
Probability, 10(2):349–374, Apr. 1997.
17
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
[4] P. Diaconis and J. A. Fill. Strong stationary times via a new form of duality. The
Annals of Probability, 18(4):1483–1522, 1990.
[5] G. Falin. Monotonicity of random walks in partially ordered sets. Russian Mathematical Surveys, 167, 1988.
[6] J. A. Fill and V. Lyzinski. Strong Stationary Duality for diffusion processes. arXiv,
pages 1–33, 2014.
[7] R. D. Foley and D. R. McDonald. Large deviations of a modified Jackson network:
Stability and rough asymptotics. The Annals of Applied Probability, 15(1B):519–541,
Feb. 2005.
[8] J. M. Harrison and R. J. Williams. Multidimensional Reflected Brownian Motions
Having Exponential Stationary Distributions. The Annals of Probability, 15(1):115–
137, Jan. 1987.
[9] T. Huillet. Siegmund duality with applications to the neutral Moran model conditioned on never being absorbed. Journal of Physics A: Mathematical and Theoretical,
43(37), Sept. 2010.
[10] T. Huillet and S. Martínez. On Möbius Duality and Coarse-Graining. Journal of
Theoretical Probability, 29(1):143—-179, July 2016.
[11] B. Hunter. Gambler’ s Ruin with Catastrophes and Windfalls. Journal of Statistical
Theory and Practice, 2(2):199—-219, 2008.
[12] T. M. Liggett. Interacting Particle Systems. 2004.
[13] D. V. Lindley. The theory of queues with a single server. Mathematical Proceedings
of the Cambridge Philosophical Society, 48(02):277–289, Oct. 1952.
[14] P. Lorek and P. Markowski. Monotonicity requirements for exact sampling with
Markov chains. Technical report, 2016.
[15] P. Lorek and R. Szekli. Strong stationary duality for Möbius monotone Markov chains.
Queueing Systems, 71(1-2):79–95, Mar. 2012.
[16] P. Lorek and R. Szekli. Strong Stationary Duality for Möbius monotone Markov
chains: examples. Probability and Mathematical Statistics, 36(1):75—-97, 2016.
[17] L. Miclo. Strong stationary times for one-dimensional diffusions. arXiv preprint
arXiv:1311.6442, pages 1–40, 2013.
[18] G.-C. Rota. On the foundations of combinatorial theory I. Theory of Möbius functions. Probability theory and related fields, 368:340–368, 1964.
[19] D. Siegmund. The equivalence of absorbing and reflecting barrier problems for
stochastically monotone Markov processes. The Annals of Probability, 1976.
18