ETH Zürich, FS 2017
D-MATH
Prof. Dr. A-S Sznitman
Coordinator
Chong Liu
Applied Stochastic Processes
Solution sheet 11
Solution 11.1 Xt denotes the number of customers being in the shop at time t. (Xt )t≥0 is a
Markov chain with finite state space {0, 1, 2}. We obtain the following rates:
λ0 = λ,
λ1 = λ + µ,
λ2 = µ.
Recall that q1,0 denotes the probability that when only one customer is in the shop this customer
is served before another customer arrives. q1,0 is thus specified as P (Tµ < Tλ ). Note that this
probability is given by the convolution of Tλ and Tµ by:
Z ∞Z x
µ
q1,0 = P (Tµ < Tλ ) =
µe−µy λe−λx dy dx =
.
λ+µ
0
0
λ
Furthermore, q0,1 = 1, q0,2 = 0, q1,2 = 1 − q1,0 = λ+µ
, q2,0 = 0, q2,1 = 1, and qi,i = 0 for all
i ∈ {0, 1, 2}. Hence, we obtain the Q-matrix Q = (qi,j )i,j∈{0,1,2} :


0 λ+µ 0
1 
µ
0
λ 
Q=
λ+µ
0 λ+µ 0
Solution 11.2
We have seen in Lemma 4.7 that the non-explosion assumption is equivalent to
X
λ(X̄n )−1 = ∞, P̄x -a.s. for all x ∈ E.
n>0
(a) (1)
X
λ(X̄n )−1 >
n>0
X
c−1 = ∞.
n>0
(2) We have supx∈E λ(x) = c < ∞, hence (2) follows from (1).
(3) P̄x [∩n>0 {X̄n ∈ T }] = 0 implies that for P̄x -almost all ω there is n0 (ω) < ∞ such that
c
X̄n (ω) ∈ T c for all n > n0 (ω). This implies that
P∞P̄x -a.s. there is a state y ∈ T , which
the chain visits infinitely often. Define Nx = n=0 1{X̄n =x} . Then P̄x -a.s. there exists
y ∈ T c , such that Ny = ∞. We have
X
n>0
λ(X̄n )−1 =
X Nx
,
λ(x)
x∈E
hence the assumption λ(y) < ∞ implies the claim.
(b) Take the Markov chain in continuous time on the state space N that starts at 0, that has the
following jump rate and transition probability
λ(x) =(x + 1)2 for x ∈ N,
(
1 if y = x + 1
qx,y =
0 otherwise.
1/2
Applied Stochastic Processes, FS 2017
D-MATH
Solution sheet 11
We have then
X
λ(X̄n )−1 =
n>0
X 1
π2
=
< ∞,
n2
6
n>1
and this chain does not satisfy the non-explosion assumption.
Solution 11.3 At the time when the process enters in state 0, there is a machine being repaired.
Thus, using the memoryless property of Exp, the remaining service time (holding time in state 0)
is still Exp(λ) distributed, hence λ(0) = λ.
When the process enters state M , all machines are operational. Again, using the memoryless
property, the remaining time until failure for each machine is Exp(µ) distributed (independently).
(µ) i.i.d
Let Tk ∼ Exp(µ), k = 1, . . . , M . Then the time until first failure (holding time in state M ) is
(µ)
distributed as min Tk ∼ Exp(M µ), hence λ(M ) = M µ.
k=1,...,M
(λ)
Let T n ∼ Exp(λ). Thenoanalogously, the holding time in state n ∈ {1, . . . , M − 1} is distributed
(µ)
as min
min Tk , T (λ) ∼ Exp(nµ + λ). Thus, λ(n) = nµ + λ for n ∈ {1, . . . , M − 1}.
k=1,...,n
Furthermore, q0,1 = qM,M −1 = 1, and for n = 1, . . . , M − 1,
(µ)
(λ)
qn,n−1 = P
min Tk < T
=
k=1,...,n
qn,n+1 = P
(µ)
min Tk
k=1,...,n
> T (λ) =
nµ
,
nµ + λ
λ
.
nµ + λ
For (n, m) with |n − m| > 1, we have qn,m = 0.
2/2