MAS328/MTH354
Solutions to the final exam.
QUESTION 1.
(20 marks)
Consider a Markov chain (Xn )n≥0 on the state space {0, 1, 2, 3, 4} with transition probability matrix P given by


1/3 2/3
0
0
0
 1/2 1/2
0
0
0 


0
1
0
0 
P =
 0
.
 0
0
1/7 6/7 0 
1
0
0
0
0
(i) Draw the graph of this chain.
The chain has the following graph:
2/3
1/3
0
1/7
1/2
1
6/7
3
1/2
2
4
1
1
(ii) Identify the communicating class(es).
The communicating classes are {0, 1}, {2}, {3}, and {4}.
(iii) Find the recurrent, transient, and absorbing state(s) of this chain.
States 3 and 4 are transient, states 0 and 1 are recurrent, and
state 2 is absorbing (hence recurrent).
(iv) Find limn→∞ IP(Xn = 0 | X0 = 4).
1
MAS328/MTH354
We have
lim IP(Xn = 0 | X0 = 4) = lim IP(Xn = 0 | X0 = 0) =
n→∞
n→∞
1/2
3
= .
2/3 + 1/2
7
QUESTION 2.
(20 marks)
Each individual in a population has a random number ξ of offsprings, with
distribution
IP(ξ = 0) = 0.2,
IP(ξ = 1) = 0.5,
IP(ξ = 2) = 0.3.
Let Xn denote the number of individuals in the population at the nth generation, with X0 = 1.
(i) Compute the mean and variance of X2 .
We have G1 (s) = 0.2 + 0.5s + 0.3s2 and
E[X1 ] = E[ξ] = G01 (1) = 0.5 + 2 × 0.3 = 1.1,
hence
E[X2 ] = (G01 (1))2 = (E[ξ])2 = (1.1)2 .
On the other hand we have
G2 (s) =
=
=
=
G1 (G1 (s))
G1 (0.2 + 0.5s + 0.3s2 )
0.2 + 0.5(0.2 + 0.5s + 0.3s2 ) + 0.3(0.2 + 0.5s + 0.3s2 )2
0.312 + 0.31s + 0.261s2 + 0.09s3 + 0.027s4 ,
with
G02 (s) = 0.31 + 0.522s + 0.27s2 + 0.108s3
and
G002 (s) = 0.522 + 0.54s + 0.324s2 ,
2
MAS328/MTH354
hence
G02 (1) = 1.21
and
G002 (1) = 1.386,
and
E[X22 ] = G002 (1) + G02 (1)
= 1.386 + 1.21
= 2.596.
This yields
Var[X2 ] = 2.596 − (1.21)2 .
(ii) Give the probability distribution of the random variable X2 .
We have
G2 (s) = 0.312 + 0.31s + 0.261s2 + 0.09s3 + 0.027s4 ,
hence
P (X2 = 0) = 0.312,
P (X2 = 1) = 0.31,
P (X2 = 2) = 0.261,
and
P (X2 = 3) = 0.09,
P (X2 = 4) = 0.027.
(iii) Compute the probability that the population is extinct by the 4th generation.
We have
P (X4 = 0) = G4 (0)
= G2 (G2 (0))
= 0.312 + 0.31 × 0.312 + 0.261 × 0.3122 + 0.09 × 0.3123 + 0.027 × 0.3124
' 0.443143592.
(iv) What is the probability of extinction of this population?
The extinction probability α solves the equation
α = G1 (α) = 0.2 + 0.5α + 0.3α2 ,
3
MAS328/MTH354
i.e.
0.3α2 − 0.5α + 0.2 = 0.3(α − 1)(α − 2/3) = 0,
i.e. α = 2/3.
QUESTION 3.
(20 marks)
Customers arrive at a processing station according to a Poisson process with
rate λ = 0.1, i.e. on average one customer per ten minutes. Processing of
customer queries starts as soon as the third customer enters the queue.
(i) Compute the expected time until the start of the customer service.
This time is the expected value of the third jump time, i.e. 3/λ = 30
minutes.
(ii) Compute the probability that no customer service occurs within the
first hour.
This probability is
P (N60 < 3) =
=
=
'
P (N60 = 0) + P (N60 = 1) + P (N60 = 2)
e−60λ (1 + 60λ + (60λ)2 /2)
25e−6
0.062.
QUESTION 4.
(20 marks)
Consider a Markov chain (Xn )n≥0 on the
tion probability matrix P given by

0
1
 0.2
0
P =
 0.3
0
0.4 0.6
4
state space {0, 1, 2, 3} with transi0
0.8
0.7
0

0
0 
.
0 
0
MAS328/MTH354
(i) Draw the graph of this chain. Is the chain reducible?
The chain has the following graph:
0.6
3
0.4
0
1
1
0.8
2
0.2
0.7
0.3
The chain is reducible and its communicating classes are {0, 1, 2} and
{3}.
(ii) Find the recurrent, transient, and absorbing state(s) of this chain.
State 3 is transient, and states 0 , 1 , 2 are recurrent.
(iii) Compute the fraction of time spent at state 0 in the long run.
It suffices to consider the subchain

0
P̃ =  0.2
0.3
on {0, 1, 2} with transition matrix

1
0
0 0.8  ,
0 0.7
and to solve π = π P̃ , i.e.

 π0 = 0.2π1 + 0.3π2
π1 = π0

π2 = 0.8π1 + 0.7π2
which yields π1 = π0 and 0.3π2 = 0.8π1 = 0.8π0 , with
8
1 = π0 + π1 + π2 = 2π0 + π0 ,
3
5
MAS328/MTH354
i.e.
3
3
4
, π1 = , π2 = ,
14
14
7
and the fraction of time spent at state 0 in the long run is 3/14 '
0.214.
π0 =
(iv) On the average, how long does it take to reach state 0 after starting
from state 2 ?
Letting h0 (k) denote the mean hitting time of state 0 starting from
state k , we have

h0 (0) = 0



h0 (1) = 0.2(1 + h0 (0)) + 0.8(1 + h0 (2))
h0 (2) = 0.3(1 + h0 (0)) + 0.7(1 + h0 (2))



h0 (3) = 0.4(1 + h0 (0)) + 0.6(1 + h0 (1)),
i.e.
or
hence

h0 (0) = 0



h0 (1) = 1 + 0.8h0 (2)
h0 (2) = 1 + 0.7h0 (2)



h0 (3) = 1 + 0.6h0 (1)

h0 (0) = 0



h0 (1) = 1 + 0.8h0 (2)
0.3h0 (2) = 1



h0 (3) = 1 + 0.6h0 (1)

h0 (0) = 0








11


h0 (1) =


3

10



h0 (2) =


3







 h0 (3) = 16 ,
5
and the mean time to reach state 0 starting from state 2 is 10/3.
6
MAS328/MTH354
QUESTION 5.
(20 marks)
Consider two machines, operating simultaneously and independently, where
both machines have an exponentially distributed time to failure with mean
1/µ. There is a single repair facility, and the repair times are exponentially
distributed with rate λ.
(i) In the long run, what is the probability that no machines are operating
when λ = µ = 1?
The number Xt of machines operating at time t is a birth and death
process on {0, 1, 2} with infinitesimal generator


−λ
λ
0
Q =  µ −(λ + µ) λ  .
0
2µ
−2µ
The stationary distribution π = (π0 , π1 , π2 ) is solution of πQ = 0, i.e.

 0 = −λπ0 + µπ1
0 = λπ0 − (λ + µ)π1 + 2µπ2

0 = λπ1 − 2µπ2
under the condition π0 + π1 + π2 = 1, which yields
λµ
λ2
µ2
,
,
,
(π0 , π1 , π2 ) =
µ2 + λµ + λ2 /2 µ2 + λµ + λ2 /2 2µ2 + 2λµ + λ2
i.e. the probability that no machine is operating is π0 = 2/5 when
λ = µ = 1.
(ii) We now assume that at most one machine can operate at any time.
How does this modify your answer to question 5-(i)?
The number Xt of machines operating at time t is now a birth and
death process on {0, 1}. The time spent in state 0 is exponentially
distributed with average 1/µ. When the chain is in state 1 , one machine is working while the other one may still be under repair, and the
7
MAS328/MTH354
mean time t1 spent in state 1 has to be computed using first step
analysis. We have
1
λ
t1 = +
t1 ,
µ λ+µ
since λ/(λ+µ) is the probability that repair of the idle machine finishes
before failure of the working machine. This yields
t1 =
λ+µ
,
µ2
hence the corresponding rate is µ2 /(λ + µ) and the infinitesimal generator of the chain becomes
−λ
λ
−λ
λ
Q=
=
.
1/t1 −1/t1
µ2 /(λ + µ) −µ2 /(λ + µ)
The stationary distribution π = (π0 , π1 ) is solution of πQ = 0, i.e.

 0 = −λπ0 + π1 µ2 /(λ + µ)

0 = λπ0 − π1 µ2 /(λ + µ)
under the condition π0 + π1 = 1, which yields
λµ + λ2
µ2
,
,
(π0 , π1 ) =
µ2 + λµ + λ2 µ2 + λµ + λ2
i.e. the probability that no machine is operating when λ = µ = 1 is
π0 = 1/3.
8