MAS328
Solutions to the final exam.
Question 1.
(20 marks)
Four players A, B, C, D are connected by the following network, and play by
exchanging one token.
D
A
B
C
At each step of the game, the player who holds the token chooses another
player he is connected to, and sends the token to that player.
(i) Assuming that player choices are made at random and equally distributed, model the states of the token as a Markov chain and give its
transition matrix.
The transition matrix P is given by


0 1/2 0 1/2
 1/3 0 1/3 1/3 
.
P =
 0
1
0
0 
1/2 1/2 0
0
(ii) Compute the stationary distribution (πA , πB , πC , πD ) of (Xn )n≥1 .
Hint: To simplify the resolution, start by arguing that we have πA = πD .
Solving for πP = π we have

 
1
π + 12 πD
0 1/2 0 1/2
3 B
1
 1/3 0 1/3 1/3   πA + πC + 1 πD
2
= 2
πP = [πA , πB , πC , πD ] 
1
 0
π
1
0
0  
3 B
1
π + 13 πB
1/2 1/2 0
0
2 A
= [πA , πB , πC , πD ],
1




MAS328
i.e. πA = πD = 2πC and πB = 3πC , which, under the condition πA +
πB + πC + πD = 1, gives πA = 1/4, πB = 3/8, πC = 1/8, πD = 1/4.
(iii) In the long run, what is the probability that player D holds the token ?
This probability is πD = 0.25.
(iv) On average, how long does player D has to wait to recover the token ?
This average time is 1/πD = 4.
Question 2.
(20 marks)
A system consists of two machines and two repairmen. The amount of time
that an operating machine works before breaking down is exponentially distributed with mean 5. The amount it takes a single repairman to fix a
machine is exponentially distributed with mean 4. Only one repairman can
work on a failed machine at any given time.
(i) Let Xt be the number of machines in operating condition at time t ∈
R+ . Show that Xt is a continuous time Markov process and complete
the missing entries in the matrix


0.5
0

Q =  0.2 0 −0.4
of its generator.
The generator Q of Xt is given by


−0.5 0.5
0
Q =  0.2 −0.45 0.25  .
0
0.4 −0.4
2
MAS328
(ii) Calculate the long run probability distribution (π0 , π1 , π2 ) for Xt .
Solving for πQ = 0 we have


−0.5 0.5
0
πQ = [π0 , π1 , π2 ]  0.2 −0.45 0.25 
0
0.4 −0.4

T
−0.5 × π0 + 0.2 × π1
=  0.5 × π0 − 0.45 × π1 + 0.4 × π2 
0.25 × π1 − 0.4 × π2
= [0, 0, 0],
i.e. π0 = 0.4 × π1 = 0.64 × π2 under the condition π0 + π1 + π2 = 1,
which gives π0 = 16/81, π1 = 40/81, π2 = 25/81.
(iii) Compute the average number of operating machines in the long run.
In the long run the average is
0 × π0 + 1 × π1 + 2 × π2 = 40/81 + 50/81 = 90/81.
(iv) If an operating machine produces 100 units of output per hour, what
is the long run output per hour of the system ?
We find
100 × 90/81 = 1000/9.
Question 3.
(20 marks)
Families in a distant society continue to have children until the first girl, and
then cease childbearing. Let X denote the number of male offsprings of a
particular husband.
(i) Assuming that each child is equally likely to be a boy or a girl, give the
probability distribution of X.
We have P(X = k) = (1/2)k+1 , k ∈ N.
3
MAS328
(ii) Compute the probability generating function G(s) of X.
The probability generating function of X is given by
∞
G(s) = E[sX ] =
1X
1
.
(s/2)k =
2 k=0
2−s
(iii) What is the probability that the husband’s male line of descent will
cease to exist by the third generation ?
The probability we are looking for is
G(G(G(0))) =
1
2−
1
2−1/2
3
= .
4
Question 4.
(20 marks)
Suppose that X(A) is a spatial Poisson process of discrete items scattered
on the plane R2 with intensity λ = 0.5 per square meter. No evaluation of
numerical expressions is required in this question.
(i) What is the probability that 10 items are found within the disk D(0, 3)
with radius 3 meters centered at the origin ?
This probability is
10
−9π/2 (9π/2)
e
10!
.
(ii) What is the probability that 5 items are found within the disk D(0, 3)
and 3 items are found within the disk D(x, 3) with x = (7, 0) ?
This probability is
e−9π/2
(9π/2)5
(9π/2)3
× e−9π/2
.
5!
3!
4
MAS328
(iii) What is the probability that 8 items are found anywhere within D(0, 3)∪
D(x, 3) with x = (7, 0) ?
This probability is
e−9π
(9π)8
.
8!
Question 5.
(20 marks)
Let (ξn )n≥0 be a two-state Markov chain on {0, 1} with transition matrix
0
1
,
1−α α
let (Nt )t∈R+ be a Poisson process with parameter λ > 0, and let the two-state
birth and death process Xt be defined by
t ∈ R+ .
Xt = ξNt ,
(i) Compute the mean return time E[τ0 | X0 = 0] to 0 of Xt .
We have
E[τ0 | X0 = 0] =
and
E[τ0 | X0 = 1] =
1
+ E[τ0 | X0 = 1],
λ
1
+ αE[τ0 | X0 = 1],
λ
hence
E[τ0 | X0 = 1] =
1
λ(1 − α)
E[τ0 | X0 = 0] =
2−α
.
λ(1 − α)
and
5
MAS328
(ii) Compute the mean return time E[τ1 | X0 = 1] to 1 of Xt .
We have
1
+ αE[τ1 | X0 = 1] + (1 − α)E[τ1 | X0 = 0]
λ
2−α
=
+ αE[τ1 | X0 = 1],
λ
E[τ1 | X0 = 1] =
since E[τ1 | X0 = 0] = 1/λ, hence
E[τ1 | X0 = 1] =
2−α
.
λ(1 − α)
(iii) Determine the generator of the process Xt in terms of α and λ.
The generator Q of Xt is given by
−λ
λ
Q=
.
λ(1 − α) −λ(1 − α)
6