MH3512/MTH354/MAS328
QUESTION 1.
(20 marks)
Consider a 2-dimensional random walk (Sn )n2N started at S0 = (0, 0) on Z2 , where starting
from a location Sn = (i, j) the chain can move to any of the points (i ± 1, j ± 1) with equal
probabilities.
(a) Suppose that we assume in addition that the random walk cannot visit any site more
than once. Is the resulting system a Markov chain? Prove your answer. (10 marks)
Answer: The resulting system cannot be a Markov chain because the behavior of the
chain at time n + 1 may depend on the past of the chain from time 0 to time n 1.
(b) Let Sn = (Xn , Yn ) denote the coordinates of Sn at time n and let Zn := Xn2 + Yn2 . Is
(Zn )n2N a Markov chain? Prove your answer.
(10 marks)
Hint: Use the fact that a same value of Zn may correspond to di↵erent locations
of (Xn , Yn ) on the circle, for example (Xn , Yn ) = (5, 0) and (Xn , Yn ) = (4, 3) when
Zn = 25.
Answer: The process (Zn )n2N cannot be a Markov chain because when e.g. Zn = 25
the distribution of the next value of the chain at time n + 1 is not uniquely determined
from the value of Zn = 25.
QUESTION 2.
(25 marks)
Consider the Markov chain on {0, 1, 2} with transition matrix
2
3
1/3 1/3 1/3
4 1/4 3/4 0 5 .
0
0
1
a) Is the chain irreducible? Give its communicating classes.
(5 marks)
Answer: The chain is reducible and its communicating classes are {0, 1} and {2}.
b) Which states are absorbing, transient, recurrent, positive recurrent?
(5 marks)
Answer: The states 0 and 1 are transient and the state 2 is recurrent.
c) Find the period of every state.
(5 marks)
Answer: All states have period 1.
1
MH3512/MTH354/MAS328
d) Compute the probability P(T2 < 1 | X0 = 1) of hitting state 2 in finite time starting
from state 1 , and the probability P(T1r < 1 | X0 = 1) of returning to state 1 in
finite time.
!!
(5 marks)
Answer: Let g2 (k) = P(T2 < 1 | X0 = k), k = 0, 1, 2. We have g2 (2) = 1 and by first
step analysis,
8
1
1
1
>
>
g2 (0) = g2 (0) + g2 (1) +
>
>
3
3
3
>
>
>
<
1
3
g
(1)
=
g
(0)
+
g2 (1)
2
2
>
>
4
4
>
>
>
>
>
: g (2) = 1,
2
hence
8
2
1
1
>
>
g2 (0) = g2 (1) +
>
>
3
3
3
>
<
g2 (1) = g2 (0)
>
>
>
>
>
:
g2 (2) = 1,
hence P(T2 < 1 | X0 = 1) = g2 (0) = g2 (1) = g2 (2) = 1.
Next, let g1 (k) = P(T1r < 1 | X0 = k), k = 0, 1, 2. We have g1 (2) = 0 and by first step
analysis we have
8
1
1
>
>
g1 (0) = g1 (0) +
>
>
3
3
>
>
>
<
1
3
g
(1)
=
g
(0)
+
1
1
>
>
4
4
>
>
>
>
>
: g (2) = 0,
1
hence
8
1
>
>
g1 (0) =
>
>
2
>
>
>
<
7
g1 (1) =
>
>
8
>
>
>
>
>
: g (2) = 0,
1
and P(T1r < 1 | X0 = 1) = g1 (1) = 7/8.
e) Compute the mean return time µ1 (1) = E[T1r | X0 = 1] to state 1 and the mean
hitting time h2 (1) = E[T2 | X0 = 1] of state 2 starting from state 1 .
2
MH3512/MTH354/MAS328
(5 marks)
Answer: We have E[T1r | X0 = 1] = +1 because state 1 is transient.
On the other hand, let h2 (k) = E[T2 < 1 | X0 = k], k = 0, 1, 2. We have h2 (2) = 0
and by first step analysis we have
8
1
1
>
>
h2 (0) = 1 + h2 (0) + h2 (1)
>
>
3
3
>
>
>
<
1
3
h
(1)
=
1
+
h
(0)
+
h2 (1)
2
2
>
>
4
4
>
>
>
>
>
: h (2) = 0,
2
hence
hence
8
2h2 (0) = 3 + h2 (1)
>
>
>
>
<
h2 (1) = 4 + h2 (0)
>
>
>
>
:
h2 (2) = 0,
8
h2 (0) = 7
>
>
>
>
<
h2 (1) = 11
>
>
>
>
:
h2 (2) = 0,
and E[T2r < 1 | X0 = 1] = h2 (1) = 11.
QUESTION 3.
(20 marks)
Customers arrive at a processing station according to a Poisson process with rate = 0.1,
i.e. on average one customer per ten minutes. Processing of customer queries starts as
soon as the third customer enters the queue.
a) Compute the expected time until the start of the customer service.
Answer: This time is the expected value of the third jump time, i.e. 3/ = 30 minutes.
(10 marks)
b) Compute the probability that no customer service occurs within the first hour.
!!
(10 marks)
3
MH3512/MTH354/MAS328
Answer: This probability is
P(N60 < 3) = P(N60 = 0) + P(N60 = 1) + P(N60 = 2)
= e 60 (1 + 60 + (60 )2 /2) = 25e 6 ' 0.062.
QUESTION 4.
(20 marks)
A cell culture is started with one red cell at time 0. After one minute the red cell dies and
two new cells are born according to the following probability distribution:
Color configuration
2 red cells
1 red cell + 1 white cell
2 white cells
Probability
1/3
1/2
1/6
The above procedure is repeated minute after minute for any red cell present in the culture.
On the other hand, the white cells can only live for one minute, and disappear after that
without reproducing. We assume that the cells behave independently.
a) What is the probability that no white cells have been generated for n + 1/2 minutes?
!!
(10 marks)
Answer: We compute the probability that only red cells are generated, which is
1
⇥
4
n
✓ ◆2
✓ ◆2n
1
1
⇥ ··· ⇥
4
4
1
=
n
Y1 ✓
k=0
1
4
◆2 k
✓ ◆2n
1
=
4
1
,
0.
b) Compute the extinction probability of the whole cell culture.
(10 marks)
Answer: Since white cells cannot reproduce, the extinction of the culture is equivalent
to the extinction of the red cells. The probability distribution of the number Y of red
cells produced from one red cell is
P(Y = 0) =
1
,
12
2
P(Y = 1) = ,
3
1
P(Y = 2) = ,
4
which has the generating function
G1 (s) = P(Y = 0) + sP(Y = 1) + s2 P(Y = 2)
1
2s s2
1
=
+
+
= (1 + 8s + 3s2 ),
12
3
4
12
4
MH3512/MTH354/MAS328
hence the equation G1 (↵) = ↵ reads
3↵2
4↵ + 1 = 3(↵
1/3)(↵
3) = 0,
which has ↵ = 1/3 for smallest solution. Consequently the extinction probability of
the culture is equal to 1/3.
QUESTION 5.
(15 marks)
Let (Xt )t2R+ be a birth and death process on {0, 1, 2} with birth and death parameters
k) and µk = k, k = 0, 1, 2. Determine the stationary distribution of (Xt )t2R+ .
k = ↵(2
END OF PAPER
5