MAT 1302 – MARKOV CHAIN EXAMPLE
PROF: ALISTAIR SAVAGE
Suppose a city has 3 internet service providers (A, B, and C). A starts with 200 000
customers and B and C each start with 400 000 customers. Suppose that, each year, the
following migration occurs
.7
A
D _
.3
.2
.4
BL m
.4
.6
.2
-
CR
.2
How many customers does each company have
(a) after one year?
(b) after two years?
(c) after many years?
Solution:
(a). The migration matrix is


.7 .4 .2
M = .3 .2 .6
0 .4 .2
and the initial state vector is
 
.2
x0 = .4 .
.4
Therefore,

  

.7 .4 .2 .2
0.38
x1 = M x0 = .3 .2 .6 .4 = 0.38 .
0 .4 .2 .4
0.24
So after one year, A and B have 380 000 customers each and C has 240 000 customers.
2
MAT 1302 – MARKOV CHAIN EXAMPLE, PROF. ALISTAIR SAVAGE
(b).

  

.7 .4 .2 .38
.466
x2 = M x1 = .3 .2 .6 .38 = .334 .
0 .4 .2 .24
.2
So after two years, A has 466 000 customers, B has 334 00 customers and C has 200 000
customers.
(c).


.61 .41 .42
M 2 = .27 .4 .3 
.12 .16 .28
2
Since all the entries of M are strictly greater than zero, M is a regular stochastic matrix
(we needed to check higher powers of M since M itself had a zero entry). Therefore xk
approaches the unique steady-state vector q as k → ∞. To find the steady state vector, we
solve
M q = q ⇐⇒ (M − I)q = 0
which amounts to row reducing
 
 


−.3
.4
.2 0
−3
4
2 0
1 0 − 10
0
3
.6 0  ∼  3 −8
6 0  ∼  0 1 −2 0  .
(M − I) 0 ∼  .3 −.8
0
.4 −.8 0
0
4 −8 0
0 0
0 0
The general solution is
10
x3
3
x2 = 2x3
x3 free
x1 =
Switching to vector notation gives


10/3
x = x3  2  .
1
Any choice of x3 6= 0 gives an eigenvector of M with eigenvalue 1. However, we want to
choose x3 so that the resulting vector is a probability vector (that is, its entries add to one).
So we pick x3 to be the reciprocal of the sum of the entries
−1 −1
10
19
3
x3 =
+2+1
=
= .
3
3
19
Therefore

 

10/3
10/19
3 
2  =  6/19  .
q=
19
1
3/19
10
Thus, in the long term A has 19
(1 000 000) ∼
= 526 316 customers, B has
3
∼
315 789 customers, and C has 19 (1, 000 000) = 157 895 customers.
6
(1 000 000)
19
∼
=