Finite Math B – Mrs. Leahy
10.1 Basic Properties of Markov Chains
Suppose two local television news networks go through a “ratings week” at the end of each month. They try to
offer special segments that will draw viewers from the other station. During each period, Network A wins over
20% of Network B’s viewers, but loses 10% of its viewers to Network B.
Draw a graph to represent the movement between stations. This is called a TRANSITION DIAGRAM.
Draw a circle for each network.
Use arrows to represent the percentage
of viewers who change networks.
Use arrows to represent the percentage
of viewers who STAY the same.
Set up a TRANSITION MATRIX to represent the diagram.
Label the rows and columns “A” and “B”, one for
each.
The position (row, column) represents the Probability
of moving from the “row state” to the “column
state.” Or, in this problem, the probability of moving
from the row network to the column network.
PROPERTIES OF TRANSITION MATRIX:
1. It is a square matrix since all possible states must be used as rows and as columns.
2. All entries are between 0 and 1, inclusive, since all entries represent probabilities.
3. Each row must have a sum of 1, since the numbers represent the probabilities of changing from the
state on the left to one of the states across the top.
Example 1: Which of the following could be a transition matrix (by definition)? Sketch a transition diagram
for any transition matrix.
0.7 0.3
a) 

 0.5 0.5
b)
1
 3
 0
2 
3
1 
 0.5 0.3 0.2 


c)  0 0.4 0.6 
 0.1 0.1 0.8 
(example 1 continued)
d)
 12
1

 2 3
1
4
0
1
5

0 
1 
5
1
4
e)
Example 2: Write the transition diagram as a transition matrix.
a)
b)
 12
2
 5
 1 3
3
3
1
8
5
3

0 
1 
2
1
8
Example 3: The British scientist Sir Francis Galton studied inheritance by
looking at distributions of the heights of parents and children. In 1886 he
published data from a large sample of parents and their adult children
showing the relation between their heights. The following matrix is based
on his data.
CHILD
Tall
Med
Short
Tall .53
.32 .15

PARENT Med .30 .34 .36   T
Short .15 .32 .53
a) What percentage of “tall” parents had “tall” children?
b) What percentage of “short” parents had “medium” height children?
CHILD'S CHILD
Tall
Use matrix multiplication, or a graphing calculator to find
T 2 . This would represent another generation of children.
c) What proportion of “tall” parents had “tall”
grandchildren?
Tall  ___
PARENT Med  ___
Short  ___
Med
___
___
___
Short
___ 
___   T 2
___ 
d) What proportion of “short” parents had “short” grandchildren?
10 generations later
Use matrix multiplication or a graphing calculator to find
T 10 . This would represent 10 generations of children
later.
e) What do you think these numbers represent?
In the “long run,” what proportion of people do we
expect to be:
Tall
Tall  ___
PARENT Med  ___
Short  ___
Med
___
___
___
Short
___ 
___   T 10
___ 
Tall? ______ Medium Height? ______ Short? _______
2
Example 4: Find the next two powers of each transition matrix ( A
.6
.3
a) A  
.4 
.7 
, A3 ).
Then find the indicated probability.
Find the probability that state 1 changes to state 2 after 3 repetitions.
(Example 4 continued)
.3

b) A  .5

.7
.3
.2
.15
.4 
.3 
.15
Find the probability that state 3 changes to state 1 after 1 time period.
Find the probability that state 2 changes to state 1 after 3 time periods.
Example 5: Find A5
 .1
 .5

.25
A

c)

 .2
.75
.1
.1
.1
.5
0
0
.25
.2
.2
.2
.2
.2
0
0
.25
.6 
0 
.1

.2 
0 
Find the probability that state 1 changes to state 5 after 1 repetition experiment.
Find the probability that state 1 changes to state 5 after 5 repetitions of the experiment.
Find the probability that state 3 changes to state 4 after 5 repetitions of the experiment.
Homework: pg 564-568 #10-18all, 20-26even, 28
10.1B Basic Properties of Markov Chains - Continued
Suppose two local television news networks go through a “ratings week” at the end of each month. They try to
offer special segments that will draw viewers from the other station. During each period, Network A wins over
20% of Network B’s viewers, but loses 10% of its viewers to Network B.
Suppose Network A currently holds 60% of the “market share” of viewers. At the end of six months, what
percentage of the market share will it hold?
A PROBABILITY VECTOR is a matrix of only one row that usually represents the initial state of the population…
sometimes we call this the “market share.”
Probability Vector for our television networks:
We can PREDICT the probability vector/market share
after a period of time using matrix multiplication.
Suppose a Markov Chain has an initial probability vector:
X 0  i1
Example 1:
i2
i3
...
in  and a transition matrix P.
Then the probability vector after n repetitions of the
experiment is:
Initial Probability Vector for news example:
X 0  Pn
Transition Matrix for news network:
“At the end of six months, what percentage of the market share will it hold?”
Example 2: If the current population has 20% tall, 40% medium, and 40% short
people, what percentage of the population will be tall in 3 generations?
CHILD
Tall
Tall .53
Med
Short
.32 .15

PARENT Med .30 .34 .36   T
Short .15 .32 .53
Example 3: A study is done to classify people by income as “Lower-class,”
“Middle-class,” and “Upper-class.” For convenience we will call these “state
1, 2, and 3,” respectively. It is determined that, for example, the probability
of children of state 1 (lower-class income) to also be state 1 as 0.65. The
other probabilities are given in the following transition matrix.
After a census, it is determine that currently 21% of the population is state 1,
68% of the population is state 2, and 11% of the population is state 3.
What percentage of the population will be in state 1 in 3 generations?
Homework: p564-568 #1-9, 33abc, 37ab, 38abcd, 40abcd
1
1  0.65
2  0.15
3  0.12
2
3
0.28
0.07 
0.18 
0.52 
0.67
0.36