Finite Mathematics
Chapter 9
Name ________________________________ Date ______________ Class ____________
Section 9-1 Properties Of Markov Chains
Goal: To solve problems using Markov chains
Definition: A transition matrix is a constant square matrix P of order n such that the entry in
the ith row and jth column indicates the probability of the system moving moving from the
ith state to the jth state on the next observation or trial.
Theorem: If P is the transition matrix and S0 is an initial state matrix for a Markov chain,
then the kth state matrix is given by Sk = S0 P k
1. A Markov process has two states, A and B. The probability of going from state A to state
B in one trial is 0.25, and the probability of going from state B to state A in one trial is 0.35.
a) Draw the transition diagram.
0.75
0.25
A
B
0.65
0.35
b) Find the transition matrix.
A
P=
B
A é0.75 0.25ù
B êë0.35 0.65ú
û
c) If S0 = [.4 .6] , find S1, and S2 .
S1 = S0 P
S2 = S1P
é0.75 0.25ù
= [0.51 0.49] ê
ú
ë0.35 0.65û
= [0.554 0.446]
é0.75 0.25ù
= [0.4 0.6] ê
ú
ë0.35 0.65û
= [0.51 0.49]
9-1
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
2. A Markov chain has three states, A, B, and C. The probability of going from state A to
state B in one trial is 0.8, the probability of going from state B to state A in one trial is 0.15,
the probability of going from state A to state C in one trial is 0.15, the probability of going
from state C to B in one trial is 0.2, the probability of going from state B to state C in one
trial is 0.15, and the probability of going from state C to state A in one trial is 0.35.
a) Draw the transition diagram.
0.15
0.8
0.05
A
B
0.15
0.65
0.2
0.35
C
0.2
0.45
b) Find the transition matrix.
A
B
C
A é0.05 0.8 0.15ù
P = B ê0.15 0.65 0.2 ú
ê
ú
C êë0.35 0.2 0.45ú
û
c) If S0 = [.2 .3 .5] , find S1 and S2 .
S1 = S0 P
é0.05 0.8 0.15ù
= [0.2 0.3 0.5] ê0.15 0.65 0.2 ú
ê
ú
êë0.35 0.2 0.45ú
û
= [0.23 0.455 0.315]
S2 = S1P
é0.05 0.8 0.15ù
= [0.23 0.455 0.315] ê0.15 0.65 0.2 ú
ê
ú
êë0.35 0.2 0.45ú
û
= [0.19 0.54275 0.26725]
9-2
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
3. Two popular brands of macaroni and cheese, Smack A Lot and Good Eats are sold at a
grocery. The buying habits of the customers are followed for several weeks during an
advertising campaign. It is found that 20% of those using Smack A Lot will switch to Good
Eats and that 40% of those who started by buying Good Eats will switch to Smack A Lot.
a) Draw a transition diagram.
0.8
Smack
A Lot
0.2
Good
Eats
0.6
0.4
b) Find the transition matrix.
S
P=
G
S é0.8 0.2ù
G êë0.4 0.6ú
û
c) If 50% of the customers used Smack A Lot at the start of the advertising
campaign, what percentage will be using Smack A Lot after 1 week? after 2 weeks?
S0 = [0.5 0.5]
S1 = S0 P
S2 = S1P
é0.8 0.2ù
= [0.5 0.5] ê
ú
ë0.4 0.6û
= [0.6 0.4]
é0.8 0.2ù
= [0.6 0.4] ê
ú
ë0.4 0.6û
= [0.64 0.36]
After 1 week, 60% of the customers will buy Smack A Lot.
After 2 weeks, 64% of the customers will buy Smack A Lot.
9-3
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
4. The buying patterns of customers who buy two brands of running shoes, Flying Feet and
Great Grips, are noted each year. It is found that 80% customers who buy Flying Feet one
year will buy Great Grips the next year and 20% will buy Flying Feet again the next year.
Also, it is found that 30% of the customers who buy Great Grips will buy Flying Feet the
next year and 70% will buy Great Grips again.
a) Draw a transition diagram.
0.8
Flying
Feet
0.2
Great
Grips
0.7
0.3
b) Find the transition matrix.
F
P=
G
F é0.2 0.8ù
G êë0.3 0.7 ú
û
c) If 60% of the customers bought Flying Feet at the start and 40% bought Great
Grips, what percentage will buy Flying Feet after the first year? Great Grips? After 2
years?
S0 = [0.6 0.4]
S1 = S0 P
S2 = S1P
é0.2 0.8ù
= [0.6 0.4] ê
ú
ë0.3 0.7 û
= [0.24 0.76]
é0.2 0.8ù
= [0.24 0.76] ê
ú
ë0.3 0.7 û
= [0.276 0.724]
After 1 year, 24% of the customers will buy Flying Feet.
After 2 years, 27.6% of the customers will buy Flying Feet.
9-4
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
5. If a voter votes Republican in one election, the probability that the voter will vote
Democratic in the next election is .15 and the probability the voter will vote for an
independent candidate is .05. If a voter votes Democratic in one election, the probability that
the voter will vote Republican in the next election is .05 and the probability that the voter
will vote for an independent candidate is .1. If a voter votes for an independent candidate in
one election, the probability that the voter will vote Republican in the next election is .3 and
the probability that the voter will vote Democratic in the next election is .5. Assume that
these are the only three choices available to the voter.
a) Draw the transition diagram.
0.15
0.8
R
D
0.05
0.05
0.85
0.1
0.3
0.5
I
0.2
b) Find the transition matrix.
R
D
I
R é 0.8 0.15 0.05ù
P = D ê0.05 0.85 0.1 ú
ê
ú
I ëê 0.3 0.5 0.2 ú
û
9-5
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
c) If 42% of the electorate votes Republican one year and 48% vote Democratic, find
the percentage of voters who vote Republican the next year. The percentage who vote
Democratic the next year. The percentage of voters who vote for an independent
candidate. After 2 years?
S0 = [0.42 0.48 0.1]
S1 = S0 P
é 0.8 0.15 0.05ù
= [0.42 0.48 0.1] ê0.05 0.85 0.1 ú
ê
ú
ëê 0.3 0.5 0.2 ú
û
= [0.39 0.521 0.089]
S2 = S1P
é 0.8 0.15 0.05ù
= [0.39 0.521 0.089] ê0.05 0.85 0.1 ú
ê
ú
êë 0.3 0.5 0.2 ú
û
= [0.36475 0.54585 0.0894]
After one year, 39% will vote Republican, 52.1% will vote Democratic, and 8.9%
will vote Independent.
After two years, 36.5% will vote Republican, 54.6% will vote Democratic, and 8.9%
will vote Independent.
9-6
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
Name ________________________________ Date ______________ Class ____________
Section 9-2 Regular Markov Chains
Goal: To solve problems using Markov chains
Definition: Stationary Matrix for a Markov Chain
The state matrix S = [s1 s2
transition matrix P if SP = S
where si ³ 0, i = 1,
, n, and s1 + s2 +
sn ] is a stationary matrix for a Markov chain with
+ sn = 1
1. A Markov process has two states, A and B. The probability of going from state A to state
B in one trial is 0.25, and the probability of going from state B to state A in one trial is 0.35.
Find the stationary matrix that describes the long-run behavior of this process.
[s1
é0.75 0.25ù
s2 ] ê
ú = [s1 s2 ]
ë0.35 0.65û
ì 0.75s1 + 0.35s2 = s1
ï
í 0.25s1 + 0.65s2 = s2 Þ
ï
s1 + s2 = 1
î
ì - 0.25s1 + 0.35s2 = 0
ï
í 0.25s1 - 0.35s2 = 0
ï
s1 = 1 - s2
î
- 0.25(1 - s2 ) + 0.35s2 = 0
- 0.25 + 0.25s2 + 0.35s2 = 0
0.6 s2 = 0.25
s1 = 1 - s2
s1 = 1 - 0.4167
s1 = 0.5833
s2 = 0.4167
Therefore, [s1 s2 ] = [0.5833 0.4167]
9-7
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
2. A Markov chain has three states, A, B, and C. The probability of going from state A to
state B in one trial is 0.8, the probability of going from state B to state A in one trial is 0.15,
the probability of going from state A to state C in one trial is 0.15, the probability of going
from state C to state B in one trial is 0.2, the probability of going from state B to state C in
one trial is 0.15, and the probability of going from state C to state A in one trial is 0.35. Find
the stationary matrix that describes the long-run behavior of this process.
[s1
s2
é0.05 0.8 0.15ù
s3 ] ê0.15 0.65 0.2 ú = [ s1
ê
ú
êë0.35 0.2 0.45ú
û
ì 0.05s1 + 0.15s2 + 0.35s3 = s1
ï 0.8s + 0.65s + 0.2 s = s
ï
1
2
3
2
Þ
í
ï 0.15s1 + 0.2 s2 + 0.45s3 = s3
ïî
s1 + s2 + s3 = 1
s2
s3 ]
ì - 0.95s1 + 0.15s2 + 0.35s3 = 0
ï 0.8s - 0.35s + 0.2 s = 0
ï
1
2
3
í
ï 0.15s1 + 0.2 s2 - 0.55s3 = 0
ïî
s1 + s2 + s3 = 1
Using Gauss – Jordan elimination to solve this system of four equations with three
variables, we obtain: [s1 s2 s3 ] = [0.1826 0.5629 0.2545]
3. Two popular brands of macaroni and cheese, Smack A Lot and Good Eats are sold at a
grocery. The buying habits of the customers are followed for several weeks during an
advertising campaign. It is found that 20% of those using Smack A Lot will switch to Good
Eats and that 40% of those who started by buying Good Eats will switch to Smack A Lot. If
this trend holds up what percentage of customers will use Smack A Lot and what percentage
of customers will use Good Eats in the long run?
[s1
é0.8 0.2ù
s2 ] ê
ú = [ s1
ë0.4 0.6û
ì 0.8s1 + 0.4s2 = s1
ï
í 0.2s1 + 0.6s2 = s2 Þ
ï
s1 + s2 = 1
î
s2 ]
ì - 0.2s1 + 0.4s2 = 0
ï
í 0.2s1 - 0.4s2 = 0
ï
s1 = 1 - s2
î
- 0.2(1 - s2 ) + 0.4 s2 = 0
s1 = 1 - s2
- 0.2 + 0.2 s2 + 0.4 s2 = 0
s1 = 1 - 0.3333
0.6 s2 = 0.2
s1 = 0.6667
s2 = 0.3333
Therefore, [s1 s2 ] = [0.6667 0.3333] . So 67% of the customers will buy Smack A
Lot and 33% will buy Good Eats.
9-8
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
4. The buying patterns of customers who buy two brands of running shoes, Flying Feet and
Great Grips, are noted each year. It is found that 80% customers who buy Flying Feet one
year will buy Great Grips the next year and 20% will buy Flying Feet again the next year.
Also, it is found that 30% of the customers who buy Great Grips will buy Flying Feet the
next year and 70% will buy Great Grips again. In the long run what percentage of the
customers will be using Flying Feet? Great Grips?
[s1
é0.2 0.8ù
s2 ] ê
ú = [ s1 s2 ]
ë0.3 0.7 û
ì 0.2s1 + 0.3s2 = s1
ï
í 0.8s1 + 0.7 s2 = s2 Þ
ï
s1 + s2 = 1
î
ì - 0.8s1 + 0.3s2 = 0
ï
í 0.8s1 - 0.3s2 = 0
ï
s1 = 1 - s2
î
- 0.8(1 - s2 ) + 0.3s2 = 0
s1 = 1 - s2
- 0.8 + 0.8s2 + 0.3s2 = 0
s1 = 1 - 0.7273
1.1s2 = 0.8
s1 = 0.2727
s2 = 0.7273
Therefore, [s1 s2 ] = [0.2727 0.7273] . So 27% of the customers will buy Flying
Feet and 73% will buy Good Grips.
9-9
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
5. If a voter votes Republican in one election, the probability that the voter will vote
Democratic in the next election is 0.15 and the probability the voter will vote for an
independent candidate is 0.05. If a voter votes Democratic in one election, the probability
that the voter will vote Republican in the next election is 0.05 and the probability that the
voter will vote for an independent candidate is 0.1. If a voter votes for an independent
candidate in one election, the probability that the voter will vote Republican in the next
election is 0.3 and the probability that the voter will vote Democratic in the next election is
0.5. Assume that these are the only three choices available to the voter. If this trend holds
up, what percentage of the voters will vote Republican in the long run? Democratic?
Independent?
[s1
s2
é 0.8 0.15 0.05ù
s3 ] ê0.05 0.85 0.1 ú = [ s1
ê
ú
ëê 0.3 0.5 0.2 ú
û
s2
s3 ]
ì 0.8s1 + 0.05s2 + 0.3s3 = s1
ì - 0.2 s1 + 0.05s2 + 0.3s3 = 0
ï 0.15s + 0.85s + 0.5s = s
ï 0.15s - 0.15s + 0.5s = 0
ï
ï
1
2
3
2
1
2
3
Þ í
í
ï 0.05s1 + 0.1s2 + 0.2 s3 = s3
ï 0.05s1 + 0.1s2 - 0.8s3 = 0
ïî
ïî
s1 + s2 + s3 = 1
s1 + s2 + s3 = 1
Using Gauss–Jordan elimination to solve this system of four equations with three
variables, we obtain: [s1 s2 s3 ] = [0.2947 0.6105 0.0948]. So 29.5% will vote
Republican, 61.0% will vote Democratic, and 9.5% will vote Independent.
9-10
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
Name ________________________________ Date ______________ Class ____________
Section 9-3 Absorbing Markov Chains
Goal: To solve application problems using Markov chains
é I 0ù
ú
ëFR 0û
The limiting matrix: P = ê
-1
where F = ( I - Q ) (F is called the fundamental matrix for
P).
A computer game has two levels. Level one is called Flying Low and level two is called
Flying High. To win the game the player must successfully complete Flying Low before
getting to Flying High. On their first attempt at playing the game, 30% of players are able to
successfully navigate Flying Low and move on to Flying High, 50% make an error and are
eliminated from the game, and the rest continue to play at the Flying Low level. After
making it to Flying High, 10% of the players successfully navigate Flying High and win the
game, 15% make a fatal error and are eliminated from the game, and the rest continue to play
at the Flying High level.
a) Draw a transition diagram.
0.3
0.2
FLYING
LOW
(FL)
FLYING
HIGH
(FH)
0.5
0.15
0.75
0.1
END (L)
WIN (W)
1
1
9-11
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
b) Find the transition matrix P.
FL
FH
L
W
FL é0.2 0.3 0.5 0 ù
FH ê 0 0.75 0.15 0.1ú
ê
ú
P=
L ê0
0
1
0ú
ê
ú
W ë0
0
0
1û
c) Write the transition matrix P in a standard form.
L
W
FL
FH
L é 1
0
0
0 ù
ê
W
0
1
0
0 ú
ê
ú
P=
FL ê 0.5 0 0.2 0.3 ú
ê
ú
FH ë0.15 0.1 0 0.75û
d) Subdivide matrix P and then find matrix R and matrix Q.
L
W
FL
FH
L é 1
0
0
0 ù
ê
W
0
1
0
0 ú
ê
ú
P=
FL ê 0.5 0 0.2 0.3 ú
ê
ú
FH ë0.15 0.1 0 0.75û
0ù
é 0.5
R=ê
ú
ë0.15 0.1û
-1
e) Find matrix F. (Remember F = ( I - Q )
).
é1 0ù é0.2 0.3 ù é0.8 - 0.3ù
I- Q=ê
ú- ê
ú= ê
ú
ë0 1û ë 0 0.75û ë 0 0.25 û
D = (0.8)(0.25) - (0)(- 0.3) = 0.2 - 0 = 0.2
é0.25 0.3ù é1.25 1.5ù
F = ( I - Q) - 1 = 0.2 ê
=ê
0.8ú
4ú
ë 0
û ë 0
û
9-12
Copyright © 2015 Pearson Education, Inc.
é0.2 0.3 ù
Q=ê
ú
ë 0 0.75û
Finite Mathematics
Chapter 9
f) Find FR.
0 ù é0.85 0.15ù
é1.25 1.5ùé 0.5
FR = ê
=ê
úê
ú
4 ûë0.15 0.1ú
ë 0
û ë 0.6 0.4 û
g) Write the limiting matrix P .
0
0
é 1
ê 0
1
0
P=ê
ê0.85 0.15 0
ê
ë 0.6 0.4 0
0ù
0ú
ú
0ú
ú
0û
h) In the long run, what percentage of players will lose before getting to Flying High?
In the long run, 85% of players will lose before getting to Flying High.
i) In the long run, what percentage of players will lose after getting to Flying High?
In the long run, 60% of players will lose after getting to Flying High.
j) In the long run, what percentage of players that make it to Flying High will win the game?
In the long run, 40% of players that make it to Flying High will win.
k) What is the average number of trials that a player spends in Flying Low?
The average number of trials that a person will spend in Flying Low is
1.25 +1.5 = 2.75 trials.
l) What is the average number of trials that a player spends in Flying High?
The average number of trials that a person will spend in Flying High is
0 + 4 = 4 trials.
9-13
Copyright © 2015 Pearson Education, Inc.
Finite Mathematics
Chapter 9
9-14
Copyright © 2015 Pearson Education, Inc.