Chapter 16
Markov Analysis
To accompany Quantitative Analysis
for Management, 8e
by Render/Stair/Hanna
16-1
© 2003 by Prentice Hall, Inc.
Upper Saddle River, NJ 07458
Learning Objectives
Students will be able to
• Determine future states or
conditions using Markov analysis.
• Compute long-term or steadystate conditions using only the
matrix of transition.
• Understand the use of absorbing
state analysis in predicting future
conditions.
To accompany Quantitative Analysis
for Management, 8e
by Render/Stair/Hanna
16-2
© 2003 by Prentice Hall, Inc.
Upper Saddle River, NJ 07458
Chapter Outline
16.1 Introduction
16.2 States and State Probabilities
16.3 Matrix of Transition Probabilities
16.4 Predicting Future Market Share
16.5 Markov Analysis of Machine
Operations
16.6 Equilibrium Conditions
16.7 Absorbing States and the
Fundamental Matrix: Accounts
Receivable Applications
To accompany Quantitative Analysis
for Management, 8e
by Render/Stair/Hanna
16-3
© 2003 by Prentice Hall, Inc.
Upper Saddle River, NJ 07458
Assumptions of
Markov Analysis
1. A finite number of possible states.
2. Probability of change remains the
same over time.
3. Future state predictable from current
state.
4. Size of system remains the same.
5. States collectively exhaustive.
6. States mutually exclusive.
To accompany Quantitative Analysis
for Management, 8e
by Render/Stair/Hanna
16-4
© 2003 by Prentice Hall, Inc.
Upper Saddle River, NJ 07458
The Markov Process
  P  
Current
State
To accompany Quantitative Analysis
for Management, 8e
by Render/Stair/Hanna
Matrix of
Transition
16-5
New
State
© 2003 by Prentice Hall, Inc.
Upper Saddle River, NJ 07458
Markov Process
Equations
(i) = State probabilities = [1 2 3 … n]
Matrix of
= P=
transition
probabilities
P11 P12 P13...P1n
P21 P22 P23...P2n
Pm1 ...
Pmn
(i+1) = (i)P
To accompany Quantitative Analysis
for Management, 8e
by Render/Stair/Hanna
16-6
© 2003 by Prentice Hall, Inc.
Upper Saddle River, NJ 07458
Predicting Future
States
π  State probabilit ies
 (1)  .4 .3 .3
.8 .1 .1
P  .1 .7 .2
.2 .2 .6
 (2)   (1) P
.8 .1 .1
 (2)  .4 .3 .3.1 .7 .2
.2 .2 .6
 (2)   .4 * .8  .3 * .1 * .3 * .2 
.4 * .1  .3 * .7 * .3 * .2
.4 * .1  .3 * .2  .3 * .6 
 (2)  0.41 0.31 0.28
To accompany Quantitative Analysis
for Management, 8e
by Render/Stair/Hanna
16-7
© 2003 by Prentice Hall, Inc.
Upper Saddle River, NJ 07458
Machine Example:
Periods to Reach
Equilibrium
Period
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
To accompany Quantitative Analysis
for Management, 8e
by Render/Stair/Hanna
State 1
1.0
.8
.66
.562
.4934
.44538
.411766
.388236
.371765
.360235
.352165
.346515
.342560
.339792
.337854
16-8
State 2
0.0
.2
.34
.438
.5066
.55462
.588234
.611763
.628234
.639754
.647834
.653484
.657439
.660207
.662145
© 2003 by Prentice Hall, Inc.
Upper Saddle River, NJ 07458
Equilibrium Equations
 (i  1)   (i ) P
Assume :  (i)   1
 p11
 2 , P  
 p 21
p12 
p 22 
Then :
 1
 2    1 P11   2 P21  1 P12   2 P22 
or :
 1   1 P11   2 P21,
 2   1 P12   2 P22
Therefore :
 2 p 21
1 
1  p11
To accompany Quantitative Analysis
for Management, 8e
by Render/Stair/Hanna
 1 p12
and  2 
1  p 22
16-9
© 2003 by Prentice Hall, Inc.
Upper Saddle River, NJ 07458
Markov Process
Fundamental Matrix
I
Let P  
A
0

B
Where I = Identify matrix, and 0 = Null
matrix
Then
F  I  B
1
And FA indicates the probability that an
amount in one of the non-absorbing states
will end up in one of the absorbing states.
To accompany Quantitative Analysis
for Management, 8e
by Render/Stair/Hanna
16-10
© 2003 by Prentice Hall, Inc.
Upper Saddle River, NJ 07458