Introduction to
Management Science
with Spreadsheets
Stevenson and Ozgur
First Edition
Part 3 Probabilistic Decision Models
Chapter 12
Markov Analysis
McGraw-Hill/Irwin
Copyright © 2007 by The McGraw-Hill Companies, Inc. All rights reserved.
Learning Objectives
After completing this chapter, you should be able to:
1. Give examples of systems that may lend
themselves to be analyzed by a Markov model.
2. Explain the meaning of transition probabilities.
3. Describe the kinds of system behaviors that Markov
analysis pertains to.
4. Use a tree diagram to analyze system behavior.
5. Use matrix multiplication to analyze system
behavior.
6. Use an algebraic method to solve for steady-state
probabilities.
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–2
Learning Objectives (cont’d)
After completing this chapter, you should be able to:
7. Analyze absorbing states, namely accounts
receivable, using a Markov model.
8. List the assumptions of a Markov model.
9. Use Excel to solve various problems pertaining to a
Markov model.
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–3
Characteristics of a Markov System
1. It will operate or exist for a number of periods.
2. In each period, the system can assume one of a
number of states or conditions.
3. The states are both mutually exclusive and collectively
exhaustive.
4. System changes between states from period to period
can be described by transition probabilities, which
remain constant.
5. The probability of the system being in a given state in a
particular period depends only on its state in the
preceding period and the transition probabilities. It is
independent of all earlier periods.
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–4
Markov Analysis: Assumptions
• Markov Analysis Assumptions
–The probability that an item in the system either will
change from one state (e.g., Airport A) to another or
remain in its current state is a function of the transition
probabilities only.
–The transition probabilities remain constant.
–The system is a closed one; there will be no arrivals to
the system or exits from the system.
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–5
Table 12–1
Examples of Systems That May Be Described as Markov
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–6
Table 12–2
Transition Probabilities for Car Rental Example
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–7
System Behavior
• Both the long-term behavior and the short-term
behavior of a system are completely determined
by the system’s transition probabilities.
• Short-term behavior is solely dependent on the
system’s state in the current period and the
transition probabilities.
• The long-run proportions are referred to as the
steady-state proportions, or probabilities, of the
system.
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–8
Figure 12–1
Expected Proportion of Period 0 Rentals Returned to Airport A
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–9
Methods of System Behavior Analysis
• Tree Diagram
– A visual portrayal of a system’s transitions composed of a series
of branches, which represent the possible choices at each stage
(period) and the conditional probabilities of each choice being
selected.
• Matrix Multiplication
– Assumes that “current” state proportions are equal to the product
of the proportions in the preceding period multiplied by the matrix
of transition probabilities.
– Involves the multiplication of the “current” proportions, which is
referred to as a probability vector, by the transition matrix.
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–10
Methods of System Behavior Analysis
(cont’d)
• Algebraic Solution
–The basis for an algebraic solution is a set of
equations developed from the transition matrix.
–Because the states are mutually exclusive and
collectively exhaustive, the sum of the state
probabilities must be 1.00, and another equation can
be developedf rom this requirement.
–The result is a set of equations that can be used to
solve for the steady-state probabilities.
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–11
Figure 12–2
Tree Diagrams for the Car Rental Example for One Period
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–12
Figure 12–3
Two-Period Tree Diagrams for Car Rental Example
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–13
Table 12–3
Period-by-Period Proportions for the Rental Example, and the
Steady-State Proportions Based on Matrix Multiplications
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–14
Figure 12–4
Development of Algebraic Equations
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–15
Table 12–4
Transition Probabilities for the Machine Maintenance Example
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–16
Exhibit 12-1
Worksheet for the Markov Analysis of the Machine
Maintenance Problem
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–17
Figure 12–5
Decision Tree Representation of the Machine Maintenance
Problem: Initial State = Operation
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–18
Figure 12–6
Decision Tree Representation of the Machine Maintenance
Problem Initial State = Broken
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–19
Exhibit 12-2
Solver Parameters Specification Screen of the Machine
Maintenance Problem
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–20
Table 12–5
Transition Matrix for Examples 12-5, 12-6, and 12-7
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–21
Figure 12–7
Tree Diagram for Example 12-5, Starting from X
(Initial State = X)
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–22
Figure 12–8
Tree Diagram for Example 12-5, Starting from Y
(Initial State =Y)
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–23
Exhibit 12–3
Worksheet for the Markov Analysis of the Acorn University
Problem
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–24
Exhibit 12–4
Second Worksheet for the Markov Analysis of the Acorn
University Problem
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–25
Exhibit 12–5
Third Worksheet for the Markov Analysis and Steady-State
Probabilities of the Acorn University Problem
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–26
Exhibit 12–6
Parameters Specification Screen for the Acorn University
Problem
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–27
Cyclical, Transient, and Absorbing Systems
• Cyclical system
–A system that has a tendency to move from state to
state in a definite pattern or cycle.
• Transient system
–A system in which there is at least one state—the
transient state—where once a system leaves it, the
system will never return to it.
• Absorbing system
–A system that gravitates to one or more states—once
a member of a system enters an absorbing state, it
becomes trapped and can never exit that state.
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–28
Table 12–6
An Example of a Cyclical System
Table 12–7
An Example of System with a Transient State
Table 12–8
An Example of a System with Absorbing States
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–29
Figure 12–9
Probability Transition Diagrams for the Transition Matrices
Given in Tables 12-6, 12-7, and 12-8
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–30
Exhibit 12–7
Excel Worksheet for Example 12-10: The Acorn Hospital
Absorbing State Problem
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–31
Table 12–9
Answers to Example 12-10, Part 1 a through f
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–32
Exhibit 12–8
Worksheet for The Markov Analysis of Solved Problem 4
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–33
Exhibit 12–9
Solver Parameters Specification Screen for the Steady-State
Calculations for Solved Problem 4
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–34
Exhibit 12–10 Worksheet for the Steady-State Calculations of Solved
Problem 5
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–35
Exhibit 12–11
Solver Parameters Specification Screen for the Steady-State
Calculations for Solved Problem 5
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–36
Exhibit 12–12 Excel Worksheet for Solved Problem 6: Accounts
Receivable—Absorbing State Problem
Copyright © 2007 The McGraw-Hill Companies. All rights reserved.
McGraw-Hill/Irwin 12–37