IMPORTANCE SAMPLING SIMULATION OF SAN-BASED REWARD MODELS
Walter Douglas Obal II, M. S.
The University of Arizona, 1993
Director: William H. Sanders
Importance sampling is a potentially powerful technique for eciently estimating the
probabilities of rare events in discrete event systems. The work presented here makes three
contributions to importance sampling research. First, a convenient mechanism for specifying complex importance sampling heuristics is presented. Second, a method for applying importance sampling to stochastic activity networks at the network level is developed.
Third, the specication mechanism and execution method are implemented as an integrated
tool that can automatically generate an importance sampling simulation program from the
specication of a heuristic and a stochastic activity network. The eectiveness of the new
software tool is illustrated through its application to a machine-repairman model with delayed group repair. Orders of magnitude reduction in the CPU time required to obtain a
specied relative accuracy were achieved.
IMPORTANCE SAMPLING SIMULATION OF SAN-BASED
REWARD MODELS
by
Walter Douglas Obal II
A Thesis Submitted to the Faculty of the
DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING
In Partial Fulllment of the Requirements
For the Degree of
MASTER OF SCIENCE
WITH A MAJOR IN ELECTRICAL ENGINEERING
In the Graduate College
THE UNIVERSITY OF ARIZONA
1993
2
STATEMENT BY AUTHOR
This thesis has been submitted in partial fulllment of requirements for an advanced
degree at The University of Arizona and is deposited in the University Library to be made
available to borrowers under rules of the Library.
Brief quotations from this thesis are allowable without special permission, provided that
accurate acknowledgment of source is made. Requests for permission for extended quotation
from or reproduction of this manuscript in whole or in part may be granted by the head of
the major department or the Dean of the Graduate College when in his or her judgment
the proposed use of the material is in the interests of scholarship. In all other instances,
however, permission must be obtained from the author.
SIGNED:
APPROVAL BY THESIS DIRECTOR
This thesis has been approved on the date shown below:
William H. Sanders
Assistant Professor of
Electrical and Computer Engineering
Date
3
ACKNOWLEDGMENTS
Many people in many ways helped me to complete this work. First of all, I would
like to thank Bill Sanders, my thesis advisor, for his condence in me and his guidance
throughout this project. Also, I want to thank the other members of my thesis committee,
Professors Pamela Delaney and Bernard Zeigler, for reviewing this thesis. In addition,
Professor Delaney earned my gratitude for her patience with me during my introduction
to probability theory. Finally, Julie Darnell, Latha Kant, Aad van Moorsel, and Akber
Qureshi, as well as my family, deserve special thanks for their support and encouragement,
particularly during the dicult periods of this work.
4
To my parents
5
TABLE OF CONTENTS
List of Figures :
List of Tables : :
Abstract : : : : :
1. Introduction
:::::::::::::::::::::::::::::::::::::
6
:::::::::::::::::::::::::::::::::::::
7
:::::::::::::::::::::::::::::::::::::
8
:::::::::::::::::::::::::::::::::::::
9
10
12
1.1. Background : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
1.2. Research Objectives : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
2. SAN-Based Framework for Importance Sampling : : : : : : : : : : : : : 14
2.1. Introduction to Importance Sampling
2.2. SAN-Based Reward Models : : : : : :
2.2.1. SANs : : : : : : : : : : : : : :
2.2.2. Reward Variables : : : : : : :
2.3. Importance Sampling Governor : : :
2.4. Execution of Governed SANs : : : : :
2.4.1. Example Execution : : : : : :
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
14
15
15
20
21
25
27
3. Sample Path Probabilities and the Likelihood Ratio : : : : : : : : : : : 30
3.1.
3.2.
3.3.
3.4.
Sample Path Probabilities :
Likelihood Ratio : : : : : : :
Estimating Reward Variables
Algorithm : : : : : : : : : :
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
30
33
34
36
4. Implementation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 38
4.1. UltraSAN : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
4.2. Importance Sampling Facility : : : : : : : : : : : : : : : : : : : : : : : : : :
38
40
5. Examples and Results : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 45
5.1. Machine-Repairman Models : : : : : : : : : : : : : : : : :
5.1.1. Results for Markov Machine-Repairman Model : : :
5.1.2. Non-Markov Machine-Repairman Model : : : : : :
5.1.3. Results for non-Markov Machine-Repairman Model
5.2. M/M/1/K Queue : : : : : : : : : : : : : : : : : : : : : : :
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
45
45
50
54
57
6. Conclusions : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 61
6.1. Future Work : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
62
6
LIST OF FIGURES
2.1. SAN Model of Delayed Repair System : : : : : : : : : : : : : : : : : : : : :
2.2. Governor for Modied Failure Biasing : : : : : : : : : : : : : : : : : : : : :
2.3. Possible Sequence of Events in the Machine-Repairman Model : : : : : : :
:
:
:
:
:
5.1. Governor for MFB with Approximate Forcing :
:
:
:
:
:
:
5.2. Non-Markov M-R Model with Delayed Repair Policy :
:
:
:
:
:
:
:
5.3. Governor for Modied Failure Biasing with Approximate Forcing :
5.4. SAN Model of M/M/1/K Queue : : : : : : : : : : : : : : : : : : :
4.1.
4.2.
4.3.
4.4.
4.5.
Organization of UltraSAN : : : : : : : : : :
Importance Sampling Governor Editor : :
Governor State Editor : : : : : : : : : : : :
Activity Bias Editor : : : : : : : : : : : : :
Governor State Transition Function Editor
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
18
24
28
39
41
41
42
42
47
51
54
57
7
LIST OF TABLES
Activity Time Distributions : : : : : : : : : : : : : : : : : : : : : : : : : :
Input Gate Predicate and Function for Markov Machine-Repairman Model
Governor States : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
Governor State Transition Function : : : : : : : : : : : : : : : : : : : : : :
18
19
25
26
5.1. Unreliability of Markov M-R Model Using MFB : : : : : : : : : : : : : : :
5.2. Activity Time Distributions for the Sti M-R Model : : : : : : : : : : : : :
5.3. Unreliability of Sti Markov M-R Model Using MFB : : : : : : : : : : : :
5.4. State Specication for Governor for MFB with Approximate Forcing : : : :
5.5. State Transition Function for Governor for MFB with Approximate Forcing
5.6. Unreliability of Sti Markov M-R Model Using MFB and Approx. Forcing
5.7. Activity Time Distributions for non-Markov M-R Model : : : : : : : : : :
5.8. Input Gate Predicates and Functions for non-Markov M-R Model : : : : :
5.9. Activation and Reactivation Predicates for non-Markov M-R Model : : : :
5.10. State Specication for Governor for MFB with Approximate Forcing : : : :
5.11. State Transition Function for Governor for MFB with Approximate Forcing
5.12. Importance Sampling Strategies for the non-Markov M-R Model : : : : : :
5.13. Unreliability in [0; 10] of non-Markov M-R Model : : : : : : : : : : : : : :
5.14. Results for Modied Non-Markov M-R Model : : : : : : : : : : : : : : : :
5.15. Activity Time Distributions for M/M/1/K Queue : : : : : : : : : : : : : :
5.16. Input Gate Predicate and Function for M/M/1/K Queue : : : : : : : : : :
5.17. Estimates for Probability Queue Empties in One Time Unit : : : : : : : :
5.18. Biased Activity Time Distributions for M/M/1/K Queue : : : : : : : : : :
5.19. Estimates for Probability Queue Empties in Two Time Units : : : : : : : :
46
46
46
48
49
50
52
52
53
54
55
56
56
57
57
58
58
59
59
2.1.
2.2.
2.3.
2.4.
8
ABSTRACT
Importance sampling is a potentially powerful technique for eciently estimating the
probabilities of rare events in discrete event systems. The work presented here makes three
contributions to importance sampling research. First, a convenient mechanism for specifying complex importance sampling heuristics is presented. Second, a method for applying importance sampling to stochastic activity networks at the network level is developed.
Third, the specication mechanism and execution method are implemented as an integrated
tool that can automatically generate an importance sampling simulation program from the
specication of a heuristic and a stochastic activity network. The eectiveness of the new
software tool is illustrated through its application to a machine-repairman model with delayed group repair. Orders of magnitude reduction in the CPU time required to obtain a
specied relative accuracy were achieved.
9
CHAPTER 1
Introduction
Discrete event model-based evaluation is playing an increasingly prominent role in the
life cycle of many modern systems. In the design phase, the size and complexity of modern
systems often preclude prototyping and testing because of the high costs involved. Modeling
is a relatively inexpensive method whereby designers can investigate a large number of
alternative designs. For existing systems, model-based evaluation is an attractive technique
for discerning the impact of proposed modications. The primary advantage of modeling
over testing is, in this case, the fact that model-based evaluation avoids service disruptions
that may be necessary for testing.
Once a system model has been constructed, it must be solved for performance/dependability measures of interest. If the model corresponds to a Markov chain in discrete or
continuous time, and the state space is not too large relative to available computer resources,
analytic techniques may be applied to obtain an exact solution. However, many times the
model is not analytically tractable. In that case, discrete event simulation [15] is usually employed to estimate the desired performance/dependability measures. Although simulation
can provide good estimates of a wide variety of such measures, traditional techniques take
too long to collect a statistically signicant sample when confronted with measures based
on rare events. Fortunately, a technique known as importance sampling [9] is an eective
tool for increasing the eciency of rare event simulations. Importance sampling allows one
to change the probability measure governing the evolution of the model so that rare events
10
occur more frequently. To compensate, the observations taken from the altered model are
weighted by a factor called the likelihood ratio. Informally, the likelihood ratio is the ratio
of the probability of obtaining the observation under the original probability measure to
the probability of obtaining the observation under the new measure.
1.1 Background
Importance sampling was developed in the context of Monte Carlo calculations. An
early application of the importance sampling method can be found in Kahn [12], which
looks at neutron attenuation problems. The survey by Kahn and Marshall [13] provides a
lucid introduction to the topic, and Hammersley and Handscomb [9] is widely referenced.
Another source is Kleijnen [14], who discusses an attempt at applying importance sampling
to the general problem of discrete event simulation. Much of this early work noted that
importance sampling, while potentially very powerful, can be very dicult to apply in
general. The main problem is deciding how to change the probability measure governing the
model to achieve a large reduction in the variance of the estimator of the performance/dependability measure of interest. A poor choice of the change of measure can result in
an increase in the variance of the estimator. Calculating the optimal change of measure
requires an analytical expression for the probability measure governing the model as well
as knowledge of the quantity to be estimated. The probability measure governing the
evolution of a discrete event system is rarely available in closed form, and if the value of the
quantity being estimated was known beforehand, there would be no need for simulation.
So, nding the optimal change of measure for the model types we are considering is a very
hard problem, and remains an open research question.
Despite the diculty of nding a technique that works well in general, the power of importance sampling to provide orders of magnitude reductions in the variance of an estimator
11
motivated researchers to look for strategies that would work well within certain classes of
models. For example, Lewis and Bohm [16] applied importance sampling to Markov unreliability models. They developed techniques called failure biasing and forced transition and
obtained signicant variance reductions. The failure biasing technique increases the total
probability of a failure transition from a state, thus making system failure a more likely
event. When the time horizon is short relative to the mean time to failure of components in
the system, forcing is employed. The forcing technique samples the failure time of a component from a conditional distribution. The distribution is modied such that the failure
will occur before the time horizon is reached.
Goyal, Shahabuddin, Heidelberger, Nicola, and Glynn (see [8], and the references therein)
applied these techniques to the class of Markov dependability models representable in the
language of the SAVE [7] software package. The work in [8] also considers the use of importance sampling for estimation of steady state measures. Steady state measures are estimated
in [8] using regenerative simulation and ratio estimators, which lead to the measure specic
dynamic importance sampling (MSDIS) technique, where the numerator and denominator
are estimated using dierent probability measures. Carrasco [1] has proposed a modication of the basic failure biasing approach that utilizes estimator decomposition techniques
to estimate the Laplace transforms of unreliability and unavailability. The transform is then
numerically inverted to obtain the desired answer. Juneja and Shahabuddin [11] proved that
straightforward application of failure biasing can lead to estimators with innite variance
in models with delayed group repair, and suggested an eective alternative strategy.
Results from the theory of large deviations have been used to choose the \asymptotically
optimal exponential change of measure" in the context of slow Markov walks. The seminal
work was done by Cottrell, Fort, and Malgouyres [2], who applied their technique to an
ALOHA system. Large deviations theory was then applied by Parekh and Walrand [24]
12
to the problem of estimating the probabilities of excessive backlogs in stable open Jackson networks of queues. The authors found that classical large deviations results did not
directly apply to queuing networks, but were able to formulate a useful heuristic. Shahabuddin [33] used large deviations results to derive \asymptotically optimal exponential changes
of measure" for systems that achieve high dependability through massive redundancy.
Some work has also been done on importance sampling for more general discrete event
systems. Glynn and Iglehart [6] have examined the application of importance sampling to
the broad class of discrete event simulations representable by a generalized semi-Markov
process (GSMP) [5], but they did not consider any specic changes of measure. Nicola,
Nakayama, Heidelberger, and Goyal [20] applied the failure biasing and forcing techniques
to systems with general failure and repair distributions. The systems considered in [20] have
the same structure as SAVE type models, only the holding time distributions are altered.
Van Moorsel, et al., [35] have proposed an interesting alternative to importance sampling
for rare event simulation.
1.2 Research Objectives
The surveyed research demonstrates that importance sampling is most eective when
tailored to the application at hand. However, for complex applications such as dependability or performability modeling it can be very dicult to nd good changes of measure.
Furthermore, small dierences in model characteristics, such as a change in the repair policy
in a dependability model, can have a large impact on the eectiveness of a particular change
of measure. Therefore, before utilization of importance sampling in discrete event modelbased evaluation problems can become widespread, much research is required to identify
eective changes of measure for various types of systems, and to illuminate the conditions
under which these changes of measure will fail. To meet this need, researchers must have
13
tools that will expedite the process of experimenting with dierent changes of measure on
a wide variety of systems, thereby improving their understanding of the power and pitfalls
of the method.
The primary objective of this work is the development of a software tool suitable for
experimenting with the application of importance sampling to discrete event simulation.
The milestones en route to the primary objective are:
1. A formal framework for specifying importance sampling heuristics in terms of stochastic activity networks.
2. A method for applying importance sampling to stochastic activity networks at the
network level, avoiding the generation of an underlying stochastic process.
3. A user-interface to the framework for specifying importance sampling heuristics.
4. Software for simulating stochastic activity networks under a specied importance sampling heuristic.
5. Integration of the user-interface and simulation engine into an importance sampling
tool capable of automatically generating an importance sampling simulation from a
given heuristic and stochastic activity network model.
6. Demonstration of the utility of the new tool.
The remainder of this thesis is organized as follows. Chapter 2 details a formal framework for specifying heuristics for importance sampling in terms of stochastic activity networks. A method for simulating a stochastic activity network under an importance sampling
heuristic is developed in Chapter 3. Chapter 4 introduces the user-interface for the new tool.
Examples demonstrating the utility of the methods and software are given in Chapter 5.
Finally, Chapter 6 summarizes the contributions of this thesis and suggests some possible
directions for future research.
14
CHAPTER 2
SAN-Based Framework for Importance Sampling
2.1 Introduction to Importance Sampling
The essence of importance sampling is the following transformation. Let Y be a random
variable [23] dened on the probability space (
; F ; P ). If P 0 is another probability measure
on (
; F ), and is absolutely continuous with respect to P (i.e., P (A) 6= 0 implies P 0 (A) 6= 0
for all A 2 ), then
EP [Y ] =
Z
Y dP =
Z
dP dP 0 = E 0 [Y L];
Y dP
P
0
where L = dP=dP 0 is called the likelihood ratio, and the subscript of the expectation
operator refers to the probability measure with respect to which the expected value is
taken. This transformation suggests that an alternative method of estimating Y under P
is to estimate Y L under P 0 .
In discrete event simulation, the probability space under consideration is the space of
possible sample paths. In this case, P is rarely available in an analytic form. Instead, P is
induced by the stochastic components of the model. Therefore, the likelihood ratio is also
rarely available in an analytic form.
The framework for importance sampling discussed in this chapter is designed to expedite
the process of inducing P 0 by altering the stochastic components of the model, and to provide
a method for calculating the likelihood ratio to compensate for the change of measure.
The framework has a natural decomposition into two components. The rst component,
15
used for model and performance/dependability measure specication, is the SAN-based
reward model (SBRM) [32]. The second component, used to induce alternative probability
measures, is the new concept of the importance sampling governor.
Before discussing how to induce an alternative probability measure on the sample path
space of a SBRM, we must rst describe the stochastic elements and execution rules of
this component of the framework. We briey review concepts related to SAN-based reward
models in the following section.
2.2 SAN-Based Reward Models
2.2.1 SANs
Stochastic extensions to Petri nets have been widely recognized as a powerful, versatile,
tool for eciently describing complex discrete event systems. For example, stochastic activity networks (SANs) [19] have been demonstrated as an eective framework for discrete
event system modeling [28]. The primitive elements used to construct a SAN are places,
activities, arcs, and gates. Places in SANs have the same interpretation as in Petri nets.
Places hold tokens, which represent system resources, such as customers in a queue, working
components, etc. The number of tokens in a place is called the marking of that place. The
vector consisting of the marking of each place in a SAN is called the marking of the SAN.
There are two types of activities, timed and instantaneous. Timed activities are used to
represent delays in the system that aect the performance/dependability measure of interest, while instantaneous activities are used to abstract delays deemed insignicant relative
to the performance/dependability measure. Uncertainty about the length of the delay represented by a timed activity is described by a continuous probability distribution function
called the \activity time distribution function". Given a timed activity a, and a marking
, the activity time distribution function of a in is denoted by F (; ; a). The semi-colon
16
notation used here indicates F is parameterized by the activity. In the formal denition of
SANs [31], F is an assignment of continuous functions to timed activities, and parameters
are subscripts. We have found it more convenient to use the semi-colon notation.
Activities have cases. Cases are used to represent uncertainty about the action taken
upon completion of an activity. C (; ; a) is the case distribution of a in , a discrete
probability distribution over the cases.
Gates connect activities and places, dening interdependencies among system resources
and system delays. There are two types of gates. Input gates have a predicate, a Boolean
function of the markings of the connected places, and a function. Input gates are connected
to one or more places and the input of a single activity. When the predicate is true, the
gate holds.
Output gates are connected to one or more places, and the output side of an activity. If
the activity has more than one case, output gates are connected to a single case. Output
gates have only a function. Gate functions provide exibility in dening how the markings
of connected places change when the delay represented by an activity expires.
Arcs in SANs are default gates, dened to duplicate the behavior of arcs in Petri nets.
Thus, arcs are directed. Each arc connects a single place and a single activity. The arc is
an input arc if it is drawn from a place to an activity. An output arc is drawn from an
activity to a place. An input arc holds if there is at least one token in the connected place.
The function of an input arc removes a token from the connected place, while the function
of an output arc adds a token to the connected place.
An activity is enabled when all of its input gates hold. A marking in which only timed
activities are enabled is called a stable marking. If instantaneous activities are enabled, the
marking is unstable. We represent the set of enabled activities in a stable marking, , by
en().
17
When an activity becomes enabled, it is activated, that is, scheduled to complete. Activation also occurs when an activity completes, but remains enabled. The marking at
activation time is known as the activation marking. The time between activation and
scheduled completion of an activity, called the activity time, is sampled from the activity
time distribution, F (; ; a), where is the activation marking. The completion of an activity in a SAN is analogous to the ring of a transition in a Petri net. Upon completion of an
activity, input gate functions are executed rst, followed by output gate functions. These
functions operate on the current marking of the SAN to produce the next marking.
Activities that are activated are not required to complete. An activity is aborted when
the SAN moves into a new stable marking in which one or more of the activity's input
gates no longer hold. Reactivation occurs when the SAN enters a stable marking that is
an element of the set of reactivation markings of a in , denoted G(; a). The marking
argument of G refers to the activation marking of a. G(; a) returns the set of reactivation
markings for a when a is activated in . When an activity is reactivated, it is aborted and
activated. Given a timed activity, a, and reactivation marking of a in , 0 , the new activity
time of a is sampled from F (; 0 ; a).
Figure 2.1 shows the SAN model for a machine-repairman system that uses a delayed
group repair policy [11]. There are two types of components in the system, with dierent
failure rates. The places (represented by circles in Figure 2.1) labeled type 1 and type 2
model the two types of components. The marking of each place represents the number
of working components of that type. In this case, there are two type-one components,
and four type-two components. Timed activities (thin ovals) fail 1 and fail 2 model the
time between failures for each component. As shown in Table 2.1, the failure times are
exponentially distributed with marking dependent rate parameters. A component of typeone fails with rate 0.005, while components of type-two fail at twice that rate. The use
18
Initial marking = 2
failed_1
type_1
fail_1
repair
policy
type_2
failed_2
fail_2
Initial marking = 4
Figure 2.1: SAN Model of Delayed Repair System
Table 2.1: Activity Time Distributions
Activity Distribution
fail 1
fail 2
repair
exponential(0:005 MARK (type 1))
exponential(0:01 MARK (type 2))
exponential(1.0)
of marking dependent rate parameters allows us to avoid including an activity for each
component's failure time distribution, resulting in a more compact representation.
The markings of places failed 1 and failed 2 represent the number of components of each
type that have failed. There is one repairman in the system. The repair policy is to wait
until at least two components of the same type have failed, and then begin to repair the
whole group. Type-one component repair is given preemptive priority over repair of typetwo components. When the repair is completed, all components of that type are as good
as new. This repair policy is implemented in the input gate (triangle) policy. Graphically,
input gates can have many inputs, but only one output, while output gates may have many
19
Table 2.2: Input Gate Predicate and Function for Markov Machine-Repairman Model
Gate Enabling Predicate
policy (MARK (type 1) > 0 jj
MARK (type 2) > 0) &&
(MARK (failed 1) == 2 jj
MARK (failed 2) > 1)
Function
if (MARK (failed 1) == 2) f
MARK (failed 1) = 0;
MARK (type 1) = 2; g
else f
MARK (failed 2) = 0;
MARK (type 2) = 4; g
outputs but only one input. Policy has four inputs, and one output. Table 2.2 shows the
description of the predicate and function of policy. As in the C programming language,
the double ampersand (&&) stands for AND, the double bar (jj) stands for OR, and the
double equals (==) tests for equivalence. The single equals makes an assignment. The
predicate of input gate policy will be true when at least two components of either type have
failed, and the system is still operational. Since the repair rate is the same for both types
of components, the preemptive priority of type-one components is easily handled. If two or
more components of either type have failed, the repair activity is enabled. Upon completion,
the action taken depends on the marking of place failed 1. The dependency is implemented
in the function of policy. If two components of type-one have failed before repair completes,
then the type-one components are repaired. Otherwise, type-two components are repaired.
There is no need to explicitly model the preemption, due to the memoryless property of the
exponential distribution. When the repair activity completes, the failure activity for the
component type that was repaired is reactivated. (Because each of the failure activities use
marking dependent rates, they must be reactivated upon a change in the marking of the
place determining the rate of the activity, in order to maintain the correct failure rate.)
If all components of both types fail, the system fails, and all repair activity halts; the
failed state is an absorbing state.
20
2.2.2 Reward Variables
Upon completing the model of the system, one must specify the performance/dependability measures of interest in terms of the model. In the SAN modeling framework, performance/dependability measures are specied in terms of reward variables [32]. A reward
variable consists of a reward structure [10] together with a variable type. The reward
structure associates rate rewards with markings of the SAN, and impulse rewards with
activity completions. If P is the set of places in a SAN, and A is the set of activities in the
SAN, then the rate reward is dened as a function R : }(P IN ) ! IR, where }() indicates
the power set, and IN is the set of natural numbers. For D 2 }(P IN ), R(D) is the
reward earned when for all (p; n) 2 D, the marking of p is n. The impulse reward is dened
as a function C : A ! IR, where for a 2 A, C (a) is the reward earned upon completion of
a. Given a reward structure, the variable type determines how the structure is evaluated.
For example, one may wish to consider the reward accumulated over an interval of time,
such as for performability [18] measures. Alternatively, one may be interested in examining
the reward at a particular instant of time, as for measures like point availability. A reward
variable collected at an instant of time is conventionally denoted Vt and is dened as
Vt =
X
D2}(P IN )
R(D) ItD +
X
a2A
C (a) Ita ;
where ItD is the indicator of the event that the SAN is in a marking at time t such that
for each (p; n) 2 D, the marking of p is n, and Ita is the indicator of the event that a was
the activity that completed to bring the SAN into the marking observed at time t. Several
other variable types are considered in [32].
As an example reward variable specication, we consider the unreliability of the system
modeled by the machine-repairman SAN in Figure 2.1. The failed state of the system is the
state in which all components have failed. Recall that the number of working components
21
of each type is represented in the SAN as the number of tokens in places type 1 and type 2.
A reward structure identifying the failed state is
C (a) = 0; 8a 2 A
R(D) =
(
1 if D = f(type 1; 0); (type 2; 0)g
0 otherwise:
(2.1)
Using this reward structure, the unreliability during the interval [0; t] is P fVt = 1g, because
the failed state is an absorbing state.
A pairing of a stochastic activity network with a reward structure is called a SAN-based
reward model (SBRM). A technique for composing individual SBRMs into a hierarchical
composed SAN-based reward model exists, and has been used to reduce the size of the state
space considered by analytic solution methods [29], and to reduce the future event list
management in simulation [26]. The methods discussed here extend to composed SBRMs,
but, for simplicity, we discuss only single SBRMs. This concludes the treatment of SANbased reward models. The next section presents a new approach to inducing an alternative
probability measure on the sample paths of an SBRM.
2.3 Importance Sampling Governor
Given a SAN-based reward model, the next step in applying importance sampling is to
describe the new probability measure that will govern the evolution of the sample paths of
the model. The second component of our framework, the importance sampling governor [21],
is a concise vehicle for describing the new measure in terms of the components of the SAN.
The impetus behind this new approach is the desire for a new probability measure that is
easy to sample. If the eort required to obtain a sample from the new measure is too large,
it may nullify the benets of the variance reduction. By specifying the new measure in
terms of the SAN, one can reasonably assume that the sampling eort for the new measure
22
is comparable to that of the original measure. Furthermore, as will be shown later, the new
approach will allow one to describe the desired modications very precisely, such that the
resulting specication may be automatically incorporated into an executable importance
sampling simulation.
Informally, the importance sampling governor, or \the governor," is similar to a nite
state machine. The states of the governor correspond to biased versions of the original
model. That is, each governor state describes modications to the activity time distributions and case distributions in the original SAN. The modications are restricted to timed
activities. This restriction does not signicantly impact the versatility of the governor, since
the behavior due to an instantaneous activity can always be duplicated with a gate on a
timed activity.
Input to the governor is a sequence of acm-states. An acm-state of a SAN is a triple
(a; c; ), where a is the timed activity that completed and c is the case chosen that brought
the SAN into marking . For a given SAN, say SAN, the importance sampling governor is
formally dened as a four-tuple, M = (Q; q0 ; Z ; T ), where:
1. Q is a nite set of state symbols. Each q 2 Q consists of an activity time distribution
assignment and a case distribution assignment for SAN. In governor state q, a timed
activity a in marking has case distribution C (; ; a; q), and activity time distribution
function F (; ; a; q).
2. q0 2 Q is the initial state of M .
3. Z is a countable set of input symbols that is identical to the set of possible acm-states
for SAN. In cases where Z is countably innite, the state transition function allows
only a nite subset of Z to cause transitions to new states.
4. T : QZ ! Q is the state transition function. That is, for a given state of M, q, and
23
SAN acm-state z , T (z ; q) denotes the next M state. T is a partial function, dened
only on a nite subset of Q Z . Elements for which T is undened result in a self
loop.
Specifying importance sampling strategies at the SAN level, rather than at the state
space level, is very convenient. The primary benet is derived from the expressive power of
stochastic activity networks. In particular, one can usually arrive at a compact representation for an importance sampling strategy that spans a large portion of the state space. For
example, consider the application of the failure biasing heuristic to the machine-repairman
model (Figure 2.1). The failure biasing heuristic [16] increases the total probability of a
failure event relative to a repair event, while maintaining the conditional probabilities of
each type of failure event, given a failure event occurs. Suppose one wishes to increase the
total probability of a failure event to p(), depending on , the current marking of the SAN.
For Markov models, one can increase the probability of a failure event by reactivating [19]
the failure activities after scaling their rates with respect to the repair activity. The scale
factor, denoted (), will depend on the total probability of a failure transition in . For
the delayed repair model,
()rr ()
() = (1 , p(p))(
r () + r ()) ;
1
2
(2.2)
where rr () is the rate repair, r1 () is the rate of fail 1, and r2 () is the rate of fail 2.
For n failure activities and m repair activities, the expression for the scale factor becomes
() = p() Pmi=1 ri()=[(1 , p()) Pnj=1 rj ()], where ri() is the rate associated with the
i-th repair activity in marking and rj is the rate associated with the j -th failure activity
in marking .
The machine-repairman model provides an example of a system that requires a complicated importance sampling strategy. In the case of delayed repair, simple failure biasing
24
p2
p1
p3
Delta Failure
Biasing
Unbiased
p6
Normal Failure
Biasing
p4
p5
Figure 2.2: Governor for Modied Failure Biasing
overbiases unlikely paths to failure, possibly resulting in an estimator with innite variance.
Juneja and Shahabuddin [11] studied the mean time to failure of the system modeled by
the machine-repairman SAN, using regenerative simulation. Based on an analysis of the
orders of magnitude of the probabilities of cycles inherent in a delayed repair system, they
modied the basic failure biasing scheme to avoid overbiasing high probability cycles.
Their modied failure biasing heuristic (MFB) [11] calls for three dierent levels of bias.
For markings in which only failure activities are enabled, no biasing is needed. For markings
in which exactly two components of one type have failed and one component of the other
type has failed, so called \delta failure biasing" is used. Delta failure biasing is failure
biasing with a small probability. The analysis in [11] indicates that delta failure biasing
should be an order of magnitude lower than normal failure biasing. In all other markings,
normal failure biasing is applied.
Figure 2.2 shows the state diagram of a governor that implements the MFB scheme
for the machine-repairman model. The biasing done in each state is shown in Table 2.3;
the transition function is detailed in Table 2.4. The rates of the biased failure activities
shown in Table 2.3 were obtained by applying the scale factor from equation 2.2 to the
rates in the original model. For governor state delta, p() = 0:09, while for governor state
normal, p() = 0:5. In this case, the rate of the repair activity is 1:0, so the rr () term in
25
Table 2.3: Governor States
State
unbiased
delta
normal
Bias
Activity Distribution Parameter
unbiased
fail 1
exponential 0:09 0:005 MARK(type 1)=
[(0:005 MARK(type 1) + 0:01 MARK(type 2))
(1:0 , 0:09)]
fail 2
exponential 0:09 0:01 MARK(type 2)=
[(0:005 MARK(type 1) + 0:01 MARK(type 2))
(1:0 , 0:09)]
repair unbiased
fail 1
exponential 0:005 MARK(type 1)=
((0:005 MARK(type 1) + 0:01 MARK(type 2))
fail 2
exponential 0:01 MARK(type 2)=
((0:005 MARK(type 1) + 0:01 MARK(type 2))
repair unbiased
equation 2.2 does not appear in Table 2.3. The unbiased state of the governor contains no
bias. The governor will remain in its unbiased state until one of the predicates associated
with unbiased (see Table 2.4) holds.
2.4 Execution of Governed SANs
Once a governed SAN has been constructed, it must be simulated to obtain the desired
performance measure. SAN simulation is carried out at the network level [26, 31]. In this
section, we describe the network-level execution of a governed SAN, using the machinerepairman model in Figure 2.1 to help clarify the main points.
Starting in the initial marking, enabled activities are activated. The initial marking of
the machine-repairman model consists of two tokens in type 1 and four tokens in type 2 .
Therefore, each of the corresponding failure activities is enabled in the initial marking.
Upon activation, an activity time is sampled according to the activity time distribution
function assigned to the activity by the importance sampling governor, and the calculated
completion time is placed on the future events list. For the machine-repairman model, the
26
Table 2.4: Governor State Transition Function
Current
State Arc Predicate
unbiased p1
p2
delta
p6
p3
normal
p4
p5
(MARK(failed 1) == 1 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 1)
(MARK(failed 1) == 0 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 0)
MARK(failed 1) <= 1 && MARK(failed 2) <= 1
(MARK(failed 1) == 2 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 1 && MARK(failed 2) == 3)
MARK(failed 1) <= 1 && MARK(failed 2) <= 1
(MARK(failed 1) == 1 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 1)
Next
State
delta
normal
unbiased
normal
unbiased
delta
future events list will be initialized with completion times for each of the failure activities.
The activity with the earliest completion time is identied, and a case is chosen according to
the case distribution for that activity in the current marking. The activities in the machinerepairman model all have one case, so their case distributions are trivial. Following the
selection of a case, the functions of each input gate connected to the activity are executed,
followed by the functions of output gates connected to the chosen case, to determine the
next marking. If the next marking has instantaneous activities enabled, they are completed
immediately, cases are chosen, and gate functions are executed until a marking is reached
in which there are no instantaneous activities enabled. The new marking is referred to as
the next stable marking. There are no instantaneous activities in the machine-repairman
model, so all of the markings are stable.
When the next stable marking is reached, the governor state transition function is
executed to determine whether the governor should change state. Hence, for the machinerepairman model with the governor in Figure 2.2, the governor will evaluate the transition
function associated with initial governor state \unbiased," shown in Table 2.4. If one of
the predicates is true, the governor will transition to the corresponding next state. Input
27
gates connected to each activity are checked to see which activities become disabled, which
activities remain enabled, and which activities are newly enabled. Disabled activities are
aborted; their completion times are removed from the future events list. Activities that
remain enabled are checked to see if they must be reactivated in the new marking.
Activities must be reactivated if the governor has changed to a state that assigns a new
activity time distribution function to the activity, or if a reactivation predicate holds. This
execution policy provides great exibility in choosing the importance sampling strategy,
at the cost of constraining the activity time distributions in the original SAN to those
with closed form cumulative distribution functions. If reactivation is called for, the activity
is aborted (the old completion time is removed from the future events list), and a new
completion time is calculated based on a new sample from the activity time distribution
specied by the governor for the activity in the new marking. Newly enabled activities are
activated, and their completion times are added to the future events list. After determining
the status of the activities in the SAN, the activity with the earliest completion time is
identied, and the process repeats.
2.4.1 Example Execution
Figure 2.3 shows a possible sequence of events in the simulation of the machine-repairman
model. In the initial marking, activities fail 1 and fail 2 are activated. Activity times are
sampled for each activity from the activity time distribution functions specied in governor
state \unbiased." Fail 1 is assigned a completion time T2 , and fail 2 is scheduled to complete at time T1 . Therefore, fail 2 has the earliest completion time, and triggers a marking
transition at time T1 . The completion of fail 2 causes one token to be removed from place
type 2 and deposited in place failed 2 . Immediately following the marking change, the gov-
ernor state transition function is executed to nd the governor state for the new marking.
Since none of the predicates on the arcs out of governor state \unbiased" hold, the governor
28
fail_1
fail_2
fail_2
fail_1
repair
fail_1
fail_2
fail_1
fail_2
T0
T1
T2
T3
T4
T5 T6
T7
Figure 2.3: Possible Sequence of Events in the Machine-Repairman Model
state does not change.
In the new marking, since fail 2 remains enabled, it is activated, and scheduled to
complete at time T3 . Hence, in this marking, fail 1 is the earliest scheduled activity. At
time T2 , fail 1 completes, causing one token to be removed from type 1 and deposited in
failed 1. Fail 1 remains enabled, and is activated with new completion time T5 . Activity
fail 2, scheduled to complete at T3 , will trigger the next marking change. The governor
state does not change, since there is only one failed component of each type.
At time T3 , fail 2 completes, removing another token from type 2 and adding a token
to failed 2. At this point, one component of type-one and two components of type-two have
failed, so the governor state transition function dictates that the governor should enter the
delta failure biasing state, \delta."
Input gate policy holds in the new marking. As a result, activity repair is activated, and
scheduled to complete at T4 . Fail 2 remains enabled, and is activated, this time according
to the biased activity time distribution specied in the denition of governor state \delta."
Fail 2 is scheduled to complete at T5 . Moreover, since the activity time distribution of fail 1
has been altered by the governor, fail 1 is reactivated according to the biased distribution.
29
The original completion time of fail 1, T7 , is removed from the future events list, and the
new completion time, T6 , is added. The reactivation is depicted in Figure 2.3 by a solid
line that becomes dotted at the point of reactivation. Repair completes next, at time T4 .
The function of input gate policy is executed, removing all of the tokens from failed 2 and
setting the marking of type 2 to four. As a result of the repair, the governor returns to
state \unbiased."
Since the governor has changed state, and the new state redenes the activity time
distribution functions of fail 1 and fail 2, these activities are reactivated, and the process
continues until the time horizon or the system failure marking is reached.
30
CHAPTER 3
Sample Path Probabilities and the Likelihood Ratio
The SAN-based reward model, in conjunction with the idea of an importance sampling
governor, forms a powerful framework for experimenting with importance sampling. Of
course, to be useful, the concept must be implemented. This section addresses the four
problems solved en route to an implementation of our framework.
First, a stochastic process representation for the acm behavior of a stochastic activity
network is presented, and used to develop an expression for the probability of a SAN sample
path. Second, the likelihood ratio is formulated. Third, the estimation of the expected
value of reward variables under importance sampling is discussed. Finally, an algorithm for
importance sampling for a single SBRM is presented and discussed.
3.1 Sample Path Probabilities
Let the stochastic process (X; T ) = fXn ; Tn : n 2 IN g denote the sequence of acmstates and transition times, respectively, in the evolution of a SAN. Tn is the time of
entry to Xn . The goal of this section is an expression for the probability of the event
E0;n = fX0 = x0 ; T0 2 dt0 ; X1 = x1; T1 2 dt1 ; : : : ; Xn = xn ; Tn 2 dtng; the sample path
of the SAN through the n-th state transition. The probability of this event is most easily
described in terms of the conditional transition probabilities. Let Fn be the sigma eld
generated by the sample path E0;n . Knowledge of this set is knowledge of the complete
history of the process through the n-th transition. Let h(i+1 ; ai+1 ; ci+1 ; i ) represent the
31
probability that the next stable marking is i+1 , given ai+1 completes in i , and case ci+1
is chosen. (In terms of SAN components, h captures the eects of instantaneous activities
on the transition probabilities of the SAN. Note that if there are no instantaneous activities
in the SAN, or if there are no instantaneous activities with multiple cases, h will be trivial
(i.e. 0 or 1) for all markings.) Let Ei = fXi = (ai ; ci ; i ); Ti 2 dti g. Then
P (Ei+1 j Fi ) = h(i+1; ai+1 ; ci+1 ; i ) C (ci+1 ; i ; ai+1 ) Y
f (Ti+1 , Ti ; i+1 ; ai+1 j Fi ) dti+1
(3.1)
(1 , F (Ti+1 , Ti ; ; a j Fi )):
a2en(i ),fai+1 g
Equation 3.1 expresses the probability that activity ai+1 completes in marking i in an
innitesimal interval around Ti+1 , case ci+1 of ai+1 is chosen, and the resulting marking
is i+1 . Recall from Section 2.2.1 that C (; i ; ai+1 ) is the case distribution on activity
ai+1 in marking i , and F (; ; a j Fi ) is the activity time distribution function of a in ,
conditioned on the history of the sample path; f is the corresponding density function.
For obvious practical reasons, one would like to avoid maintaining a record of the complete history of the sample path. Nicola, Nakayama, Heidelberger, and Goyal [20] presented
an iterative approach for maintaining an accurate description of the conditional density and
distribution functions. We adapt their approach to the context of stochastic activity networks. Suppose activity a is activated upon entry to the i-th acm-state. Let Ta be the
random variable representing the activity time of a. P fTa > tg = 1 , F (t; i ; a). Upon the
next transition, if a has not completed and is still enabled,
P fTa > t + Ti+1 , Ti j Ta > Ti+1 , Ti g = (1(1,,F (Ft(+T Ti+1,,T T; i ; ; ai ;))a)) :
i+1
i i
32
Let F (; ; a) = 1 , F (; ; a), and let
F i+1 (t; i ; a) = F i (t + Ti+1 , Ti; i ; a) ;
F i (Ti+1 , Ti ; i ; a)
(3.2)
fi+1 (t; i ; a) = fi(t + Ti+1 , Ti; i ; a) :
F i (Ti+1 , Ti ; i ; a)
(3.3)
Equations 3.2 and 3.3 show how to update the conditional complementary distribution,
and conditional density, respectively, of the activity time, after each transition. Using
equations 3.2 and 3.3, equation 3.1 may be restated:
P (Ei+1 j Fi ) = h(i+1 ; ai+1 ; ci+1 ; i ) C (ci+1 ; i ; ai+1 )
Y
fi(Ti+1 , Ti ; ; ai+1 ) dti+1
F i(Ti+1 , Ti ; ; a); (3.4)
a2en(i ),fai+1 g
where the marking parameter in the density and complementary distribution functions is
the activation marking of the associated activity. Finally,
P (E0;n ) = P fXn = xn; Tn 2 dtn j Fn,1g
P fXn,1 = xn,1; Tn,1 2 dtn,1 j Fn,2g : : : PfX0 = x0; T0 2 dt0g;
=
nY
,1
i=0
h(i+1 ; ai+1; ci+1 ; i ) C (ci+1; i ; ai+1 ) fi(Ti+1 , Ti ; ; ai+1 ) dti+1
Y
a2en(i ),fai+1 g
F i (Ti+1 , Ti ; ; a):
(3.5)
To demonstrate how equations 3.2 and 3.3 can be used to iteratively calculate the
probability of a sample path, we give an example based on the delayed repair SAN. Consider
the rst two transitions shown in Figure 2.3. In the initial marking, activities fail 1 and
fail 2 are activated. Fail 2 completes rst, at time T1 . The probability that fail 2 completes
33
rst and T1 2 dt1 , given the initial marking and T0 = 0, in terms of the activity time
probability distribution functions, is, according to equation 3.5,
f0(T1 ; 0 ; fail 2) dt1 F 0 (T1 ; 0 ; fail 1):
Fail 1 and fail 2 were activated in 0 , so f0 (; 0 ; fail 2) = f (; 0 ; fail 2), and F 0 (; 0 ; fail 1)
= F (; 0 ; fail 1). In the next marking, since fail 2 remains enabled, it is activated. At time
T2 , fail 1 completes, remains enabled, and is activated, while activity fail 2 remains enabled.
The probability that fail 1 completes in 1 and T2 2 dt2 , given 0 and T1 , is
f1 (T2 , T1; 0 ; fail 1) dt2 F 1 (T2 , T1 ; 1 ; fail 2):
f1(T2 , T1 ; 0 ; fail 1) = f0(T2 ; 0 ; fail 1)=F 0 (T1 ; 0 ; fail 1), but F 1(T2 , T1 ; 1 ; fail 2) =
F (T2 , T1 ; 1 ; fail 2), because fail 2 was activated in 1 . Combining the last few results, we
see that
P (E0;2 ) = h(1 ; fail 2; c1 ; 0 ) C (c1 ; 0 ; fail 2) f (T1 ; 0 ; fail 2) dt1 F (T1 ; 0; fail 1)
h(2 ; fail 1; c2 ; 1) C (c2 ; 1; fail 1) f1(T2 ; ; fail 1) dt2 F (T2 , T1 ; 1; fail 2)
= h(1 ; fail 2; c1 ; 0 ) C (c1 ; 0 ; fail 2) f (T1 ; 0 ; fail 2) dt1 F (T1 ; 0 ; fail 1)
1) dt2
h(2 ; fail 1; c2 ; 1) C (c2 ; 1; fail 1) f (FT(2T; ;0; fail
F (T2 , T1; 1 ; fail 2)
; fail 1)
1
0
= h(1 ; fail 2; c1 ; 0 ) C (c1 ; 0 ; fail 2) h(2 ; fail 1; c2 ; 1 ) C (c2 ; 1 ; fail 1)
f (T1; 0 ; fail 2) dt1 f (T2; 0 ; fail 1) dt2 F (T2 , T1; 1 ; fail 2):
3.2 Likelihood Ratio
The likelihood ratio is the ratio of the probability of a sample path in the original SAN
(equation 3.5) to the probability of the same path in the biased SAN. The probability of a
34
sample path in a biased SAN is
P 0(E0;n ) =
nY
,1
i=0
h(i+1 ; ai+1 ; ci+1 ; i ; qi ) C 0(ci+1 ; i ; ai+1 ; qi) fi0 (Ti+1 , Ti ; ; ai+1 ) dti+1
Y
a2en(i ),fai+1 g
F 0i (Ti+1 , Ti ; ; a);
(3.6)
where F 0i+1 (t; ; a) = F 0 (t; i+1 ; a; qi+1 ), and fi0+1 (t; ; a) = f 0 (t; i+1 ; a; qi+1 ), when the
i-th transition results in the activation of a. Recall that a is activated if i is a reactivation
marking in the original SAN, or the governor changes the activity time probability distribution function of a. Note that in the biased SAN, the activity time distribution and density
functions are parameterized by the state q of the governor, as well as the activity.
In equation 3.6, the eects of the governor's modications are explicit. The conditional
transition probability denoted by h is determined by the case distributions of intervening
instantaneous activities. Since modications are restricted to timed activities, h is the same
as in the original model. Altered timed activity case distributions are accounted for by C 0 ,
while f 0 and F 0 account for the eects of biased activity time distribution functions. Given
equations 3.5 and 3.6, the likelihood ratio is simply
L(E0;n ) =
nY
,1
Y
C (ci+1 ; i; ai+1 )fi (Ti+1 , Ti; ; ai+1 )
F i(Ti+1 , Ti; ; a) :
0
0
0
i=0 C (ci+1 ; i ; ai+1 ; qi )fi (Ti+1 , Ti ; ; ai+1 ) a2en( ),fa +1 g F i (Ti+1 , Ti ; ; a)
i
i
(3.7)
3.3 Estimating Reward Variables
The next problem to address is the calculation of the expected value of reward variables
under importance sampling. Reward variables are functions of the sample path of the
SAN. Suppose M is a function of the sample path of a SAN, and one wants to calculate
EP [M (E0;n)], where the subscript P denotes the probability measure with respect to which
35
the expected value is taken. In general, if P 0 (E0;n ) 6= 0 whenever M (E0;n )P (E0;n ) 6= 0,
EP [M (E0;n )] = EP 0 [M (E0;n)L(E0;n )], as long as the expected values exist and are nite.
We have presented the likelihood ratio expression for sample paths containing a xed number
of transitions.
It is often the case in transient simulation that one wishes to estimate EP [Mt ], where
Mt is the value of M when the simulation clock time is t. Because the stopping criterion
is based on the simulation clock, the number of transitions in a sample path is a random
variable. Let the random variable N = minfn : Tn > tg. The N -th transition advances the
simulation clock past t, the observation point of the function M . Since fN = ng 2 Fn , N
is a stopping time. If N is nite with probability one under P and P 0 , EP [jMt j] < 1, and
P 0(E0;n ) 6= 0 whenever Mt P (E0;n ) 6= 0, then EP [Mt ] = EP 0 [Mt L(E0;N )], where L(E0;N ) is
calculated as in equation 3.7, with N substituted for n. For more details on the extension
to stopping times, see [6].
For example, the expected value of Vt , with the reward structure given in equation 2.1,
over the sample path space of the delayed repair SAN, is
EP [Vt ] =
X
D2}(P IN )
R(D) EP [ItD ] +
X
a2A
C (a) EP [Ita ]:
Therefore, under importance sampling one has
EP [Vt ] =
X
D2}(P IN )
R(D) EP 0 [ItD L(E0;N )] +
X
a2A
C (a) EP 0 [Ita L(E0;N )]:
(3.8)
To estimate EP [Vt ] using importance sampling, one generates sample paths under P 0 and
forms the sample mean
=
K
X
i=1
Vti L(E0;N );
i
where K is the number of generated paths, Ni is the number of the transition that causes the
36
simulation clock to rst exceed t during the i-th path generation, Vti is the i-th observation
of the variable, and L(E0;N ) is the likelihood ratio corresponding to the i-th sample path.
i
As indicated earlier in this section, the expected value of any performance/dependability
measure expressed in terms of the sample path of a SAN can be estimated using importance
sampling. Included in this category are reward variables collected over an interval of time,
an instant of time t as t ! 1, and an interval of time [t; t + l], as l ! 1. The last two are
steady state measures, so of course there must exist a steady state measure on the sample
path space of the SAN for these variable types to make sense. Furthermore, one can look
at the time average of reward accumulated over a given interval.
Thus, it is apparent that the presented framework is capable of solving a wide range
of problems. The last remaining task is an algorithm amenable to implementation in a
software tool.
3.4 Algorithm
Algorithm 1 implements importance sampling simulation solution of a SAN-based reward model for a transient reward variable. It is designed to simulate until a user-specied
relative accuracy is reached, with a user-specied condence level. The rst step in the
body of the algorithm is to initialize everything. An observation of the reward variable is
then obtained by generating a sample path of the SAN up to the observation point of the
variable. The observation point of a variable is specied in terms of the simulation clock.
When the next activity that will complete (called the \trigger activity") is identied, its
completion time is compared to the variable observation point. If the completion time is
smaller, the simulation clock is advanced, the variable is updated, and the SAN is executed
to arrive at the next stable marking and the new future events list. (The algorithm for
carrying out the \execute SAN" step can be found in [26]). The next stable marking is
37
Algorithm 1 Executing a single SBRM under importance sampling.
While variable estimate half width > requested maximum error:
Initialize SAN marking.
Initialize Governor.
Initialize future events list.
Initialize likelihood ratio.
While variable remains to be observed:
Identify trigger activity.
If variable observation point < completion time of trigger:
Observe the variable.
Weight the observation by multiplying by the likelihood ratio.
Update the estimate of the sample mean.
Update the condence interval half width.
Advance the simulation clock to the completion time of the trigger activity.
Update variable with reward for this state and holding time.
If variable remains to be observed:
Execute SAN.
Update the state of the governor.
Reactivate activities aected by governor state change.
Update the likelihood ratio.
Calculate reward for this state and holding time.
then used to update the governor state. If the governor state changes as a result of the new
marking, activities that have their bias changed are reactivated. Then, the contribution of
the transition to the new marking is calculated and the likelihood ratio is updated. Finally,
the reward for the new marking is calculated.
The process continues until the observation point of the variable precedes the completion
time of the trigger activity. At this time, the current value of the variable is observed and
weighted by the likelihood ratio. Then the sample mean is updated and a new condence
interval is calculated and compared to the requested accuracy. If the condence interval is
too wide, additional observations are acquired until the requested relative error is achieved.
38
CHAPTER 4
Implementation
The last two chapters described the theory underlying the importance sampling facility.
In this chapter, the implementation of the theory and its incorporation into the UltraSAN
modeling tool is presented. We begin with an overview of the tool.
4.1 UltraSAN
UltraSAN [3] is a software tool for model-based evaluation of discrete event systems
represented as stochastic activity networks. It runs on UNIX work stations using the X
Window System, and is currently available for the Convex, DECstation, IBM RS6000 and
Sun/4 architectures. The eectiveness of the tool has been demonstrated through its use
in several model-based evaluation studies, e. g., [17, 27, 28, 30].
Figure 4.1 shows the organization of the tool. Models are specied through a graphical
interface that runs on the X Window System. The three main components of the userinterface (SAN editor, composed model editor, and performability variable editor) appear
at the top of Figure 4.1. The SAN editor allows one to specify SANs by simply drawing the
SAN. Auxiliary editors are provided for dening each SAN component. For example, an
activity editor is used to select the activity time distribution function, its parameters, and its
reactivation function. The composed model editor can be used to combine individual SANs
into a hierarchical composed model using replicate and join operations [29]. Finally, the
performability variable editor is used to specify reward variables for the desired performance
39
Composed
Model
Editor
Performability
Variable
Editor
Composed Model
Description
Performability
Variable
Description
SAN
Editor
SAN
Description
SAN
Description
....
SAN
Description
....
Model Description
Steady State
Simulator
Reduced
Base Model
Constructor
Terminating
Simulator
Reduced Base
Model
Direct
Steady State
Solver
Iterative
Steady State
Solver
Transient
Instant−of−Time
Solver
Importance Sampling
Governor
Editor
IS Governor
Description
Transient
Interval−of−Time
Solver
Figure 4.1: Organization of UltraSAN
IS Terminating
Simulator
40
measures.
The primary function of the user-interface modules is to generate a C language description of the model from its graphical specication. The generated C code description of the
model is automatically compiled and linked with other libraries to produce the reduced base
model construction program and the simulation programs.
The reduced base model construction program generates a stochastic process description
of the model that is amenable to solution by standard analytical methods for continuous
time discrete state space Markov processes. The generated stochastic process serves as input
to the analytical solvers available in UltraSAN [3, 25, 34]. Note that the model must satisfy
the Markov property to be analytically tractable. SAN models satisfy the Markov property
if all activity time distributions are exponential, and activities are reactivated often enough
so that their rates depend only on the current marking [29].
Two dierent traditional simulation programs may be generated. The steady state simulator uses an iterative batch means [15] technique after a specied initial transient period
to estimate the steady-state expected value and variance of instant-of-time reward variables.
The terminating simulator uses the method of independent replications to estimate the expected value and variance of all three reward variable types described in section 2.2.2 for
a specic time or time interval. Both simulators are designed to estimate reward variables
to within a user-specied relative accuracy at a user-specied condence level. Moreover,
both simulators are designed to take advantage of symmetries in the composed model to
increase the eciency of the simulation [26].
4.2 Importance Sampling Facility
The importance sampling framework detailed in Chapters 2 and 3 has been implemented [22] and incorporated into the UltraSAN modeling package. Consistent with the
41
Figure 4.2: Importance Sampling Governor Editor
Figure 4.3: Governor State Editor
42
Figure 4.4: Activity Bias Editor
Figure 4.5: Governor State Transition Function Editor
43
other components of UltraSAN, the importance sampling facility has an X Window Systembased user-interface that generates C code describing the importance sampling governor.
This code is automatically compiled and linked with other UltraSAN libraries to produce
an importance sampling simulation program.
The user-interface to the UltraSAN importance sampling facility has three main components. The rst component, shown in Figure 4.2, is the importance sampling governor
editor. With this editor, the user can add, edit, or delete states from the importance sampling governor, and edit the governor state transition function. In Figure 4.2, the four states
of the governor for the machine-repairman model are listed.
Recall that each governor state may contain new denitions of the timed activities in each
SAN in the composed model. We refer to a new denition of an activity as a bias on that
activity. The governor state editor, shown in Figure 4.3, allows the user to scroll through
the list of SANs in the composed model, using the Prev. and Next Submodel buttons. For
each SAN, the editor displays a list of the timed activities in the SAN. In Figure 4.3, state
\normal" of the governor for the machine-repairman example is shown. One may choose to
edit or remove a bias from any timed activity in the model. The activity bias editor, shown
in Figure 4.4, allows the user to redene the activity time distribution and case distribution.
The example activity shown in Figure 4.4 is fail 1 1 from the machine-repairman example.
Fail 1 1 has only one case, so the case distribution portion of the editor is not displayed.
Figure 4.5 shows the importance sampling governor state transition function editor. The
transition function editor is used to specify when the governor should change state. The
state transition function for the governor is dened state by state. For each governor state,
one species arcs to other governor states. Associated with each transition arc is a predicate
that denes conditions under which the transition will be taken. The predicate for the arc
is dened in terms of predicates on the SANs in the composed model. The example in
44
Figure 4.5 shows the predicate for the arc from governor state \normal" to governor state
\delta."
When the user has nished specifying the governor, the user-interface converts his description to a series of C functions and header les and compiles them into a library. The
importance sampling simulation program is generated by linking the library describing the
governor, the library containing code for the importance sampling simulation engine, and a
short C program that handles initializations and makes the initial call into the simulation
code.
45
CHAPTER 5
Examples and Results
This chapter presents some example applications of the tool to machine-repairman and
simple queuing models.
5.1 Machine-Repairman Models
The rst example is the machine-repairman model introduced in 2.2.1 and used as an
example throughout the thesis. Following the discussion of the results obtained for the
example, a non-Markov version of the model is introduced and results are obtained and
discussed. Throughout this section we use the abbreviation M-R for machine-repairman in
the captions of the tables.
5.1.1 Results for Markov Machine-Repairman Model
In this subsection, we present some results for the unreliability of the Markov machinerepair model introduced in 2.2.1. Recall from Subsection 2.2.2 that the unreliability over
an interval can be evaluated through an instant-of-time variable when the system failure
marking is an absorbing marking. In this case, examining the instantaneous rate reward
at the end of the interval will reveal whether the system has failed during the interval.
Table 5.1 shows the results of applying the modied failure biasing heuristic [11] (using the
governor in Figure 2.2) to estimating the unreliability of the delayed repair example during
46
Table 5.1: Unreliability of Markov M-R Model Using MFB
Result (10,6 )
Error
Replications
CPU Time (sec.)
Numerical
1:919596
8:471 10,14
Standard
Simulation
1:52
30%
28375000
67230
p() = 0:1
1:97
10%
880986
2940
MFB, = 0.09
0:2
0:3
0:4
1:93
1:90
1:96
10%
10%
10%
456607 338497 377536
1881
1385
1585
0 :5
1:86
10%
399968
1507
Table 5.2: Activity Time Distributions for the Sti M-R Model
Activity Distribution
fail 1
fail 2
repair
exponential(0:0005 MARK (type 1))
exponential(0:001 MARK (type 2))
exponential(1:0)
the interval [0; 100]. In terms of the reward variable,
Unreliability = EP [V100 ];
where V100 has the reward structure given in Equation 2.1. For comparison purposes, the
model was solved numerically using the transient solver in UltraSAN. The results listed for
standard and importance sampling simulation are the point estimates, and the errors listed
for the simulation results are the relative half-widths of 95% condence intervals. The error
listed for the numerical result is the upper bound on the absolute error due to truncation
of the innite series in uniformization.
In Table 5.1, the level of failure biasing has been varied (in governor state normal),
Table 5.3: Unreliability of Sti Markov M-R Model Using MFB
Result (10,11 )
Error
Replications
CPU Time (sec.)
Numerical
p() = 0:1
1:112
1:08
8:171 10,14 10%
167103647
77308
MFB, = 0.09, force = 1.0
0:2
0:3
0:4
1:14
1:13
1:09
10%
10%
10%
69177466 53334991 47310372
32233
24475
22181
0:5
1:17
10%
142829678
67446
47
p4
p2
p1
Approximate
Forcing
p6
p3
Normal Failure
Biasing
Delta Failure
Biasing
Unbiased
p7
p5
p8
Figure 5.1: Governor for MFB with Approximate Forcing
keeping the delta failure biasing constant. Recall that, according to [11], the delta failure
biasing should be kept an order of magnitude less than the failure biasing employed. The
simulations were run on a DECstation 5000/120, with 16MB of main memory. As can be
seen from Table 5.1, MFB performed much better than traditional simulation. Despite
using more than forty times more CPU time, the traditional simulator condence interval
remained three times larger than the importance sampling simulator.
Next, the performance of MFB on a stier version of the model is examined. The new
activity time distributions are listed in Table 5.2. As shown in Table 5.3, MFB does not
perform as well for the stier version of the model. These simulations were run on a DECstation 5000/200 with 48MB of main memory. Since MFB was designed for regenerative
simulation [4], its poor performance in transient simulation is not surprising. The main
problem is that the holding times in the sti delayed repair model are long enough that the
system is unlikely to reach a state with repair activity before the time horizon is reached.
We propose to solve this problem by employing the forcing technique in combination
with MFB. Traditional forcing was developed in the context of models where all states other
than the \all components working" state have repair activity [16]. To apply forcing, one
samples the holding time in the initial state from a conditional distribution, such that the
probability of leaving the initial state before the time horizon is one. In the delayed repair
48
Table 5.4: State Specication for Governor for MFB with Approximate Forcing
State
approximate
forcing
delta
normal
unbiased
Activity
fail 1
fail 2
repair
fail 1
Distribution
exponential
exponential
unbiased
exponential
fail 2
exponential
repair
fail 1
unbiased
exponential
fail 2
exponential
repair unbiased
unbiased
Bias
Parameter
0:005 MARK(type 1)
0:01 MARK(type 2)
0:09 0:0005 MARK(type 1)=
[(0:0005 MARK(type 1) + 0:001 MARK(type 2))
(1:0 , 0:09)]
0:09 0:001 MARK(type 2)=
[(0:0005 MARK(type 1) + 0:001 MARK(type 2))
(1:0 , 0:09)]
0:0005 MARK(type 1)=
((0:0005 MARK(type 1) + 0:001 MARK(type 2))
0:001 MARK(type 2)=
((0:0005 MARK(type 1) + 0:001 MARK(type 2))
model, there are four markings (other than the absorbing marking) with no repair activity.
Hence, one would like to force the system to leave this set of states before the time horizon,
so that no time is wasted simulating paths with no chance of system failure. However,
since it can be dicult to sample from the conditional probability distribution of the exit
time from a set of states, traditional forcing may not be the most ecient approach. An
approach (discussed by Nicola, et al. [20]) that is easier to implement but almost as eective
as forcing is called approximate forcing. Let S be the set of markings where the system is
operational, but no repair activity has been initiated. Approximate forcing seeks to increase
the probability that the SAN exits from S before the time horizon is reached.
It is easy to incorporate approximate forcing into the governor for the delayed repair
SAN. Let the exit time from S be denoted S , and let t be the time horizon. Figure 5.1 shows
the governor state transition diagram for a governor implementing approximate forcing in
addition to MFB. As one can see in Figure 5.1 and Table 5.4, implementing approximate
49
Table 5.5: State Transition Function for Governor for MFB with Approximate Forcing
Current
State
Arc Predicate
approximate p1
forcing
p2
delta
p4
p3
normal
p6
p5
unbiased
p8
p7
(MARK(failed 1) == 1 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 1)
(MARK(failed 1) == 0 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 0)
MARK(failed 1) <= 1 && MARK(failed 2) <= 1
(MARK(failed 1) == 2 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 1 && MARK(failed 2) == 3)
MARK(failed 1) <= 1 && MARK(failed 2) <= 1
(MARK(failed 1) == 1 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 1)
(MARK(failed 1) == 1 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 1)
(MARK(failed 1) == 0 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 0)
Next
State
delta
normal
unbiased
normal
unbiased
delta
delta
normal
forcing is simply a matter of altering the initial governor state to bias the failure activities
such that P fS < tg is increased. Experiments in [20] suggest that P fS < tg 0:8 is a
good choice. To determine the amount of bias required to raise P fS < tg to 0:8, one can
employ pilot runs. In the case of exponentially distributed activity time distributions, one
may implement approximate forcing by scaling the rates. Forcing during subsequent visits
to S is not advantageous, because this approach leads to too many long and unlikely paths,
resulting in additional variance. Therefore, as shown in Figure 5.1 and Table 5.5, forcing
is only applied until the rst exit from S . Upon return to S , the governor moves to the
unbiased state. As illustrated in Table 5.6, the use of approximate forcing to reduce the
expected exit time from S resulted in an improvement of an order of magnitude over the
results for MFB (Table 5.3).
We were able to easily experiment with a relatively new importance sampling strategy for
a class of systems that only recently began to receive attention. Our framework facilitated
these experiments by providing a powerful mechanism for dening and executing importance
50
Table 5.6: Unreliability of Sti Markov M-R Model Using MFB and Approx. Forcing
Result (10,11)
Error
Replications
CPU Time (sec.)
Numerical
p() = 0:1
1:112
1:06
8:171241 10,14 10%
3203581
4860
MFB, = 0.09, force = 10.0
0:2
0:3
0:4
1:05
1:03
1:05
10%
10%
10%
1536334 1097725 895590
2390
1714
1428
0:5
1:09
10%
1021690
1670
sampling experiments. The level of automation made possible by incorporating our current
work into the UltraSAN modeling package expedited the process of specifying the model
of the delayed repair system. The concept of the importance sampling governor enabled
us to implement and modify biasing strategies much more quickly than we could have in a
hand-coded simulation. Furthermore, we were able to suggest and test an improvement in
the case of a short time horizon, that, as shown in Table 5.6, yielded an order of magnitude
improvement in the CPU time needed to achieve the specied relative accuracy.
5.1.2 Non-Markov Machine-Repairman Model
Figure 5.2 shows a SAN model of a non-Markov machine-repairman system. The system
is composed of two types of components. There are two components of type-one, and four
components of type-two. The components are represented in the SAN by places comp 1 1,
comp 1 2, comp 2 1, etc. A token in one of these places indicates that the component
is working. Each component fails independently of the others, and the failure times of
components of the same type are identically distributed.
The failure times of the components are modeled by separate timed activities fail 1 1,
fail 2 1, etc. As shown in Table 5.7, the activities corresponding to the same type of
component have the same activity time distributions. The failure times of the components
have Weibull distributions with shape parameter and scale parameters 1 for type-one
and 2 for type-two components. For this example, = 1, 1 = 100, and 2 = 50. Later,
51
1
comp_1_1
fail_1_1
1
install_1
comp_1_2
fail_1_2
failed_1
type_1
remove
type_2
policy
1
failed_2
comp_2_1
fail_2_1
1
comp_2_2
fail_2_2
1
install_2
comp_2_3
fail_2_3
1
comp_2_4
fail_2_4
Figure 5.2: Non-Markov M-R Model with Delayed Repair Policy
52
Table 5.7: Activity Time Distributions for non-Markov M-R Model
Activity Distribution
fail 1 1
fail 1 2
fail 2 1
fail 2 2
fail 2 3
fail 2 4
install 1
install 2
remove
Weibull(,1 )
Weibull(,1 )
Weibull(,2 )
Weibull(,2 )
Weibull(,2 )
Weibull(,2 )
exponential(3600:0)
exponential(3600:0)
hyperexponential(1:0,0:5,0:9)
Table 5.8: Input Gate Predicates and Functions for non-Markov M-R Model
Gate Enabling Predicate
Function
policy (MARK (failed 1) < 2 && if (MARK (failed 1) == 2) f
MARK (failed 2) >= 2) jj
MARK (failed 1) = 0;
(MARK (failed 1) == 2 &&
MARK (comp 1 1) = 0;
MARK (failed 2) < 4)
MARK (comp 1 2) = 0;
MARK (type 1) = 1; g
else f
MARK (failed 2) = 0;
MARK (comp 2 1) = 0;
MARK (comp 2 2) = 0;
MARK (comp 2 3) = 0;
MARK (comp 2 4) = 0;
MARK (type 2) = 1; g
we will vary .
When a failure activity completes, the token is removed from its input place and deposited in failed 1 or failed 2, based on the type of the component.
Repair begins only after two components of the same type have failed. Places failed 1
and failed 2 are used to maintain a count of failed components of each type. The predicate
of input gate policy, shown in Table 5.8, enables timed activity remove only if failed 1
or failed 2 contains at least two tokens. Repair of type-one components has preemptive
priority over repair of type-two components. This requirement is met by introducing a
53
Table 5.9: Activation and Reactivation Predicates for non-Markov M-R Model
Activity Activation
remove
Reactivation
MARK (failed 1) < 2 MARK (failed 1) == 2
reactivation predicate for the remove activity. As shown in Table 5.9, the remove activity
is reactivated (aborted and restarted) if it is activated in a marking with less than two typeone components failed, and the SAN reaches a marking in which two type-one components
are failed before remove completes.
Repair consists of two stages. First, the failed components group must be removed and
repaired or replaced. Then, the component group is reinstalled. The two-phase nature of
the repair is modeled by two activities in series. Timed activity remove models the time to
remove and repair a component group. As shown in Table 5.7, remove has a hyperexponential activity time. With probability 0:9, the removal time will be exponentially distributed
with rate parameter 1, but with probability 0:1, the removal time will be exponentially distributed with rate parameter 0:5. Timed activities install 1 and install 2 model the time it
takes to install components of type-one and type-two, respectively. Both are exponentially
distributed with rate parameter 3600, the assumption being that most of the remove/repair
time is spent on repair, so that the installation of the repaired component group is almost
negligible.
When repair of a component type is complete, all components of that type are as good
as new. Since the function of input gate policy (see Table 5.8) clears the component places
upon completion of timed activity remove, when the install activity completes and replaces
the tokens, the failure activities are activated with new activity time distributions. Thus,
the components are as good as new.
When all components fail, the system fails, and all repair activity halts. The predicate
54
p4
p2
p1
Approximate
Forcing
p6
p3
Normal Failure
Biasing
Delta Failure
Biasing
p5
Unbiased
p7
p8
Figure 5.3: Governor for Modied Failure Biasing with Approximate Forcing
Table 5.10: State Specication for Governor for MFB with Approximate Forcing
State
approximate
forcing
delta
normal
unbiased
Component
type-one
type-two
type-one
type-two
type-one
type-two
unbiased
Bias
Distribution
Weibull
Weibull
Weibull
Weibull
Weibull
Weibull
Parameters
= 1:0; 1 = 25
= 1:0; 2 = 12:5
= 1:0; 1 = 10:0
= 1:0; 2 = 5:0
= 1:0; 1 = 2:0
= 1:0; 2 = 1:0
of input gate policy (Table 5.8) ensures that the remove activity becomes disabled when
the marking of the SAN is such that the marking of place failed 1 is two and the marking
of place failed 2 is four; this marking, where all components have failed, is an absorbing
marking.
Figure 5.3 shows the state diagram for a governor implementing MFB augmented by
approximate forcing. Table 5.10 contains the state denitions, while the governor state
transition function is shown in Table 5.11.
5.1.3 Results for non-Markov Machine-Repairman Model
The results obtained from the non-Markov machine-repairman model are presented in
this subsection. First, MFB is applied and tuned to achieve good performance. Then, ap-
55
Table 5.11: State Transition Function for Governor for MFB with Approximate Forcing
Current
State
Arc Predicate
approximate p1
forcing
p2
delta
p4
p3
normal
p6
p5
unbiased
p8
p7
(MARK(failed 1) == 1 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 1)
(MARK(failed 1) == 0 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 0)
MARK(failed 1) <= 1 && MARK(failed 2) <= 1
(MARK(failed 1) == 2 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 1 && MARK(failed 2) == 3)
MARK(failed 1) <= 1 && MARK(failed 2) <= 1
(MARK(failed 1) == 1 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 1)
(MARK(failed 1) == 1 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 1)
(MARK(failed 1) == 0 && MARK(failed 2) == 2) jj
(MARK(failed 1) == 2 && MARK(failed 2) == 0)
Next
State
delta
normal
unbiased
normal
unbiased
delta
delta
normal
proximate forcing is applied to further improve the performance. Finally, the shape parameter of the failure distributions is modied so that the model cannot be solved analytically,
and the developed importance strategy is applied.
Table 5.12 shows four importance sampling strategies for the machine-repairman model.
The rst three are straightforward MFB, with dierent levels of bias. Plain MFB is tantamount to specifying the approximate forcing state of the governor in Figure 5.3 as unbiased. These strategies are used to demonstrate the eects of tuning the bias to improve
the performance of a heuristic for a given model. The fourth strategy augments MFB with
approximate forcing.
Table 5.13 shows the results obtained for the unreliability of the machine-repairman system during the interval [0; 10]. All results are at a 95% condence level, and the condence
interval is reported as the point estimate and the estimated relative error. One can see
from Table 5.13 that all of the applied strategies reduced by orders of magnitude the CPU
time required to achieve the desired accuracy. The second strategy, IS-2, outperformed the
56
Table 5.12: Importance Sampling Strategies for the non-Markov M-R Model
Name Heuristic
IS-1
IS-2
IS-3
IS-4
State
MFB
normal
delta
MFB
normal
delta
MFB
normal
delta
MFB + approx. forcing force
delta
normal
1 2
1
1
1
1
1
1
1
1
1
2
30
2
10
2
4
25
10
2
1
15
1
5
1
2
12.5
5
1
Table 5.13: Unreliability in [0; 10] of non-Markov M-R Model
Strategy
Result
Error
Replications
CPU Time (sec.)
Traditional
3:3 10,7
10%
1,184,152,657
740,699
IS-1
3:8 10,7
10%
4,593,933
4,871
IS-2
3:4 10,7
10%
574,091
610
IS-3
3:8 10,7
10%
2,162,765
2,320
IS-4
3:5 10,7
10%
39,870
128
other MFB candidates. Thus, IS-2 was chosen as the strategy to use in conjunction with
approximate forcing, forming IS-4. The results for IS-4 in Table 5.13 show a signicant
improvement over IS-2.
Table 5.14 shows results of applying MFB augmented by approximate forcing to a version of the machine-repairman model with components whose failure times are Weibull
distributed with shape parameter = 2. In order to obtain traditional simulation results
in a reasonable period of time, we altered the scale parameters as well. In the modied
model, type-one components have scale parameter 1 = 30; type-two components have scale
parameter 2 = 15. The IS-4 strategy is shown in Table 5.14 to be eective despite these
signicant changes to the failure distributions of components of the model.
57
Table 5.14: Results for Modied Non-Markov M-R Model
Strategy
Result
Error
Replications
CPU Time (sec.)
Traditional
1:5 10,5
10%
26,524,623
30,344
IS-4
1:7 10,5
10%
152,730
521
size
arrival
queue
service
Figure 5.4: SAN Model of M/M/1/K Queue
5.2 M/M/1/K Queue
This section describes the application of the UltraSAN importance sampling facility to
a simple queuing model. The example is pedagogical; it was chosen to demonstrate the
versatility of the tool. The example looks at the transient behavior of an M/M/1/K queue.
Figure 5.4 shows the SAN model of an M/M/1/K queue with K = 10. Timed activity
arrival models the arrival process, while service models the service time distribution. Ta-
ble 5.15 gives the parameters of the activities. The xed size of the queue is modeled by
input gate size, whose predicate (see Table 5.16) disables arrivals when the queue length
reaches ten.
Table 5.15: Activity Time Distributions for M/M/1/K Queue
Activity Distribution
arrival
service
exponential(0:1)
exponential(1:0)
58
Table 5.16: Input Gate Predicate and Function for M/M/1/K Queue
Gate Enabling Predicate
size
Function
MARK (queue) > 0 && MARK (queue) < 10 identity
Table 5.17: Estimates for Probability Queue Empties in One Time Unit
Technique
Result
Error
Replications
CPU Time (sec.)
Analytical
Traditional
1:03 10,7 1:31 10,7
7:80 10,14 80%
45,450,000
28,223
IS
1:04 10,7
10%
82,961
158
The performance measure chosen for this example is the probability that the queue
becomes empty before a single time unit has passed, given that the queue starts o full.
One way to obtain the desired performance measure is to dene an instant-of-time variable,
Vt , such that Vt = 1 if the queue is empty at time t, and Vt = 0 otherwise. Then, by
making the empty queue marking (i. e. MARK (queue ) == 0) an absorbing marking, one
can examine the probability that the queue becomes empty within a single time unit by
examining the probability that V = 1 at the end of the interval [0; 1]. The empty queue
marking was made an absorbing marking by adding the condition that MARK (queue) > 0
to the predicate of size (see Table 5.16).
Since the average service time per customer is one time unit, the event that all ten
customers are served within one time unit is clearly rare. The queue length of an M/M/1/K
queue is a Markov process, so this model may be solved analytically. Using the UltraSAN
solver for transient instant-of-time variables, it was easy to solve for P fV1 = 1g. As shown
in Table 5.17, the probability that the queue empties this quickly is quite small. In fact, as
one can see from Table 5.17, the event is so rare that traditional simulation fails to yield
a valid condence interval, even after running for over 28,000 CPU seconds. In contrast,
59
Table 5.18: Biased Activity Time Distributions for M/M/1/K Queue
Activity Distribution Parameter
arrival
service
unbiased
exponential
MARK(queue)
Table 5.19: Estimates for Probability Queue Empties in Two Time Units
Technique
Result
Error
Replications
CPU Time (sec.)
Analytical
Traditional
4:07 10,5 4:32 10,5
3:59 10,13 10%
8,906,955
4,347
IS
4:13 10,5
10%
24,910
62
the importance sampling simulation returned a valid estimate within 160 CPU seconds|a
dramatic improvement. All of the results for the M/M/1/K model considered in this section
were obtained on a DECstation 5000/200 with 48MB of main memory.
The strategy employed was quite simple (a single governor state) in this case, but serves
to demonstrate the expressive power of the framework. Through the use of marking dependency, one can obtain a very compact description of an importance sampling heuristic.
As shown in Table 5.18, the service activity has been biased with a marking-dependent
parameter equal to the length of the queue. The heuristic shown in Table 5.15 is fairly intuitive. When examining the problem, one notices that the probability that a few customers
are serviced within the given time is relatively high compared to the probability that the
queue is emptied. A likely path to an empty queue, then, is to have a lot of short service
times in the beginning of the interval and then have the last few occur quickly enough so
that the queue is emptied before the given time. Therefore, one would like to force the
system to a state close to empty, and then allow the system to progress along its most likely
path. To that end, it seems reasonable to apply a fairly heavy bias in the beginning of the
simulation, reducing the biasing level as the queue empties. As shown in Table 5.17, the
60
developed heuristic performs well, obtaining a good estimate of the performance measure
with remarkably little CPU eort.
In an attempt to get better results from traditional simulation, the probability of the
queue emptying within two time units was considered. The hope was that by extending
the time limit, the event that the queue emptied within the given interval would occur
suciently often to obtain a valid estimate with a reasonable amount of computer eort.
As shown in Table 5.19, the traditional simulator performed better for this experiment, but
still required over 4000 CPU seconds to achieve the desired accuracy. On the other hand,
the importance sampling simulation performance was superb, returning the correct result
with very little CPU eort.
61
CHAPTER 6
Conclusions
The main objective of this work was the development of a software tool for experimenting
with importance sampling in discrete event simulation. The objective was achieved by:
1. Developing the new concept of the importance sampling governor, a formal framework
for specifying importance sampling heuristics in terms of the random elements of stochastic activity networks.
2. Adapting a known method for importance sampling simulation of discrete event systems to the simulation of governed SAN-based reward models at the network level.
3. Developing and implementing a user-interface for specifying importance sampling
heuristics via the importance sampling governor.
4. Augmenting existing software for simulating SAN-based reward models to handle the
simulation of governed SAN-based reward models.
5. Integrating the user-interface and simulation engine into an importance sampling facility for UltraSAN, capable of automatically generating a simulation program from
the specication of a governed SAN-based reward model.
The utility of the new software tool was demonstrated through its application to several
example problems. In each case, the tool facilitated the application of importance sampling.
Furthermore, in each case, the CPU time needed to estimate reward variables to within a
specied relative accuracy was reduced by orders of magnitude.
62
6.1 Future Work
The work presented here has made progress toward expanding the role of importance
sampling in discrete event simulation. However, much work remains. Research must now be
done to determine the limitations of known heuristics. Once the limitations are identied,
new strategies for choosing the change of measure must be developed for classes of systems
not supported by known methods.
63
REFERENCES
[1] J. A. Carrasco, \Ecient transient simulation of failure/repair Markovian models," in
Proceedings of the 10th Symposium on Reliable Distributed Systems, pp. 152{161, Pisa,
Italy, September 1991.
[2] M. Cottrell, J.-C. Fort, and G. Malgouyres, \Large deviations and rare events in the
study of stochastic algorithms," IEEE Transactions on Automatic Control, vol. AC-28,
no. 9, pp. 907{920, September 1983.
[3] J. Couvillion, R. Freire, R. Johnson, W. D. Obal II, M. A. Qureshi, M. Rai, W. H.
Sanders, and J. Tvedt, \Performability modeling with UltraSAN," IEEE Software, vol.
8, no. 5, pp. 69{80, September 1991.
[4] M. A. Crane and A. J. Lemoine, An Introduction to the Regenerative Method for
Simulation Analysis, volume 4 of Lecture Notes in Control and Information Sciences,
Springer-Verlag, New York, 1977.
[5] P. W. Glynn, \A GSMP formalism for discrete event systems," Proceedings of the
IEEE, vol. 77, no. 1, pp. 14{23, January 1989.
[6] P. W. Glynn and D. L. Iglehart, \Importance sampling for stochastic simulations,"
Management Science, vol. 35, no. 11, pp. 1367{1392, November 1989.
[7] A. Goyal, W. C. Carter, E. de Souza e Silva, S. S. Lavenberg, and K. Trivedi, \The
system availability estimator," in Proceedings of the Sixteenth Annual International
Symposium on Fault-Tolerant Computing, pp. 84{89, Vienna, Austria, 1986.
[8] A. Goyal, P. Shahabuddin, P. Heidelberger, V. F. Nicola, and P. W. Glynn, \A unied framework for simulating Markovian models of highly dependable systems," IEEE
Transactions on Computers, vol. 41, no. 1, pp. 36{51, January 1992.
[9] J. M. Hammersley and D. C. Handscomb, Monte Carlo Methods, Methuen & Co., Ltd.,
London, 1964.
[10] R. A. Howard, Dynamic Probabilistic Systems. Vol II: Semi-Markov and Decision Processes, Wiley, New York, 1971.
[11] S. Juneja and P. Shahabuddin, \Fast simulation of Markovian reliability/availability
models with general repair policies," in Proceedings of the Twenty-Second Annual
International Symposium on Fault-Tolerant Computing, pp. 150{159, Boston, Massachusetts, July 1992.
[12] H. Kahn, \Random sampling (Monte Carlo) techniques in neutron attenuation problems," Nucleonics, vol. 6, pp. 27{33,36,60{65, May 1950.
64
[13] H. Kahn and M. Marshal, \Methods of reducing sample size in Monte Carlo computations," Journal of the Operations Research Society of America, vol. 1, pp. 263{278,
November 1953.
[14] J. P. C. Kleijnen, Statistical Techniques in Simulation, volume 1, Marcel-Dekker, Inc.,
New York, 1974.
[15] A. M. Law and W. D. Kelton, Simulation Modeling and Analysis, McGraw-Hill, New
York, 1982.
[16] E. E. Lewis and F. Bohm, \Monte Carlo simulation of Markov unreliability models,"
Nuclear Engineering and Design, vol. 77, pp. 49{62, January 1984.
[17] B. D. McLeod and W. H. Sanders, \Performance evaluation of N-processor time warp
using stochastic activity networks," Technical Report PMRL 93-7, The University of
Arizona, March 1993. Submitted for publication.
[18] J. F. Meyer, \On evaluating the performability of degradable computing systems,"
IEEE Transactions on Computers, vol. C-22, pp. 720{731, August 1980.
[19] J. F. Meyer, A. Movaghar, and W. H. Sanders, \Stochastic activity networks: Structure, behavior, and application," in Proceedings of International Workshop on Timed
Petri Nets, pp. 106{115, Torino, Italy, July 1985.
[20] V. F. Nicola, M. K. Nakayama, P. Heidelberger, and A. Goyal, \Fast simulation of dependability models with general failure, repair and maintenance processes," in Proceedings of the Twentieth Annual International Symposium on Fault-Tolerant Computing,
pp. 491{498, Newcastle upon Tyne, United Kingdom, June 1990.
[21] W. D. Obal II and W. H. Sanders, \A framework for importance sampling based on
stochastic activity networks," Technical Report PMRL 93-8, The University of Arizona,
April 1993. Submitted for publication.
[22] W. D. Obal II and W. H. Sanders, \Importance sampling simulation in UltraSAN ,"
Technical Report PMRL 93-10, The University of Arizona, May 1993. Submitted for
publication.
[23] A. Papoulis, Probability, Random Variables, and Stochastic Processes, McGraw-Hill
Publishing Company, New York, second edition, 1984.
[24] S. Parekh and J. Walrand, \A quick simulation method for excessive backlogs in networks of queues," IEEE Transactions on Automatic Control, vol. 34, no. 1, pp. 54{66,
January 1989.
[25] M. A. Qureshi and W. H. Sanders, \Reward model solution methods with impulse
and rate rewards: An algorithm and numerical results," Submitted for publication,
September 1992.
[26] W. H. Sanders and R. S. Freire, \Ecient simulation of hierarchical stochastic activity network models," To appear in Discrete Event Dynamic Systems: Theory and
Applications .
65
[27] W. H. Sanders, L. A. Kant, and A. Kudrimoti, \A modular method for evaluating
the performance of picture archiving and communication systems," Technical Report
PMRL 93-5, The University of Arizona, January 1993. To appear in Journal of Digital
Imaging .
[28] W. H. Sanders and L. M. Malhis, \Dependability evaluation using composed SANbased reward models," Journal of Parallel and Distributed Computing, vol. 15, pp.
238{254, July 1992.
[29] W. H. Sanders and J. F. Meyer, \Reduced base model construction methods for stochastic activity networks," IEEE Journal on Selected Areas in Communications, vol. 9, pp.
25{36, January 1991.
[30] W. H. Sanders and W. D. Obal II, \Dependability evaluation using UltraSAN ," in
Proceedings of the Twenty-Third Symposium on Fault-Tolerant Computing, Toulouse,
France, June 1993.
[31] W. H. Sanders, Construction and Solution of Performability Models Based on Stochastic Activity Networks, PhD thesis, University of Michigan, 1988.
[32] W. H. Sanders and J. F. Meyer, \A unied approach for specifying measures of performance, dependability, and performability," in Dependable Computing for Critical
Applications, A. Avizienis and J. C. Laprie, editors, volume 4 of Dependable Computing and Fault-Tolerant Systems, pp. 215{238, Springer Verlag, Vienna, 1991.
[33] P. Shahabuddin, Simulation and Analysis of Highly Reliable Systems, PhD thesis, Stanford University, 1990.
[34] J. E. Tvedt, Matrix Representations and Analytical Solution Methods for Stochastic
Activity Networks, Master's thesis, The University of Arizona, 1990.
[35] A. P. A. van Moorsel, B. R. Haverkort, and I. G. Niemegeers, \Fault injection simulation: A variance reduction technique for systems with rare events," in Second International Working Conference on Dependable Computing for Critical Applications, pp.
57{64, Tucson, Arizona, February 1991.