Parameter Estimation in Queueing
Systems Using Partial Information
I. V. Basawa
Department of Statistics
University of Georgia
Athens, Georgia 30602, USA
U. N. Bhat
Department of Statistical Science and Operations Research
Southern Methodist University
Dallas, Texas, U.S.A. 75275
J. Zhou
Mathematical Biosciences Institute
Ohio State University
Columbus, Ohio 43212, USA
Abstract
Waiting time and system time (waiting plus service time) data,
adjusted for idle times when necessary, are used for estimating the parameters in GI/G/1 queues. The sampling plans presented use incomplete information on the differences between service and inter-arrival
times rather than full information on service and inter-arrival times.
A variation of the EM algorithm is proposed to derive parameter estimates. Specific examples are discussed and the performances of the
maximum likelihood and the corresponding estimates obtained by the
EM algorithm are compared via simulations.
Keywords: GI/G/1 queues; parameter estimation; maximum likelihood
method; waiting times; system times; EM algorithm; exponential family of
distributions.
1
Introduction
Parameter estimation for single server queues has been studied by several
researchers in the recent literature. Various methods of estimation including
the method of moments (Basawa and Prabhu, 1981), maximum likelihood
(Basawa and Prabhu, 1988), Bayes (McGrath and Singpurwalla, 1987), empirical Bayes (Thiruvaiyaru and Basawa, 1992), and sequential estimation
1
(Basawa and Bhat, 1992) have been studied. See also, Bhat and Rao 1987.
Most of the previous works on parameter inference for queues require complete observation of all customer interarrival and service times. Unfortunately, the complete information on all interarrival times and service times
is frequently not available in practice. There is, therefore, a need for developing methods that use only partial information which is easy to collect.
In this paper, we propose two such sampling plans which are based on only
waiting times and system times data (see Section 2 for details). We show
that the waiting times and system times can be expressed in terms of differences of service times and interarrival times. The likelihood function is
based on independent and identically distributed random variables which
are convolutions of the service times and waiting times. This novel approach leads to a simple estimation method studied in this paper. Basawa
et al. (1996) presented a maximum likelihood (ML) method for estimating the parameters of the arrival and service time distributions using only
the information on the waiting times of customers in a GI/G/1 queue with
“first come first served” queue discipline. In a followup paper Basawa et
al. (1997) used an estimating functions approach for the same problem. In
both papers, the observations used are the waiting times (equivalent to the
workload, under the assumption of the“first come first served” discipline)
of a sequence of arriving customers. The likelihood function and estimating
functions in the above two papers are based on the Markov chain property
of {Wn , n = 1, 2, . . . , } where Wn is the waiting time of the nth arriving
customer. In this paper we show that by adding data on the lengths of
idle times during the observation period a set of independent and identically distributed sample observations can be derived from the original set,
thus simplifying resulting expressions for the likelihood and the estimating
functions.
As an alternative to using the waiting times in the sampling plan, we
show that we can also use the time a customer has spent in the system at
the completion of service, i.e. the system time. In some practical examples,
it may be easier to collect data on the amount of time spent in the system
in waiting and being served than on waiting alone. For instance, if data
collection can occur only at the points of entry and exit it is impossible to
get information on when the service starts.
The remaining sections of this paper are organized as follows. Section
2 describes the procedure by which information on the difference between
appropriate paired values of service and inter-arrival times are derived using waiting times and systems times, with adjustments incorporating idle
times, when necessary. The sampling plans and the methods of estimation
of parameters of the component distributions are discussed in Section 3.
More specifically, we propose a variation of the EM algorithm (Dempster
et al. (1977)) for estimating the parameters in GI/G/1 queues. Section 4
contains three specific examples, viz., M/M/1 (Poisson arrival and exponen2
tial service), M/Ek /1 (Poisson arrival and Erlangian service), and Ek /W/1
(Erlangian arrival and Weibull service). In Section 5, we compare via simulations, the ML estimates determined by the standard Newton-Raphson
method and the corresponding estimates obtained by the EM algorithm, for
the three examples discussed in Section 4.
2
Waiting Time and System Time Processes
Let Wn denote the waiting time of the nth customer in a GI/G/1 queue
with a“first come first served” queue discipline. Let Vn be the service time
of the nth customer and Un the inter-arrival time between the (n − 1)th
and the nth customer. We assume {Vn } and {Un } are two independent sets
of independent and identically distributed (i.i.d) random variables. It is
well known that {Wn , n = 1, 2, 3, . . .} is a Markov chain with the transition
structure:
½
Wn + Vn − Un+1 if Wn + Vn − Un+1 > 0
Wn+1 =
(2.1)
0
if Wn + Vn − Un+1 ≤ 0.
See, for instance, Lindley (1952) and Prabhu (1965).
Also define In as the length of the idle time preceding the nth arrival,
if such an arrival finds the system empty. Following the sample path of the
process, we may write
Wn + Vn − Un+1 > 0 ⇒ Wn+1 = Wn + Vn − Un+1
Wn + Vn − Un+1 ≤ 0 ⇒ Wn+1 = Wn + Vn − Un+1 + In+1 .
(2.2)
Xn = Vn − Un+1 .
(2.3)
Let
Since {Vn } and {Un } are i.i.d. random variables, their differences {Xn , n =
1, 2, . . .} are also i.i.d. random variables. From (2.2) and (2.3), we have
½
Wn+1 − Wn if Wn+1 > 0
Xn =
(2.4)
−Wn − In+1 if Wn+1 = 0.
Let Yn denote the time the nth customer spends waiting and getting
served in the system (system time) in a GI/G/1 queue with Vn and Un , the
service time and inter-arrival time, and the queue discipline “first come first
served”, as described above. The process {Yn , n = 1, 2, 3, . . .} is defined by
Yn = Wn + Vn . Thus, adding Vn+1 to both sides of (2.1), we get
½
Yn + Vn+1 − Un+1 if Yn − Un+1 > 0
Yn+1 =
(2.5)
Vn+1
if Yn − Un+1 ≤ 0.
3
Following the sample path of the process, we may write
Yn − Un+1 > 0 ⇒ Yn+1 = Yn + Vn+1 − Un+1
Yn − Un+1 ≤ 0 ⇒ Yn+1 = Yn + Vn+1 − Un+1 + In+1 .
(2.6)
Zn = Vn+1 − Un+1 .
(2.7)
Let
Clearly {Zn , n = 1, 2, . . .} are i.i.d. random variables. Re-arranging (2.6)
we get
½
Yn+1 − Yn
if Yn − Un+1 > 0
Zn =
(2.8)
Yn+1 − Yn − In+1 if Yn − Un+1 ≤ 0.
Thus if we observe either the {Wn } process or the {Yn } process and
the lengths of corresponding idle times, we can construct i.i.d. sequences
{Xn } and {Zn } via (2.4) and (2.8). Note that Xn and Zn are differences of
service time and interarrival time. We shall use {Xn } and {Zn } as a basis
to construct the likelihood function.
3
Maximum Likelihood Estimation
3.1
Sampling Plans
The data collection procedure would dictate whether we use waiting time or
system time data for parameter estimation. For using waiting times, they
should be recorded at the time the customers enter service. For using system
time data, they should be recorded when customers leave the system. Under
both procedures, the lengths of idle times should also be recorded, in association with the customer number. Corresponding to these two processes,
therefore, we have the following two sampling plans.
Sampling Plan 1 (Waiting times):
Record
1. the waiting time of customers when they enter service for a specified
number N of customers in sequence; and
2. the length of idle times (when there are no customers in the system)
occurring during that period.
Let wn be the waiting time of the nth customer and γn be the idle time,
if any, just before the nth customer arrival. The sample observations are:
(w1 , w2 , . . . , wn ) and γn , n = 1, 2, . . . , N , but only when idle times occur.
The sample (x1 , x2 , . . . , xN ) of observations is now obtained using Eq. (2.4).
Sampling Plan 2 (System times):
Record
4
1. the time in system for customers at the time of their service completion, until a specified number N of customers complete their service;
and
2. the length of idle times occurring during that period.
Let yn be the system time of the nth customer and γn the idle time,
if any, just before the nth customer arrival. The sample observations are:
(y1 , y2 , . . . , yN ) and γn , n = 1, 2, . . . , N , but only when idle times are present.
The sample (z1 , z2 , . . . , zN ) of observations is now obtained using Eq. (2.8).
Since Xn and Zn have the same distribution, from here on we use X =
V − U , in a generic notation, as the basic random variable and work with
the sample (x1 , x2 , . . . , xN ).
3.2
Exponential Family of Distributions
Suppose that the probability density functions of U and V belong to exponential families of distributions. More specifically, let
fU (u) = h1 (u) exp[T1 (u)θ − K1 (θ)], and fV (v) = h2 (v) exp[T2 (v)φ− K2 (φ)]
(3.1)
where T1 (u) and T2 (v) are the sufficient statistics, and the functions K1 (θ)
and K2 (φ) are such that
Z ∞
Z ∞
fU (u)du = 1 and
fV (v)dv = 1.
0
0
From the properties of the exponential families, we have
E(T1 (U )) =
E(T2 (V )) =
dK1 (θ)
d2 K1 (θ)
, V ar(T1 (U )) =
,
dθ
dθ2
d2 K2 (φ)
dK2 (φ)
, and V ar(T2 (V )) =
.
dφ
dφ2
(3.2)
Examples of distributions belonging to the non-negative exponential family
include exponential, gamma, beta, lognormal, Pareto, Weibull and inverseGaussian among others. These examples are presented below.
5
Distributions of Exponential Family with Parameter λ
Distribution
Exponential
Density Function
λe−λx
λ(λx)k−1 e−λx
(k
Γ(k)
λ−1
λx
Gamma
Beta (λ, 1)
Lognormal
√1
x 2π
Pareto
Weibull
Inverse Gaussian
3.3
known)
exp(−(log x − λ)2 /2)
λaλ
xλ+1
(a known)
b−1 exp(−λxb ) (b known)
λbx
q
2
λ − 32
exp(− λ(x−µ)
) (µ known)
2π x
2µ2 x
T (x)
−x
K(λ)
− log λ
−x
log x
log x
−k log λ
− log λ
λ2 /2
− log x
−xb
− log λ − λ log a
− log λ
1
−( 2x
+
x
)
2µ2
− µλ − 12 log λ
The Likelihood Function and Estimating Equations
Let X = V − U , where the density functions of U and V are given by (3.1).
The density function of X is then seen to be
½ R∞
f (u)fV (x + u)du, if x ≥ 0
fX (x) = R0∞ U
(3.3)
f
U (v − x)fV (v)dv, if x < 0.
0
We can write
fX (x) = (g1 (x))I(x) (g2 (x))1−I(x) ,
− ∞ < x < ∞,
(3.4)
where
Z
g1 (x) =
0
Z
∞
fU (u)fV (x + u)du, g2 (x) =
and
½
I(x) =
∞
fU (v − x)fV (v)dv,
0
(3.5)
1, if x ≥ 0
0, if x < 0.
Suppose (x1 , x2 , . . . , xN ) is a random sample from fX (x). The likelihood
function based on the sample (x1 , . . . , xN ) is then given by
LN (θ, φ) =
N
Y
i=1
N
N
Y
Y
I(xi )
fX (xi ) = ( (g1 (x1 ))
)( (g2 (xi ))1−I(xi ) ).
i=1
(3.6)
i=1
The maximum likelihood (ML) estimates θ̂ and φ̂ based on the sample
(x1 , . . . , xN ) are then obtained as solutions of the likelihood equations
∂ log LN
∂ log LN
= 0 and
= 0.
∂θ
∂φ
6
We first note that
Z
∂g1 (x)
∂θ
∞
=
(
Z0 ∞
=
0
where K10 (θ) denotes
Z
∂g1 (x)
∂φ
∂g2 (x)
∂θ
and
∂K1 (θ)
∂θ .
∞
=
0
Z
∞
=
0
∂g2 (x)
=
∂φ
Z
0
∂fU (u)
)fV (x + u)du
∂θ
(T1 (u) − K10 (θ))fU (u)fV (x + u)du,
(3.7)
Similarly, we have
(T2 (x + u) − K20 (φ))fU (u)fV (x + u)du,
(3.8)
(T1 (v − x) − K10 (θ))fU (v − x)fV (v)dv,
(3.9)
∞
(T2 (v) − K20 (φ))fU (v − x)fV (v)dv.
(3.10)
From (3.6), we have
N
∂ log LN (θ, φ) X
=
I(xi )
∂θ
à ∂g
i=1
1 (xi )
!
∂θ
g1 (xi )
N
X
+
(1 − I(xi ))
i=1
à ∂g
2 (xi )
∂θ
g2 (xi )
!
, (3.11)
and




∂g1 (xi )
∂g2 (xi )
N
N
X
∂ log LN (θ, φ) X
∂φ 
∂φ 
=
I(xi ) 
+
(1 − I(xi )) 
. (3.12)
∂φ
g1 (xi )
g2 (xi )
i=1
i=1
Substituting (3.7)-(3.10) in (3.11) and (3.12), we finally get
∂ log LN (θ, φ)
∂θ
µR ∞
¶
(T1 (u) − K10 (θ))fU (u)fV (xi + u)du
R∞
=
I(xi )
0 fU (u)fV (xi + u)du
i=1
µR ∞
¶
N
X
xi ) − K10 (θ))fU (v − xi )fV (v)dv
0 (T1 (v −
R
,
+
(1 − I(xi ))
∞
0 fU (v − xi )fV (v)dv
i=1
N
X
0
(3.13)
and
∂ log LN (θ, φ)
∂φ
µR ∞
¶
(T2 (xi + u) − K20 (φ))fU (u)fV (xi + u)du
R∞
0 fU (u)fV (xi + u)du
i=1
¶
µR ∞
N
X
− K20 (φ))fU (v − xi )fV (v)dv
0 (T2 (v)
R∞
.
+
(1 − I(xi ))
0 fU (v − xi )fV (v)dv
i=1
=
N
X
I(xi )
0
(3.14)
7
The likelihood equations are then seen to be
K10 (θ)
N
N
1 X
1 X
0
=
Z1 (xi , θ, φ), and K2 (φ) =
Z2 (xi , θ, φ),
N
N
i=1
(3.15)
i=1
where
µR ∞
¶
µR ∞
¶
0 RT1 (u)fU (u)fV (x + u)du
0 TR1 (v − x)fU (v − x)fV (v)dv
Z1 (x, θ, φ) = I(x)
+(1−I(x))
,
∞
∞
0 fU (u)fV (x + u)du
0 fU (v − x)fV (v)dv
and
µR ∞
Z2 (x, θ, φ) = I(x)
0
¶
µR ∞
¶
T2 (v)fU (v − x)fV (v)dv
T2 (x + u)fU (u)fV (x + u)du
0
R∞
R∞
+(1−I(x))
.
0 fU (u)fV (x + u)du
0 fU (v − x)fV (v)dv
The ML estimates (θ̂, φ̂) are obtained by solving the two likelihood equations in (3.15). We propose two methods viz. the EM algorithm and the
Newton-Raphson iteration for solving the equations in (3.15). These methods are outlined below. Under standard regularity conditions (see, for instance, Lehmann (1999), Ch 7), (θ̂, φ̂) are consistent and asymptotically
normal.
3.4
EM Algorithm
Suppose the complete sample {(ui , vi ), i = 1, . . . , N } is available. The complete likelihood based on this sample is
N
N
Y
Y
Lc (θ, φ) = ( fU (ui ))( fV (vi )).
i=1
(3.16)
i=1
The likelihood equations corresponding to the complete likelihood Lc (θ, φ)
are given by
K10 (θ)
N
N
1 X
1 X
0
=
T1 (ui ), and K2 (φ) =
T2 (vi ).
N
N
i=1
(3.17)
i=1
However, the observed sample is (xi , i = 1, . . . , N ) where xi = vi − ui .
The EM algorithm involves replacing T1 (ui ) and T2 (vi ) in (3.17) by the
conditional expectations E(T1 (Ui )|Xi = xi ) and E(T2 (Vi )|Xi = xi ). We
now proceed to evaluate these conditional expectations. Using (3.3), we
have
 R∞
 0 R T∞1 (u)fU (u)fV (x+u)du ,
for x ≥ 0
R ∞ 0 fU (u)fV (x+u)du
E(T1 (U )|X = x) =
(3.18)
 0 TR1∞(v−x)fU (v−x)fV (v)dv , for x < 0.
f (v−x)f (v)dv
0
U
V
8
Similarly,


E(T2 (V )|X = x) =

R∞
(x+u)fU (u)fV (x+u)du
0 T
R2∞
,
R ∞ 0 fU (u)fV (x+u)du
0 R T2 (v)fU (v−x)fV (v)dv
,
∞
0 fU (v−x)fV (v)dv
for x ≥ 0
for x < 0.
(3.19)
The EM algorithm then reduces to solving the equations
N
N
1 X
1 X
0
E(T1 (U )|Xi = xi ), and K2 (φ) =
E(T2 (V )|Xi = xi )
=
N
N
i=1
i=1
(3.20)
for (θ, φ) iteratively. Substituting (3.18) and (3.19) in (3.20), we note that
the equations in (3.20) reduce to the exact likelihood equations in (3.15)
based on the observed sample (x1 , . . . , xN ). Hence, in our problem the EM
algorithm, by itself, does not lead to any simpler solution. We propose the
following variation of the EM algorithm. The method basically involves
replacing E(T1 (U )|X = x) and E(T2 (V )|X = x) in each iteration of the
EM algorithm by their approximations obtained by simulations. This is
described below.
K10 (θ)
Step 1. Let (θ0 , φ0 ) be some starting values of (θ, φ). Typically, one can
use moment estimates based on (x1 , . . . , xN ) as the starting values. Simulate
i.i.d. observations, (u0j , j = 1, . . . , m) from fU (u; θ0 ) and (vj0 , j = 1, . . . , m)
from fV (v; φ0 ).
Compute the simulation approximations for the conditional expectations
in (3.18) and (3.19) as follows.
 Pm
0
0

j=1 T1 (uj )fV (xi +uj ;φ0 )

Pm
, for xi ≥ 0
0
j=1 fV (xi +uj ;φ0 )
Pm
Ê(T1 (U )|X = xi ) =
(3.21)
0
0
T1 (vj −xi )fU (vj −xi ;θ0 )

 j=1Pm
,
for
x
<
0.
i
0
f (v −x ;θ )
j=1 U
j
i
0
Similarly,



Ê(T2 (V )|X = xi ) =


Pm
T2 (xi +u0j )fV (xi +u0j ;φ0 )
Pm
,
f (x +u0 ;φ )
Pm j=1 V0 i 0j 0
j=1 T2 (vj )fU (vj −xi ;θ0 )
Pm
,
0
j=1 fU (vj −xi ;θ0 )
j=1
for
xi ≥ 0
for xi < 0.
(3.22)
Find (θ1 , φ1 ) by solving
K10 (θ)
N
N
1 X
1 X
0
=
Ê(T1 (U )|Xi = xi ), and K2 (φ) =
Ê(T2 (V )|Xi = xi ).
N
N
i=1
i=1
(3.23)
Step 2. Use (θ1 , φ1 ) obtained in Step 1 as starting values and repeat Step
1 to get (θ2 , φ2 ). Repeat the process until convergence.
We now describe the Newton-Raphson method.
9
3.5
Newton-Raphson Algorithm
This is a standard iterative method of solving the likelihood equations
∂ log LN (θ, φ)
∂ log LN (θ, φ)
= 0 and
=0
∂θ
∂φ
where LN (θ, φ) is defined by (3.6). Given the solution (θi , φi ) in the ith
iteration, the solution (θi+1 , φi+1 ) in the next iteration is obtained by
Ã
!
¶
¶ µ
µ
∂ log LN (θ,φ)
θi+1
θi
−1
∂θ
=
+ JN
,
(3.24)
∂ log LN (θ,φ)
φi+1
φi
∂φ
where
Ã
JN = −
∂ 2 log LN
∂θ2
∂ 2 log LN
∂φ∂θ
∂ 2 log LN
∂θ∂φ
∂ 2 log LN
∂φ2
!
.
(3.25)
The gradients in (3.24) and the Hessian in (3.25) are evaluated at the
current estimates (θi , φi ). The iterations are continued until convergence.
Both the first and second derivatives of log LN involve certain integrals which
may be approximated by appropriate sums based on simulations. Moment
estimates can be used as starting values (θ0 , φ0 ) in (3.24).
4
Illustrative Examples
Example 1. M/M/1 Queue
Consider a single server queueing system with Poisson arrivals and exponential service times. Assume that service times are i.i.d. random variables,
generically represented as V , with the density function
fV (v) = µe−µv ,
v > 0.
Let the inter-arrival times be similarly denoted as U with the density function
fU (u) = λe−λu , v > 0.
We illustrate below the derivation of estimators for λ and µ using the method
of maximum likelihood.
The sampling plans in Section 3 result in a sample (x1 , x2 , . . . , xN ), where
X = V − U is the basic random variable. The distribution of X can be
obtained as
(
λµeλx
−∞ < x < 0
λ+µ
fX (x) =
(4.1)
λµe−µx
0 < x < ∞.
λ+µ
10
Then the likelihood function for the sample can be obtained as
L = ΠN
i=1 fX (xi ) = (
N
N
i=1
i=1
X
X
λµ N
) exp[λ
xi (1 − I(xi )) − µ
xi I(xi )]
λ+µ
Let
N
X
xi (1 − I(xi )) = −S1
i=1
N
X
xi I(xi ) = S2 .
i=1
Then
µ
L=
λµ
λ+µ
¶N
e−λS1 e−µS2 .
(4.2)
We have
L = ln L = N ln(λµ) − N ln(λ + µ) − λS1 − µS2 .
(4.3)
Differentiating with respect to λ and µ and equating to zero in the usual
manner, we get the likelihood equations
N
N
−
− S1 = 0
λ
λ+µ
N
N
−
− S2 = 0.
µ
λ+µ
(4.4)
(4.5)
The equations (4.4) and (4.5) lead to explicit solutions (ML estimates)
µ̂ =
N
√
,
S2 + S1 S2
(4.6)
λ̂ =
N
√
.
S1 + S1 S2
(4.7)
and
Thus, for the M/M/1 queue, we have the explicit expressions for the ML
estimates of λ and µ based on xi = vi − ui , i = 1, . . . , N .
Example 2. M/Ek /1 Queue
Consider an M/Ek /1 queue with
fU (u) = λe−λu , and fV (v) =
µ(µv)k−1 e−µv
.
(k − 1)!
The density of X = V − U is then given by
µ
¶
λµk
λx
fX (x) =
e
(1 − Gk (x; λ + µ))I(x) ,
(λ + µ)k
11
(4.8)
(4.9)
where I(x) is the indicator function of the set {x ≥ 0}, and Gk (x; λ+µ) is the
cumulative distribution function of a gamma distribution with parameters
(k, λ + µ), viz.,
Z
0
x
e−(λ+µ)y (λ + µ)k y k−1
dy = Gk (x; λ + µ).
(k − 1)!
(4.10)
Note that for k = 1, Gk (x; λ + µ) = e−(x+µ)x , and (4.9) will then reduce to
(4.1). For the general k the likelihood function is given by
µ
LN (λ, µ) =
λµk
(λ + µ)k
¶N
e
λ
PN
i=1
xi
N
Y
(1 − Gk (xi ; λ + µ)I(xi ) .
(4.11)
i=1
Due to the presence of the incomplete gamma function Gk (·) in the likelihood function, explicit expressions for the ML estimates of λ and µ are
not available. However, the ML estimates can be obtained by the iterative
methods discussed in Section 3. See Section 5.
Example 3. Ek /W/1 Queue
Suppose the inter-arrival time and service time densities are
fU (u) =
λ(λu)k−1 e−λu
, and fV (v) = µbv b−1 exp(−µv b ),
(k − 1)!
(4.12)
where b > 0 is a known constant i.e., U and V have gamma and Weibull
distributions, respectively.
The density function of X = V − U is then given by (3.4), viz.,
fX (x) = (g1 (x))I(x) (g2 (x))1−I(x) ,
where
λk µb
g1 (x) =
(k − 1)!
Z
∞
uk−1 (x + u)b−1 exp(−λu − µ(x + u)b ))du
0
and
g2 (x) =
λk µb
(k − 1)!
Z
∞
(v − x)k−1 v b−1 exp(−λ(v − x) − µv b )dv.
0
Since the integrals g1 (x) and g2 (x) do not have explicit expressions (as functions of λ and µ), the ML estimates of λ and µ are not available in closed
form. See Section 5 for simulation results and the implementation of the
methods discussed in Section 3.
12
5
Simulation Results
We chose the following three examples discussed in Section 4 for our simulation study.
Ex 1.
Ex 2.
Ex 3.
Inter-Arrival Time (U )
Exponential (λ)
Exponential (λ)
Gamma (k, λ)
Service Time (V )
Exponential (µ)
Gamma (k, µ), k known
Weibull (b, µ), b known
The main goal of the simulation study is to implement the two iterative
methods of computation of ML estimates of λ and µ, viz., the EM algorithm
and the Newton-Raphson method. Since for Ex 1 explicit expressions for the
ML estimates of λ and µ are available, the approximate iterative methods
are not needed. However, it is instructive to see how the iterative methods
perform when we actually know the true ML estimates.
For each of the three examples, we generated N observations X1 , X2 , . . . , XN ,
with N = 100, 500 and 1000, and λ = 1, µ = 2. The estimates of λ and
µ were computed using the EM algorithm (EM) and the Newton-Raphson
iteration (NR). For the simulation approximations in (3.21) and (3.22), we
used m = 100. In each case, the starting values (λ0 , µ0 ) were chosen to
be appropriate moment estimates. The moment estimates are obtained by
equating means and variances of X to their sample versions, in the usual
way. The process was repeated 1000 times to get 1000 values of the
√ estimates
λ̂ and µ̂. The empirical means and root-mean squared-errors ( M SE) of
the estimates are reported in Tables 1-3 below. From these results one can
conclude that
(i) both EM and NR methods give reasonably accurate estimates of λ and
µ;
√
(ii) when N increases the bias and the M SE decrease for both the methods;
√
(iii) in terms of means and M SE both methods give similar results;
(iv) it may be noted that for Ex 1, both EM and NR methods provide
close approximations to the direct ML
√ estimates; however, the true
ML estimates have smaller bias and M SE.
13
√
M SE
Mean
N
EM
(mle)
NR
EM
(mle)
NR
EM
(mle)
NR
100
500
1000
λ
1.029991
(1.015722)
1.030000
1.005455
(1.002846)
1.005462
1.002635
(1.001561)
1.002643
µ
2.132851
(2.05655)
2.132888
2.022833
(2.009136)
2.022863
2.011723
(2.006212)
2.100751
λ
0.1313489
(0.116161)
0.1313559
0.05789259
(0.05255154)
0.05789928
0.03991805
(0.03573444)
0.0399145
µ
0.4324137
(0.3063811)
0.4324435
0.1674528
(0.1270879)
0.1674835
0.1156555
(0.09089903)
0.1156285
Table 1: Exponential inter-arrival and Exponential service
√
M SE
Mean
N
100
500
1000
EM
NR
EM
NR
EM
NR
λ
1.085287
1.072953
1.010113
1.009731
0.9937589
1.008002
µ
2.041499
2.041476
2.009331
2.00933
2.003445
2.003474
λ
0.1727705
0.1638515
0.05764924
0.05754068
0.03872507
0.04013084
µ
0.2307299
0.2307177
0.1015739
0.1015737
0.06791045
0.06791019
Table 2: Exponential inter-arrival and Gamma service (k = 2)
√
M SE
Mean
N
100
500
1000
EM
NR
EM
NR
EM
NR
λ
1.01695
1.016971
1.003071
1.00309
1.001320
1.001355
µ
2.572542
2.572861
2.086351
2.086725
2.040944
2.041391
λ
0.1073067
0.1073165
0.04726319
0.04728129
0.03290075
0.03291029
µ
2.141328
2.141373
0.4702347
0.4708177
0.3296573
0.3296871
Table 3: Gamma inter-arrival (k = 2) and Weibull service (b = 2)
Whereas both EM and NR methods give estimates with similar performance, it must be noted that the NR method involves computation of the
Hessian matrix at each step and the Hessian involves several integrals which
need to be approximated via simulation. For this reason, for computational
simplicity and economy, we recommend the EM method. Since we have
used Monte Carlo (MC) approximations of various integrals occurring in
14
the standard EM and NR iterations, it may be more appropriate to use the
terms MCEM and MCNR for the methods discussed in this paper. For a
background on MCEM, see also Tanner (1993).
Acknowledgements
J. Zhou’s work is supported by NSF grant DMS–112050 (to the MBI).
15
References
Basawa, I. V. and Bhat, B. R. (1992), Sequential inference for single server
queues, in: Queueing and Related Models, eds. U. N. Bhat and I. V.
Basawa (Oxford Univ. Press, Oxford), 325-336.
Basawa, I. V. and Prabhu, N. U. (1981), Estimation in single server queues,
Nav. Res. Logist. Quart, 28, 475-487.
Basawa, I. V. and Prabhu, N. U. (1988), Large sample inference from single
server queues. Queueing Systems 3, 289-306.
Basawa, I. V., Bhat, U. N. and Lund, R. (1996), Maximum likelihood estimation for single server queues from waiting time data, Queueing Systems,
24, 155-167.
Basawa, I. V., Lund, R. and Bhat, U. N. (1997). Estimation function
methods of inference for queueing parameters, Selected Proceedings of
the Symposium on Estimating Functions (Eds. I. V. Basawa, V. P. Godambe, and R. L. Taylor), Institute of Mathematical Statistics, Lecture
Notes – Monograph Series, Vol. 32, 269-284.
Bhat, U. N. and Rao, S. S. (1987). Statistical analysis of queueing systems,
Queueing Systems 1, 217-247.
Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood
from incomplete data via the EM algorithm (with discussion). Journal
of Royal Statist. Soc. B, 39, 1-38.
Lehmann, E. L. (1999). Elements of Large-Sample Theory. Springer, New
York.
Lindley, D. V. (1952). Theory of queues with a single server. Proc. Camb.
Phil. Soc. 48, 449-461.
McGrath, M. R. and Singpurwalla, N. D. (1987). A subjective Bayesian
approach to the theory of queues, Queueing Systems 1, 335-353.
Prabhu, N. U. (1965). Queues and Inventories, Wiley, New York.
Tanner, M. A. (1993). Tools for Statistical Inference, Springer, New York.
Thiruvaiyaru, D. and Basawa, I. V. (1992). Empirical Bayes estimation for
queueing systems and networks, Queueing Systems 11, 179-202.
16