International Institute of Information Technology
Hyderabad
Transient Analysis of Toeplitz Type
Continuous Time Markov Chains
(Version 1)
By
Dr. G Rama Murthy
Rakesh Dhanireddy
Vishnu Vardhan
Transient Analysis of Toeplitz Type
Continuous Time Markov Chains
1. Introduction to the problem:
A Queue or a ‘waiting line’, involves arriving items (customers, jobs) that
demand service at a service station, such as incoming telephone calls at a trunk
station or inoperative machines that wait for a repairman for a service.
1.1 Basic Queueing Theory Ideas:
Queueing theory is concerned with the mathematical analysis of systems subject
to demands whose occurrences and lengths can, in general, be specified only
probabilistically. For example, consider a telephone system, whose function is
to provide communication paths between pairs of telephone sets (customers) on
demand. The provision of a permanent communications path between each pair
of telephone sets would be astronomically expensive and hence impossible. In
response to this problem, the facilities needed to establish and maintain a
talking path between a pair of telephone sets are maintained in a common pool,
to be used by a call when required and returned to the pool when no longer
needed. This introduces the possibility that the system will be unable to set up a
call on demand because of a lack of available equipment at that time. Thus the
question immediately arises: How much equipment must be provided so that the
proportion of calls experiencing delays will be below a specified acceptable
level?
Similar questions arise in diverse situations like number of beds required in a
hospital, number of cabs required in a city etc. In such problems, the times at
which requests for service will occur and the lengths of times that these requests
will occupy facilities cannot be predicted except in a statistical sense.
Queueing theory is a branch of applied probability theory which hopes to solve
these problems by developing mathematical models from which, it is possible to
abstract information whose analysis yields rough answers to questions like those
of above stated problems.
1.2 Parameters for describing a Queue:
If the server is busy with another item, the newly arrived items form a waiting
line until the server is free, or they may get impatient and leave the system with
or without waiting for the service. The queue so formed can be described by the
arrival (input) process, the queue discipline, and the service mechanism. The
queue discipline determines the manner in which arriving items form a queue
and behave while waiting. The input process and service mechanism are
specified by the characteristics of the interarrival times and service times.
1.3 Mathematical modeling of Queues:
A wide variety of queueing phenomena can be modeled as continuous time
Markov chains. A continuous time Markov chain
can occupy randomly a
finite or infinite number of states
at time is described by
at time . The status of the process
and it equals the state
at that time. Suppose that the process
is in state
that the process occupies
at time
process, the probability that the process goes into the state
. For a Markov
at time
is
given by
And this probability is independent of the behavior of the process
the instant
prior to
.
Sojourn Time:
All Markov processes share this interesting property that the time it takes for a
change of state (sojourn time) is an exponentially distributed random variable.
To see this, let represent the waiting time for a change of state for a Markov
process
, given that it is in state
be in the same stte
at time
at time
as at
. If
, then the prcess will
, and (being a Markov process) its
subsequent behavior is independent of . Hence
It can be easily deduced from above relation that
Or
The only function which satisfies above relation for arbitrary and is either of
the form
, where is a constant or unbounded in every interval. Thus
Or
Which shows that the sojourn time (waiting time in any state) has an
exponential distribution for all Markov processes.
The Kolmogorov Equations:
Let
for a homogeneous Markov process denote the probability that the
process changes from a state
to
after a time .
Let
here
denote the transition densities of the process. They are instrumental in
determining the evolution of the queue as shown below.
Let
and
denote the corresponding matrices.
is
generally referred to as ‘Generator matrix’. All the diagonal entries of
can be
shown to be negative, all other elements of each row being positive and sum of
all the elements of each row equals zero.
It can be shown using above equations and a bit of analysis that the time
evolution of Markov chain satisfies the following relations called Kolmogorov
equations:
Under the initial condition
.
For a finite state process
, the transient solution of the above
equation takes the form
Where
Explicit solutions for
in terms of
s are often difficult except in simple
situations.
We hope to solve above problem by employing the methods of Laplace
Transform.
2. Solution of Kolmogorov Equations by Laplace Transform based
Method:
Applying Laplace Transform to the Kolmogorov equation, we obtain the
following set of equations:
The final equation expresses the unknown
Thus
in terms of other matrices.
can be easily calculated once the inverse of the matrix
is
computed. Thus the problem now effectively reduces to the computation of
inverse of the matrix
After obtaining the inverse, we can find the
inverse Laplace form of
to obtain the desired
3. Computing the inverse of the matrix (
.
):
It is known that the standard procedure for the calculation of inverse of a matrix
takes
calculations. We aim to decrease the complexity of the calculation
by taking advantage of certain symmetry properties in the structure of the
Generating Matrix .
For example, the matrix
may turn out to be Symmetric Toeplitz matrices
under certain symmetry considerations, in which case more efficient algorithms
exist for the computation of the inverse of the matrix
.
A matrix
within bounds.
is said to be a Toeplitz matrix iff
for all
This matrix might arise in studying queueing systems under considerations of
symmetry.
For example,
As seen above, all the elements along any diagonal parallel to the principal
diagonal must be equal. A symmetric Toeplitz matrix is one which is both
symmetric and has above property.
The computational complexity in the case of Symmetric Toeplitz systems
reduces to
.
But owing to the properties inherent to a Generator matrix, it can’t be exactly
symmetric Toeplitz, as illustrated below:
As seen in the above example, due to the constraint that the sum of elements in
each row should equal to zero, the Toeplitz condition is violated on the principal
diagonal. Henceforth, we shall call such matrices symmetric ‘Toeplitz-like’
matrices.
We shall develop a method in the following to apply the Toeplitz algorithm
even in case of ‘Toeplitz-like’ matrices.
We decompose
as follows:
Where
is a symmetric Toeplitz part and
and
provide for the remaining part of the matrix. Here
are column vectors which
is the order of the square
matrix .
We will then apply the following formula recursively to obtain the inverse of
.
To evaluate the inverse of the Toeplitz symmetric part (
, we use Levinson’s
algorithm. Afterwards, above formula is repeatedly applied
times to
obtain the final inverse of
We illustrate the above procedure with the following example:
[
+
As shown above the Toeplitz-like matrix, is splitted into a symmetric Toeplitz
part and sum of products of column and row vectors.
The computational complexity of above procedure can be shown to be much
less than the original
.
4. References:
[1] G. Rama Murthy et.al , “Transient Performance Evaluation of Toeplitz Type
Markov Chains, “ Poster paper presented at International Symposium on Information
Theory (ISIT-2004), Chicago.
[2] G. Rama Murthy,”Transient and Equilibrium Analysis of Computer Networks: Finite
Memory and Matrix Geometric Recursions,” Ph.D thesis, Purdue University, West
Lafayette, USA.