Chapter 8
Queuing Theory in Continuous Time
We have learned about discrete-time Markov Chains (DTMCs) and discrete-time queueing systems
in Chapter 3. In this chapter, we will look at continuous-time queueing systems, including the
M/M/1 queue, the M/GI/1 queue, the G/G/1 queue and the Jackson network. As in the case
of discrete-time queueing systems, continuous-time queueing systems will be analyzed by relating
them to Markov chains. For this purpose, we will first introduce continuous-time Markov chains
(CTMCs) in this chapter. Then, we will introduce and study simple queueing models. Finally,
we will relate continuous-time Markov chains and queueing theory to the study of connection-level
stability in the Internet, distributed admission control, telephone networks and peer-to-peer file
sharing protocols.
8.1
Mathematical Background: Continuous-Time Markov Chains
A stochastic process {X(t)} is a continuous-time Markov chain (CTMC) if the following conditions
hold:
(i) t belongs to some interval of R,
(ii) X(t) 2 S, where S is a countable set, and
(iii) Pr (X(t + s)|X(u), u  s) = Pr(X(t + s)|X(s)), i.e., the conditional probability distribution
at time t + s given the complete history up to time s, depends only on the state of the process
at time s.
A CTMC is said to be time-homogeneous if Pr(X(t + s)|X(s)) is independent of s, i.e., the
conditional probability distribution is time-independent. Define i (t) to be the amount of time the
process stays in state i given the process is in state i at time t. In other words,
i (t)
= inf{s > 0 : X(t + s) 6= X(t) and X(t) = i}.
The following theorem says that
is time-homogeneous.
i (t)
is an exponentially distributed random variable if the CTMC
Theorem 8.1.1 The time spent in any state i 2 S of a time-homogeneous CTMC is exponentially
distributed, i.e., i (t) is exponentially distributed with a parameter depending only on state i.
209
210
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Proof Assume the Markov chain is in state i at time 0, i.e., X(0) = i. We consider the probability
that the CTMC stays in state i for more than u + v units of time given that it has stayed in state
i for v units of time. Since {X(t)} is a continuous-time Markov chain, we have
Pr( i (0) > u + v| i (0) > v) = Pr(X(t) = i, v < t  u + v|X(s) = i, 0  s  v)
= Pr(X(t) = i, v < t  u + v|X(v) = i).
Further, because X(t) is time-homogeneous, we have
Pr( i (0) > u + v| i (0) > v) = Pr(X(t) = i, 0 < t  u|X(0) = i)
= Pr( i (0) > u),
which implies that
Pr( i (0) > u + v) = Pr( i (0) > u) Pr( i (0) > v).
Therefore, the probability distribution of i (0) is memoryless. Since the exponential distribution
is the only distribution on the positive real line with this property, the theorem holds.
⇤
In this book, we will assume that the number of transitions in a finite time interval is finite
with probability one. One can construct CTMCs with strange behavior if we do not assume this
condition, so we will assume this condition throughout this book. This condition is called nonexplosiveness.
Figure 8.1: A time-homogeneous CTMC as a jump process
A time-homogeneous CTMC can be described by a jump process, where a jump is a state
transition. Let Y (k) denote the state of the CTMC after k jumps and Tk is the time duration
between (k 1)th jump and k th jump as shown in Figure 8.1. According to Theorem 8.1.1, Tk is
an exponentially distributed random variable with the parameter depending on Y (k 1) only, so
a time-homogeneous CTMC can also be described as follows:
(i) During each visit to state i, the CTMC spends an exponentially distributed amount of time
in that state. We will let 1/qi denote the mean time spent in state i.
8.1. MATHEMATICAL BACKGROUND: CONTINUOUS-TIME MARKOV CHAINS
211
(ii) After spending an exponentially distributed amount of time in state i, the CTMC moves to
state j 6= i with probability Pij .
Therefore, the CTMC is completely specified by the parameters {qi } and {Pij }.
An important quantity of interest is
pi (t) = Pr(X(t) = i).
Now we will derive a di↵erential equation to describe the evolution of pi (t). First note
pi (t + )
= Pr(X(t + ) = i)
X
=
Pr(X(t + ) = i|X(t) = j) Pr(X(t) = j)
j
= Pr(X(t + ) = i|X(t) = i) Pr(X(t) = i)
X
+
Pr(X(t + ) = i|X(t) = j)P (X(t) = j)
j6=i
= e
qi
pi (t) +
X⇣
1
e
⌘
qj
j6=i
Pji pj (t) + O( 2 ).
In the last step above, the terms that are not of the order 2 correspond to none or only one state
transition occurring in [t, t + ]. Probability that more than one transition occur in [t, t + ] is O( 2 ).
When is sufficiently small, the equality above can be approximated as:
X
pi (t + ) ⇡ (1 qi )pi (t) +
qj Pji pj (t).
j6=i
Moving pi (t) at the right-hand side to the left-hand side and dividing both sides by , we obtain
pi (t + )
pi (t)
=
qi pi (t) +
X
qj Pji pj (t),
j6=i
which implies that
ṗi (t) =
qi pi (t) +
X
Pji qj pj (t).
j6=i
Now defining Qji = Pji qj and Qii =
qi , we have
ṗi (t) = Qii pi (t) +
X
Qji pj (t)
j6=i
or
ṗ(t) = p(t)Q,
where Q is a |S| ⇥ |S| matrix, and called rate transition matrix or generator matrix of the CTMC.
Note that Q completely describes the CTMC, along with p(0). So given Q, we need not to
specify {qi } and {Pij }. According to its definition, Q has the following two properties:
212
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
(i) Qij 0 for i 6= j, and
P
(ii)
j Qij = 0.
The first property is easy to verify because Qij = qi Pij when i 6= j. The second property holds
because
X
X
Qij =
qi Pij qi
j
j6=i
0
= qi @
= 0.
X
Pij
j6=i
1
1A
Similar to DTMCs, we are interested in following questions:
P
• Does there exist a ⇡ 0 so that 0 = ⇡Q and i ⇡i = 1? Note that if such a ⇡ exists, p(t) = ⇡
for all t when p(0) = ⇡. So such a ⇡ is called a stationary distribution.
• If there exists a unique stationary distribution, does limk!1 p(t) = ⇡ for all p(0)? In other
words, does the distribution of the Markov chain converge to the stationary distribution
starting from any initial state?
Similar to DTMCs, in general, one can come up with examples to show that a CTMC may not
possess a unique stationary distribution if it is not irreducible. The definition of irreducibility for
CTMCs is presented below.
Definition 8.1.1 (Irreducibility) A CTMC is said to be irreducible if given any two states i and
j,
Pr (X(t) = j|X(0) = i) > 0
⇤
for some finite t.
Note that there is no concept of aperiodicity for CTMCs since state transitions can happen at
anytime. The following theorem states that a CTMC has a unique stationary distribution if it is
irreducible and has a finite state space.
Theorem 8.1.2 A finite state space, irreducible CTMC has a unique stationary distribution ⇡ and
limt!1 p(t) = ⇡, 8p(0).
⇤
For a finite state space, irreducible P
CTMC, the stationary distribution can be computed by
finding a ⇡ 0 such that ⇡Q = 0 and i ⇡i = 1. If the state space is infinite, then irreducibility
is not sufficient to guarantee the CTMC has a unique stationary distribution.
Example 32 Consider a CTMC X(t) with the state space to be integers. The rate transition
matrix Q is as follows:
Qij
= 1
if
j =i+1
= 1
if
j=i
=
= 0
2
if
j=i
otherwise.
1
8.1. MATHEMATICAL BACKGROUND: CONTINUOUS-TIME MARKOV CHAINS
213
It is easy to verify that Pr (X(t) = j|X(0) = i) > 0 for some finite t, so X(t) is irreducible.
If a stationary distribution ⇡ exists, it has to satisfy 0 = ⇡Q, which can be rewritten as
0 = ⇡k
1
2⇡k + ⇡k+1 ,
8k.
Thus, we have to solve for ⇡ that satisfies
2⇡k = ⇡k
1
X
k= 1
1
+ ⇡k+1 ,
8k
⇡k = 1
⇡k
0,
8k.
We have shown in Example 10 for DTMC in Chapter 3 that no ⇡ can satisfy all three equalities,
so a stationary distribution does not exist.
⇤
Similar to DTMCs, we now introduce the notion of recurrence, and conditions beyond irreducibility to ensure the existence of stationary distributions.
Definition 8.1.2 Assuming X(0) = i, the recurrence time is the first time the CTMC returns to
state i after it leaves the state. Recall that
i
= inf{t > 0 : X(t) 6= i and X(0) = i},
then the recurrent time ⌧i is
⌧i = inf{t >
i
: X(t) = i and X(0) = i}.
State i is called recurrent if
lim Pr(⌧i < T ) = 1,
T !1
and transient otherwise.
A recurrent state is positive recurrent if E[⌧i ] < 1, and is null-recurrent if E[⌧i ] = 1.
Lemma 8.1.3 For an irreducible CTMC, if one state is positive recurrent (null recurrent), then
all states are positive recurrent (null recurrent). Further
lim pi (t) =
t!1
which holds even when E[⌧i ] = 1.
1
,
E[⌧i ]( Qii )
⇤
Theorem 8.1.4 P
Consider an irreducible and non-explosive CTMC. A unique stationary distribution ⇡ 0 (i.e., i ⇡i = 1 and ⇡Q = 0) exists if and only if the CTMC is positive recurrent. ⇤
P
This theorem states that if we can find a vector ⇡ 0 such that ⇡Q = 0 andP i ⇡i < 1, and
the CTMC is non-explosive, then the CTMC is positive recurrent. Note that if i ⇡i 6= 1, we can
define ⇡
˜i = P⇡i⇡j , and ⇡
˜ is a stationary distribution.
j
214
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Theorem 8.1.5 If there exists a ⇡ such that ⇡Q = 0 and
then the CTMC is not positive recurrent and
P
i ⇡i
= 1 for an irreducible CTMC,
lim pi (t) = 0
t!1
⇤
for all i.
Now we have characterized the conditions under which the CTMC is positive recurrent and not
positive recurrent. We further examed the condition under which the CTMC is positive recurrent.
The following lemma presents an alternative characterization of the equation that has be satisfied
by the stationary distribution of a CTMC.
Lemma 8.1.6 ⇡Q = 0 is equivalent to
X
⇡i Qij = ⇡j
i6=j
X
Qji
i6=j
8j.
(8.1)
Proof Assume that equality (8.9) holds. For fixed j, we have
X
X
⇡i Qij = ⇡j
Qji + ⇡j Qjj
i
i6=j
= ⇡j
X
Qji
i
= 0,
where the last equality holds due to the definition of Q. Equation (8.9) is often called global balance
equation. Note that the equation simply shows that given a state j, the rate at which transitions
take place into state j is equal to the rate at which transitions take place out of state j.
⇤
Example 33 Consider a CTMC X(t) with the state space to be
transition matrix Q is assumed to be as follows:
0
1 1
0
0
0 0 ...
B 2
3
1
0
0 0 ...
P=B
@ 0
2
3 1
0 0 ...
...
Consider ⇡i = 2
(i+1) .
First, we have
1
X
⇡i =
i=0
Further, for j > 0,
X
⇡i Qij = ⇡j
i6=j
1 Qj 1,j
1
X
i=0
2
(i+1)
=
nonnegative integers. The rate
1
C
C.
A
1/2
= 1.
1 1/2
+ ⇡j+1 Qj+1,j = 2
j
+2
j 2
⇥2=3⇥2
j 1
,
8.1. MATHEMATICAL BACKGROUND: CONTINUOUS-TIME MARKOV CHAINS
and
⇡j
X
Qji = ⇡j (Qj,j
1
+ Qj,j+1 ) = 2
j 1
i6=j
⇥ (1 + 2) = 3 ⇥ 2
j 1
215
,
so the global balance equation holds. It is easy to verify that the global balance equation also holds
when j = 0. So X(t) has a unique stationary distribution ⇡i = 2 (i+1) .
⇤
Unlike the above example, there are many instances where one cannot easily verify the global
balance equation. Next we introduce a sufficient condition under which the global balance equation
holds.
Lemma 8.1.7 The global balance equation holds if
⇡i Qij = ⇡j Qji
8i 6= j
(8.2)
Proof For fixed j, we have
X
⇡i Qij =
i6=j
X
⇡j Qji = ⇡j
i6=j
X
Qji .
i6=j
Equation (8.2) is called the local balance equation. If the local balance equation is satisfied, then
given a pair of states i and j, the rate at which transitions take place from state i to state j is equal
to the rate at which transitions take place from state j to state i.
⇤
Example 34 The local balance equation is often much easier to verify than the global balance
equation. Consider the CTMC in the previous example. Since Qij = 0 when |i j| > 1, the local
balance equation is equivalent to
⇡i Qi,i+1 = ⇡i+1 Qi+1,i
It is straightforward to verify that ⇡ = 2
(i+1)
8i.
satisfies the equality above.
⇤
Note that the local balance equation is only a sufficient condition for a CTMC to be positive
recurrent. In particular, it is possible to have a stationary distribution ⇡ that satisfies the global
balance equation but not the local balance equation.
Often it is difficult to find the ⇡ to satisfy either the global or local balance equation. Nevertheless, in the applications, it is important to know whether ⇡ exists even if we cannot find it
explicitly. Therefore, similar to the Foster-Lyapunov Theorem for DTMCs, we present the following
Foster-Lyapunov Theorem for CTMCs, which provides another sufficient condition for a CTMC to
be positive recurrent.
Theorem 8.1.8 (Foster-Lyapunov Theorem for CTMCs) Suppose X(t) is irreducible and
non-explosive. If there exists a function V : S ! R+ such that
P
1.
V (i))  ✏ if i 2 B c , and
j6=i Qij (V (j)
P
2.
V (i))  M if i 2 B,
j6=i Qij (V (j)
216
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
for some ✏ > 0, M < 1, and a bounded set B, then X(t) is positive recurrent.
Proof We omit the details, and just present the intuition based on DTMCs. For sufficiently small
, the probability that X(t) makes the transition from state i to state j in time interval [t, t + ] is
approximately Qij , so the expected drift of the V (t) given X(t) = i is

V (X(t + )) V (X(t))
E
X(t) = i
P

V (i)
j6=i Qij V (j) + (1 + Qii )V (i)
= E
X(t) = i
X
=
Qij V (j) + Qii V (i).
j6=i
P
Substituting Qii =
j6=i Qij into the equation above, we obtain

X
V (X(t + )) V (X(t))
E
X(t) = i =
Qij (V (j)
V (i)),
j6=i
which is less than
✏ when i 2 B c and less than M otherwise.
⇤
8.2
Queueing Systems: Introduction and Definitions
We will consider queueing systems, where customers (or packets) arrive at a queue according to
some arrival process, the service time of each customer is some random variable, and the number of
servers in the system could be one or more. A pictorial illustration of a queueing system is shown
in Figure 8.2.
Figure 8.2: A pictorial illustration of queueing systems
Customers who find all servers busy form a queue. In general, the order in which the customers
in the queue has to specified to complete the description of the queueing system. Unless otherwise
stated, we will assume that the customers are served in FIFO (first-in, first-out) order. We will
encounter two types of queues: ones in which there is infinite waiting space and ones where the
number of waiting spaces is limited. When the number of waiting spaces is limited, we will assume
that a customer who finds no empty waiting space is blocked and lost forever. In other words, we
assume that blocked customers do not attempt to join the queue at a later time.
We will use to denote the average arrival rate of customers (number of arriving customer per
second) into the system and 1/µ to denote the average service time of each customer. Thus, µ is
8.2. QUEUEING SYSTEMS: INTRODUCTION AND DEFINITIONS
217
the average rate at which a server can serve the customers, i.e., number of customer served per
second is µ.
Little’s law L = W again relates the average number of customers in the system L to the
average time spent in the system W through arrival rate . This relation applies both to the
system as a whole (queue+servers), the queue alone or just the servers.
The intuition behind Little’s law can be explained as follows: suppose that the customers pay
the system $1 for each second spent in the system. Since a customer spends an average of W
seconds in the system, the average rate of revenue for the system is W dollars/second. But it is
also true that the average number of customers in the system in L. Thus, the rate of revenue is
also L dollars/second. Therefore, L has to be equal to W.
We would like to comment that Little’s law is true in great generality with very few assumptions
on the arrival process, service times, the number of servers, the number of waiting spaces, etc. The
proof is similar to the proof for discrete-time queues, and is omitted here.
A queueing system can be specified by specifying arrival process, service process, the number
of servers, and bu↵er space. Therefore, we use the following standard notation:
Definition 8.2.1 (M/M/s/k queue) The first M denotes the fact that inter-arrival times are
exponential (memoryless, hence the M), the second M denotes that service times are exponential
(memoryless), s is the number of servers, and k is the total number of customers allowed in the
system at any time. Thus, k s is the number of waiting spaces in the queue, also known as bu↵er
space. When the queue in full, any arriving customer is blocked from entering the system and lost.
Unless specified, we assume that the service order is FIFO.
⇤
If, instead of M, we use G, then it denotes either general inter-arrival times or service times,
depending upon whether the G is in the first or second position in the above notation. The notation
GI is used to further indicate that the inter-arrival times (or service times) are independent. We
also use D to denote constant (or deterministic) inter-arrival or service times.
An arrival process with exponentially distributed i.i.d. inter-arrival times is called the Poisson
process. We next present the definition and some properties of Poisson processes.
Definition 8.2.2 (Poisson process) N (t) (t
following conditions hold:
0) is a Poisson process with parameter
if the
(i) N (0) = 0,
(ii) N is a counting process, i.e., N (t) can increase at most by one at any time instant,
(iii) N (t) is an independent, increment process, and
(iv) N (t)
N (s) s Poi( (t
)).
⇤
Since N (t)
N (s) follows Poisson distribution with parameter (t
E[N (t)
N (s)] = (t
s)
and
V ar(N (t)
s),
N (s)) = (t
s).
Next we present three equivalent definitions of Poisson processes. For all three, we assume
N (0) = 0.
218
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
(i) N (t) is a counting process and the inter-arrival times are i.i.d. exponentially distributed with
parameter .
(ii) N (t) N (s) s Poi( (t
distributed in [s, t].
s)); and given N (t)
N (s) = n, the event times are uniformly
(iii) N (t) has stationary and independent increments, Pr(N ( ) = 1) =
+ o( ), and Pr(N ( ))
2) = o( ). Here, o( ) denotes a function g( ) with the property lim !0 g( )/ = 0.
Here, we do not prove the fact the above two definitions are equivalent to Definition 8.2.2. The
proof can be found in many textbooks on random processes.
Below are two well-known properties of Poisson processes, which will be used in this chapter.
The proofs of these results are left as exercises at the end of this chapter.
Result 8.2.1 N1 (t) and N2 (t) are independent Poisson processes with parameters
spectively, then N1 (t) + N2 (t) is a Poisson process with parameter 1 + 2 .
1
and
2,
re⇤
Result 8.2.2 Assume that N (t) is a Poisson process. We can generate K random processes
N1 (t), · · · , NK (t) as follows: when there P
is an arrival according to N (t), make it an arrival for
process Nk (t) with probability pk , where K
k=1 pk = 1. Then, N1 (t), · · · , NK (t) are independent
Poisson processes with parameters p1 , · · · , pK , respectively.
⇤
8.3
The M/M/1 Queue
The M/M/1 queue is a further short-hand notation for an M/M/1/1 queue. Let q(t) denote
the number of customers in the system, which forms a time-homogenous CTMC. Let ⇡(n) be the
(steady-state) probability that there are n customers in the system. Note that Pij = 0 if j 6= i 1
or i + 1, and the inter-arrival times and service times are exponential, so the rate transition matrix
Q is
8
,
if j = i + 1
>
>
<
µ,
if j = i 1
Qij =
µ, if j = i
>
>
:
0,
otherwise.
To see this, consider a small interval of time [t, t ). Then,
Pr(q(t + ) = i + 1|q(t) = i)
= Pr ({one arrival and no departures in delta time units} or
=
{two arrivals and one departure in delta time units} or · · · )
+ o( ),
where the last equality follows from one of the definitions of a Poisson process. Thus,
Qi,i+1 = lim Pr(q(t + ) = i + 1|q(t) = i)/ = .
!0
The other entries of the Q matrix can be derived similarly. The Markov chain is illustrated in
Figure 8.3. In general, a CTMC can be represented by such a figure, where the nodes represent
8.3. THE M/M/1 QUEUE
219
Figure 8.3: The Markov chain for a M/M/1/1 queue
the state of the Markov chain and the arrows represent possible transitions. The numbers on the
arrows represents the corresponding entries of the rate transition matrix Q.
We therefore have ⇡(n) = µ⇡(n + 1), or
⇡(n + 1) = ⇢⇡(n),
(8.3)
where ⇢ = /µ. We note that ⇢ is dimensionless, but is often expressed in units of Erlangs. Repeatedly using (8.3), we obtain
⇡(1) = ⇢⇡(0)
⇡(2) = ⇢⇡(1) = ⇢2 ⇡(0)
..
.
⇡(n) = ⇢⇡(n 1) = ⇢n ⇡(0).
P
P1 n
Since 1
n=0 ⇡(n) = 1, we have ⇡(0)
n=0 ⇢ = 1.
Suppose ⇢ < 1, then
1
X
1
⇢n =
,
1 ⇢
n=0
which yields
⇡(n) = ⇢n (1
⇢).
We note that ⇢ < 1 implies µ > , i.e., the service rate of the server is larger than the arrival
rate. Intuitively, this is required since, otherwise, the number of customers in the system will grow
forever. In other words, ⇢ < 1 is required to ensure the stability of the queueing system.
Now using ⇡(n) = ⇢n (1 ⇢) and Little’s law, we can derive the following results:
P
⇢
• The mean number of customers in the system: L = 1
n=0 n⇡(n) = 1 ⇢ .
• From Little’s law, the mean delay (i.e, waiting time in the queue + service time) of a customer:
W = L = µ1 .
• The mean waiting time of a customer in the queue: Wq = W
1
µ
=
⇢
µ
.
• The mean number of customers in the queue (again, by Little’s law): Lq = Wq =
⇢2
1 ⇢.
We next study the fraction of time the server is busy, i.e., the mean server utilization. Note
that
The fraction of time the server is busy=Prob(There is at least one customer in the system),
220
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
so
U , The mean server utilization = 1
⇡0 = ⇢.
As the server utilization becomes close to 1, L ! 1. Thus, the mean delay, the waiting time, the
number of customers in the queue, all go to 1.
One can also see the fact that U = ⇢ as a consequence of Little’s law. Let us apply Little’s law
to the server. Note that at most one customer can be in the server, so
L = The average number of customers in the server
= The fraction of time that the server is busy
= U.
Considering the waiting time, we have
W
= The average amount of time spent by a customer in service
1
=
.
µ
Thus, L = W implies that
U = L = /µ = ⇢.
Note that, in applying Little’s law to compute U, we did not use the fact that the queue is an M/M/1
queue. Indeed, the result applies to any general inter-arrival and service time distributions, i.e.,
the fact that U = ⇢ applies to any G/G/1 queue.
8.4
The M/M/s/s Queue
Consider a M/M/s/s loss model, where the queueing system has s servers and no bu↵er. Again,
the number of customers in the system is a time-homogenous CTMC. We are interested in the
blocking probability, i.e., the probability that an arriving customer is blocked and lost due to the
fact that all servers are busy. As in the case of the M/M/1 queue, the number of customers in the
M/M/s/s queue evolves as a Markov chain, and is shown in Figure 8.4. The transition rates can
be derived as in the case of the M/M/1 queue.
Figure 8.4: The Markov chain for M/M/s/s queue
We next derive the stationary distribution ⇡ of the number of customers in the queue for the
M/M/s/s loss model.
8.4. THE M/M/S/S QUEUE
221
According to the local balance equations, we have that
⇡(0) = µ⇡(1)
⇡(1) = 2µ⇡(2)
..
.
⇡(s
1) = sµ⇡(s)
Thus, for 0  n  s,
⇢n
⇡(n) =
⇡(0),
n!
P
where, as before, ⇢ = /µ. Using the fact that sk=0 ⇡(k) = 1, we obtain
1
⇡(0) = Ps
⇢k
k=0 k!
and
⇡(n) = Ps
⇢n
n!
⇢k
k=0 k!
,
.
Thus, the probability that all servers are busy is given by
⇡(s) = Ps
⇢s
s!
⇢k
k=0 k!
.
(8.4)
The PASTA Property and Blocking Probability
It turns that ⇡(s), the probability that all servers are busy, is also equal to the blocking probability.
To see that the two quantities are equal, we have to use that fact, when the arrival process is
Poisson, average computed at customer arrival times is equal to the corresponding time average.
In the case of the M/M/s/s model, the customer average is the average fraction of time that an
arriving customer sees that all servers are busy, i.e., the blocking probability. The time average is
the fraction of time that all servers are busy, i.e., the probability that all servers are busy. The
fact that these two quantities are equal is a consequence of a more general property called Poisson
Arrivals See Time Averages or PASTA. The intuition behind the PASTA property is nearly identical
to the BASTA property discussed earlier for discrete-time queues (see Section 3.4.3).
Formula (8.4) for the blocking probability is called the Erlang-B formula in honor of A. K. Erlang
who was the father of queueing theory and derived the blocking probability formula. The ErlangB formula also holds when the service times are non-exponential. The proof of this insensitivity
to service-time distribution is similar to the insensitivity proof for a di↵erent model provided in
Section 8.9.
To see that non-Poisson arrivals may not see time averages, consider the following example.
Consider a D/D/1/1 model, i.e., deterministic (or constant) inter-arrival times, deterministic service
times, one server and no bu↵er. Let the inter-arrival time and service-time be 10 seconds. Then,
the server is always busy, but no customer is blocked since each arrival occurs immediately after the
previous customer has been served. Thus, the fraction of time that the system is full is 1 whereas
the fraction of blocked customers is 0.
222
8.5
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
The M/M/s Queue
Figure 8.5: The Markov chain for M/M/s queue
Consider the M/M/s model, where the queueing system has s servers and infinite bu↵er. The
time-homogeneous Markov chain describing the number of customers in the system is shown in
Figure 8.5. From the local balance equation, we have
⇡(0) = µ⇡(1)
⇡(1) = 2µ⇡(2)
..
.
⇡(s
1) = sµ⇡(s)
⇡(s) = sµ⇡(s + 1)
..
..
Thus,
⇡(n) =
⇡(n) =
where ⇢ =
sµ
< 1. Using the fact
P1
n=0 ⇡(n)
⇡(0) =
(s⇢)n
⇡(0), n  s,
n!
ss ⇢n
⇡(0), n > s,
s!
= 1, we get
"s 1
X (s⇢)n
n=0
n!
(s⇢)s
+
s!(1 ⇢)
#
1
.
The probability that an arrival finds all servers busy, is given by
PQ =
1
X
n=s
⇡(n) =
⇡(0)(s⇢)s
.
s!(1 ⇢)
This is called the Erlang-C formula.
8.6
The M/GI/1 Queue
Consider a queueing system with exponential inter-arrival times (mean ) and general but identical
and independent service times (mean 1/µ) as shown in Figure 8.6. Note for general service times, the
8.6. THE M/GI/1 QUEUE
223
Figure 8.6: An M/GI/1 queue
amount of time a customer has been served determines the remaining service time of the customer.
So unless the service times are exponentially distributed, the number of customers in the system
is not a Markov chain. To compute the stationary distribution of M/GI/1 queues, we sample the
system at departure times. Denote by qk the number of customers in the system immediately after
the k th departure, and Ak+1 is the number of arrivals during the service time of the (k + 1)th
customer.
When qk > 0, it is easy to see that
qk+1 = qk + Ak+1
1.
When qk = 0, the number of arrivals between the k th departure and the (k + 1)th departure is
1 + Ak+1 , where the one accounts for the arrival of the (k + 1)th customer, and Ak+1 are the
number of arrivals when the (k + 1)th customer is served. Therefore when qk = 0,
qk+1 = qk + 1 + Ak+1
1.
Therefore, the sampled queue can be described as the following dynamical system:
qk+1 = qk + Ak+1
Uk+1 ,
where Uk+1 = 1 when qk > 0 and Uk+1 = 0 when qk = 0. Since the arrival process is Poisson, qk is
a DTMC.
Denote by Sk+1 the service time of the (k + 1)th customer. Given Sk+1 = sk+1 , Ak+1 follows
Poisson distribution with parameter sk+1 . We therefore can compute the first moment and second
moment of Ak+1 given Sk+1 :
E[Ak+1 |Sk+1 ] =
E[A2k+1 |Sk+1 ]
Sk+1
=
Sk+1 +
2 2
Sk+1 .
Recall that E[Sk+1 ] = 1/µ. Therefore,
E[Ak+1 ] =
E[A2k+1 ] =
1
=⇢
µ
µ
+
2
E[S 2 ] = ⇢ +
2
E[S 2 ].
We ignore the subscript k + 1 of Sk+1 since the service times are i.i.d..
Note that {Ak } are i.i.d. because the inter-arrival times are exponential and the service times
are independent. So {qk } is similar to DTMCs that we have analyzed before. If ⇢ < 1, then {qk } is
positive recurrent and has a well-defined stationary distribution. As in previous chapters, we also
say the queueing system is stable if q(t) has a well-defined stationary distribution. Now the question
is does {q(t)} at any arbitrary time have a well-defined stationary distribution? This question is
answered by the following theorem.
224
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Theorem 8.6.1 Consider the M/GI/1 queue introduced at the beginning of this section. When
⇢ < 1, the queueing system has a well-defined stationary distribution, the same distribution as that
of {qk }.
Proof We present the key ideas behind the proof here. First we sample the system at arrival times
and denote by Qk the number of customers in the queue before the k th arrival. We claim that Qk
and qk have the same stationary distribution.
To prove this fact, let Fn (t) denote the number of up crossings from n to (n + 1) in [0, t], and
Gn (t) denote the number of down crossings from (n + 1) to n in [0, t]. Note that Fn (t) is the number
of the samples that the system is in state n when the system is sampled at arrival times, so
Fn (t)
,
t!1 A(t)
⇡Q (n) = lim
where A(t) is the number of arrivals in [0, t]. Similarly, Gn (t) is the number of the samples that the
system is in state n when the system is sampled at departure times, so
Gn (t)
,
t!1 D(t)
⇡q (n) = lim
where D(t) is the number of departures in [0, t].
Note that |Fn (t) Gn (t)|  1 since the number of customers in the system has to come down
before it can go up again, and vice versa, so
lim
t!1
|Fn (t) Gn (t)|
= 0.
A(t)
Therefore, we obtain
Fn (t)
Gn (t)
Gn (t) D(t)/t
= lim
= lim
.
t!1 A(t)
t!1 A(t)
t!1 D(t) A(t)/t
lim
Since {qk } is stable, we have
D(t)
A(t)
= lim
= ,
t!1 t
t!1 t
lim
which implies that
Fn (t)
Gn (t)
= lim
,
t!1 A(t)
t!1 D(t)
lim
and
⇡Q (n) = ⇡q (n).
Next, by the PASTA property, ⇡Q (n) must also be the steady-state probability distribution of
having n packets in the system:
⇡Q (n) =
=
=
lim Pr(q(t) = n|A(t, t + ) = 1)
t!1
lim Pr(q(t) = n)
t!1
lim Pr(q(t) = n)
t!1
= ⇡(n).
The theorem, therefore, holds.
Pr(A(t, t + ) = 1|q(t) = n)
Pr(A(t, t + ) = 1)
8.6. THE M/GI/1 QUEUE
225
⇤
The Mean Queue Length and Waiting Time
Next, we compute the mean queue length of the M/GI/1 queue. Since the stationary distribution
of q(t) is the same as that of qk , the system sampled at departure times, we compute E[qk ]. We
first compute the second moment of qk :
⇥ 2 ⇤
E qk+1
=
Uk+1 )2 ]
E[(qk + Ak+1
2 ] + 2E[q (A
= E[qk2 ] + E[A2k+1 ] + E[Uk+1
k
k+1
Since Uk 2 {1, 0}, we have
Uk+1 )]
2E[Ak+1 Uk+1 ].
(8.5)
2
E[Uk+1
] = E[Uk+1 ] = Pr(qk > 0) = ⇢.
If ⇢ < 1, then qk has a stationary distribution and
2
lim E[qk+1
] = lim E[qk2 ].
k!1
k!1
Recall that
E[Ak+1 ] = ⇢ and
E[A2k+1 ] = ⇢ +
2
E[S 2 ].
Moving E[qk2 ] in (8.5) to the left-hand side and letting k go to infinity, we obtain
2
0=⇢+
E[S 2 ] + ⇢ + 2L⇢
2L
2⇢2 ,
where L = limk!1 E[qk ], and we used the fact that qk Uk+1 = qk . Rearranging the terms in the
equation above, we get
0 = 2 E[S 2 ] + 2(L ⇢)(⇢ 1),
which leads to
L=⇢+
2 E[S 2 ]
.
2(1 ⇢)
Next, we can compute the number of customers in the queue and the waiting time in the queue
using Little’s law:
Lq = L
⇢=
2 E[S 2 ]
2(1 ⇢)
Lq
E[S 2 ]
Wq =
=
,
2(1 ⇢)
where the second equality is the Pollaczek-Khinchine formula, also called P-K formula.
Note that if S is exponential,
E[S 2 ] =
which implies that
Wq =
1
1
2
+ 2 = 2,
2
µ
µ
µ
2
1
⇢
=
.
µ2 2(1 ⇢)
µ(1 ⇢)
We recover the average waiting time for M/M/1 queueing systems.
226
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Di↵erent Approaches to Deriving the P-K Formula
We now introduce two alternative approaches to deriving the P-K formula. Let R(t) denote the
remaining service time of the customer in the server at time t. Figure 8.7 illustrates a sample path
of R(t), where si is the service time of customer i.
Figure 8.7: A sample path of R(t)
Let R denote the average remaining service time of the customer in the server when a new
customer joins the queue. Due to the PASTA property,
Wq =
Lq
+ R,
µ
where R is the time required to serve the customer in the server, and Lq /µ is the amount of time
required to serve all customers in the queue. According to Little’s law, Lq = Wq , so
Wq (1
or
Wq =
⇢) = R,
R
.
(8.6)
1 ⇢
We index the customers according to the sequence they depart the system and let D(t) denote
the number of customers who have departed up to and including time t. Further we define
Z
1 t
R̄t =
R(s) ds,
t 0
where si is the service time of customer i.
Assume customer i started to receive service at time ti , then
Z ti +si
s2
R(s) ds = i
2
ti
8.6. THE M/GI/1 QUEUE
227
as shown in Figure 8.7, which yields
D(t)
D(t)
2
1 X s2i
1 X s2i
1 sD(t)+1
 R̄t 
+
.
t
2
t
2
t
2
i=1
i=1
Note that as t ! 1,
2
1 sD(t)+1
! 0,
t
2
so
D(t)
D(t) 1 X 1 2 1
R = lim R̄t = lim
s =
E[S 2 ].
t!1
t!1 t D(t)
2 i
2
i=1
Combining the equality above with equality (8.6), we obtain the P-K formula:
Wq =
E[S 2 ]
.
2(1 ⇢)
Another way to derive the P-K formula is to compute the entire distribution of q(t). Consider
the probability generating function for qk :
Q(z) , E[z qk ] =
1
X
z i ⇡i .
i=0
Assuming that qk is in steady state, we have
⇥
Q(z) = E z qk
⇥
= E z qk
⇥
⇤
where A(z) = E z Ak . Recall that Uk = 1 if qk
1
1
Uk +Ak
1
Uk
⇤
⇤
A(z),
> 0 and Uk = 0 otherwise, so
P1
i 1 A(z)
i=1 ⇡(i)z
P
i
= ⇡(0) + z1 1
i=1 ⇡(i)z A(z)
Q(z)
= ⇡(0) +
= ⇡(0) + z1 (Q(z) ⇡(0)) A(z)
⇣
⌘
= ⇡(0)(1 z1 ) + Q(z)
A(z).
z
Moving the term containing Q(z) to the left-hand side, we obtain
✓
Q(z) 1
A(z)
z
◆
= ⇡(0)A(z) 1
which implies that
Q(z) =
✓
⇡(0)A(z)(z 1)
.
(z A(z))
1
z
◆
,
228
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Recall that ⇡(0) = 1
definition of Ak , we have
⇢, so Q(z) can be obtained after A(z) is known. According to the
⇥
⇤
A(z) = E z Ak
⇥ ⇥
⇤⇤
= E E z Ak service time = t
"1
#
X ( t)i e t
i
= E
z
i!
i=0
h
i
= E e zt e t
h
i
= E e (z 1)t
= M ( (z
1)),
where M (✓) = E[e✓t ] is the moment generating function of the random variable t.
Assuming the closed-form expression of Q(z) is known, we can obtain L as follows:
L = E[qk ] =
d
E [z qk ]
dz
= Q0 (1),
z=1
which implies that
Lq = L
⇢ and
Wq = Lq .
In the exercise, you will be asked to compute Q(z) and recover the P-K formula.
8.7
The GI/GI/1 Queue
We now consider a general queueing system where inter arrival times are general i.i.d. random
variables and service times are general i.i.d. random variables. This queueing system is either
written as the G/G/1 or the GI/GI/1 if one wants to emphasize that the inter-arrival times are
independent and the the service times are independent. Here we use the GI/GI/1 terminology. We
introduce the following notations:
• Wk : the waiting time in the queue of the k th customer,
• Sk : the service time of the k th customer, and
• Ik+1 : the inter-arrival time between the k th customer and (k + 1)th customer.
Then the evolution of Wk can be written as:
Ik+1 )+ ,
Wk+1 = (Wk + Sk
see Figure 8.8.
We define Xk = Sk
Ik+1 . Then the evolution of Wk can be re-written as:
Ik+1 )+
Wk+1 = (Wk + Sk
= (Wk + Xk )+
= max {0, Wk + Xk }
1}
+ Xk }
= max{0, max{0, Wk
1)
+ Xk
= max{0, Wk
1
+ Xk , Xk }.
1
+ Xk
8.7. THE GI/GI/1 QUEUE
229
Figure 8.8: The evolution of Wk . The waiting time of the (k + 1)th customer equals to the overall
time the k customer spends in the system minus the inter-arrival time between the k th customer
and (k + 1)th customer.
Iterating backwards and noting W1 = 0, we get
Wk = max{0, X1 + X2 + · · · + Xk ,
X2 + X3 + · · · + Xk ,
..
.
Xk }.
Recall that {Xk } are i.i.d., so Wk is distributionally equivalent to
W̃k = max{0, X1 , X1 + X2 , · · · , X1 + X2 + X3 + · · · + Xk }.
Since W̃k is a non-decreasing sequence, it has limit in [0, 1]. Suppose E[X] = E[Sk
1
1
< µ, then by the Strong Law of Large Number (SLLN),
µ < , or
N
1 X
Xk = E[X]
N !1 N
lim
k=0
with probability one. In other words,
lim
N !1
N
X
Xk =
k=0
1
with probability one, which implies
W̃1 < 1
where
W̃1 = lim W̃k .
k!1
Therefore, we can conclude that
Pr(W̃1 < 1) = Pr(W1 < 1) = 1,
and W1 is a well-defined random variable. Thus,
d
Wk ! W1
if
< µ, where d means convergence in distribution.
Ik+1 ] < 0, or
230
8.8
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Reversibility
In this section, we introduce an important concept which is quite useful in the study of CTMCs —
reversibility. Reversibility is a power tool to obtain stationary distributions of complicated CTMC
models. Consider a stationary CTMC X(t), i.e., p(t) = ⇡ for all t. We also assume t 2 ( 1, 1).
We define
Y (t) = X( t),
which is called the reversed chain of X(t). We can view Y (t) as a movie of X(t) running backwards.
Below we show that Y (t) is also a CTMC.
Lemma 8.8.1 Y (t) is a CTMC, and the transition rate matrix of Y (t), denoted by Q⇤ satisfies
Q⇤ji = Qij ⇡⇡ji .
Proof Consider a sequence of times such that t
probability:
t1
t2
···
tn , and the following conditional
Pr(Y (t)|Y (t1 ), Y (t2 ), · · · , Y (tn )) = Pr(X( t)|X( t1 ), X( t2 ), · · · , X( tn )).
Note that
t
t1 
t2  · · · 
tn . If we can prove
Pr(X( t)|X( t1 ), X( t2 ), · · · , X( tn )) = Pr(X( t)|X( t1 )),
then
Pr(Y (t)|Y (t1 ), Y (t2 ), · · · , Y (tn )) = Pr(X( t)|X( t1 )) = Pr(Y (t)|Y (t1 )),
and Y (t) is a CTMC.
According to the definition of conditional probability, we have
=
=
=
=
Pr(X( t)|X( t1 ), X( t2 ), · · · , X( tn ))
Pr(X( t), X( t1 )|X( t2 ), · · · , X( tn ))
Pr(X( t1 )|X( t2 ), · · · , X( tn ))
Pr(X( t), X( t1 )) Pr(X( t2 ), · · · , X( tn )|X( t), X( t1 ))
⇥
Pr(X( t2 ), · · · , X( tn ))
1
Pr(X( t1 )|X( t2 ), · · · , X( tn ))
Pr(X( t), X( t1 )) Pr(X( t2 ), · · · , X( tn )|X( t1 ))
Pr(X( t1 )) Pr(X( t2 ), · · · , X( tn )|X( t1 ))
Pr(X( t), X( t1 ))
,
Pr(X( t1 ))
so equality (8.7) holds.
(8.7)
8.8. REVERSIBILITY
231
Recall that Q is the transition rate matrix of X(t) and Q⇤ is the transition rate matrix of Y (t).
We now establish a connection between Q and Q⇤ . For i 6= j and sufficiently small , we have
Pr(Y (t + ) = i|Y (t) = j)
Pr(Y (t + ) = i)
Pr(Y (t) = j)
⇡i
) = i)
⇡j
= Pr(Y (t) = j|Y (t + ) = i)
= Pr(X( t) = j|X( t
= Qij
⇡i
.
⇡j
Therefore, we have
Q⇤ji = Qij
⇡i
.
⇡j
⇤
The following theorem is a very useful result for computing the stationary distribution of a
CTMC.
Theorem 8.8.2 Let X(t) be a CTMC with rate transition matrix Q. If there exists a rate transition
matrix Q⇤ and probability vector ⇡ such that
Q⇤ij ⇡i = Qji ⇡j ,
8i, j,
then Q⇤ must be the rate transition matrix of the reverse chain and ⇡ must be the stationary
distribution of the forward and reverse chain.
Proof First, ⇡ must be the stationary distribution because
X
X Q⇤ij
⇡j Qji =
⇡j
⇡i
⇡j
j
j
X
=
Q⇤ij ⇡i
j
= 0,
where the last equality holds because the row sum of a rate transition matrix equals to zero.
Further, Q⇤ must be the rate transition matrix of the reversed chain since we have already
shown that if Q⇤ is the rate transition matrix of the reversed chain, then
Q⇤ij ⇡i = Qji ⇡j
8i 6= j.
⇤
Next, we introduce the concept of reversible CTMCs, and a lemma which states the local balance
equation implies that a CTMC is reversible.
Definition 8.8.1 (Reversibility) A CTMC X(t) is time-reversible (or simply reversible) if Y (t) =
X( t) has the same stationary distribution as X(t). Since Q⇤ completely determines the statistics
of Y (t), a CTMC is reversible if and only if
Q⇤ij = Qij .
⇤
232
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Lemma 8.8.3 A CTMC is reversible if and only if the local balance equation is satisfied, i.e.,
⇡i Qij = ⇡j Qji
Proof Recall that
Q⇤ij =
Qji ⇡j
.
⇡i
According to the definition, a CTMC is reversible if
Q⇤ij = Qij ,
which implies that
⇡i Qij = ⇡j Qji .
Now if ⇡Qij = ⇡j Qji holds, then
Q⇤ij =
Qji ⇡j
= Qij .
⇡i
So the CTMC is reversible.
⇤
We next discuss several applications of the concept of reversibility, including applications to
M/M/1 queue and the Jackson network.
8.8.1
The M/M/1 Queue
Consider an M/M/1 queue with arrival rate and service rate µ. Assume that the queue is in
steady-state, which is important since it is part of the definition of reversibility.
For the M/M/1 queue, the local balance equation holds, so an M/M/1 queue in steady-state
is reversible, which implies that the reversed Markov chain is indistinguishable from the forward
Markov chain. Note that, in the forward chain, the queue size goes up by one at each arrival; while
in the reversed chain, the queue size goes up by one at each departure. Since the Markov chain is
reversible, the following claims hold:
(i) Since the time instants at which arrivals occur form a Poisson process, so the time instants
at which departures occur must also form a Poisson process with the same rate, i.e., the
departure process is also Poisson of rate .
(ii) Since future arrivals are independent of current queue size, the queue size must be independent
of past departures.
Results (i) and (ii) together are called Burke’s theorem, which also holds for M/M/s and M/M/1
queues.
8.8. REVERSIBILITY
233
Figure 8.9: A tandem M/M/1 queue
8.8.2
The Tandem M/M/1 Queue
Using our observations about M/M/1 queues, we can study a tandem M/M/1 queue, which shown
in Figure 8.9.
First we know that the departure process from queue 1 is Poisson in steady-state. So the second
queue is also an M/M/1 queue. Further, the queue length at the second queue depends only on the
past arrivals to the second queue. But the past arrivals to the second queue are the past departures
from the first queue which are independent of the current queue length of the first queue. Therefore,
the first queue and the second queue are independent. If ⇢ = µ1 < 1 and ⇢2 = µ2 < 1, then
⇡(i, j) = the steady state Pr(q1 = i, q2 = j)
= ⇢i1 (1
⇢1 )⇢j2 (1
⇢2 ).
Such a solution is called a product form solution:
⇡(i, j) = ⇡1 (i)⇡2 (j),
where ⇡1 and ⇡2 are the steady-state distributions of the first and second queues, respectively.
The above example motivates us to ask the following question: are there more complicated
networks for which the product-form solution holds, i.e., where the steady-state distribution at each
queue can be multiplied to get the overall steady-state distribution for the network? We next present
several such examples.
8.8.3
The Multi-class M/M/1 Queue
Consider an M/M/1 queue where arrivals consist of K classes. The arrival rate of class-k customers
is k and the mean service time of a class-k customer is 1/µk . Assume that the network is in state
x such that x = (c1 , · · · , cn ), where ci is the class of the ith customer in the queue, and the 1st
customer is the one at the head of the queue. We next show that if µk = µ for all k, then the
steady-state distribution of this multi-class M/M/1 queue is in product form and
⇡(x) = (1
⇢)
n
Y
⇢ci ,
(8.8)
i=1
P
where ⇢ = K
k=1 ⇢k and ⇢k = k /µk .
Given the queue is at state x = (c1 , · · · , cn ), the system moves to state (c1 , · · · , cn , cn+1 ) with
rate cn+1 (with an arrival belonging to class cn+1 ), and moves to state (c2 , · · · , cn ) with rate µc1
(with the departure of the customer at the head of the queue). For n > 0, we first prove that
following equality:
!
K
K
X
X
⇡(x)
+
µ
=
⇡(c
,
·
·
·
,
c
)
+
⇡(k, c1 , · · · , cn )µk ,
(8.9)
c1
1
n 1 cn
k
k=1
k=1
234
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
as illustrated in Figure 8.10.
Figure 8.10: Illustration of local transitions when the queue is in state (c1 , · · · , cn )
Substituting ⇡(x) in (8.8) in the equation above, we obtain
! K
!
n
Y
X
(1 ⇢)
⇢ci
k + µ c1
= (1
i=1
n
Y1
⇢)
i=1
⇢ci
!
k=1
cn
+
K
X
(1
n
Y
⇢)⇢k
⇢ci
i=1
k=1
Canceling out the common terms at both sides, we have
⇢cn (
K
X
k + µ c1 ) =
cn
k=1
+
K
X
⇢k ⇢cn µk ,
k=1
which is equivalent to
⇢cn
K
X
k
+ µ c1
k=1
!
=
=
cn
cn
+ ⇢cn
+ ⇢cn
K
X
k=1
K
X
⇢k µk
k,
k=1
which implies that
⇢cn
K
X
k=1
k
+ µ c1
!
= ⇢cn
µ cn +
K
X
k=1
k
!
.
!
µk .
8.8. REVERSIBILITY
235
If µcn = µ for any n, then the above inequality is satisfied, and (8.9) holds when n > 0.
Now consider the case where n = 0. That equality (8.9) holds requires
K
X
⇡(x)
k
k=1
!
=
K
X
⇡(i)µi .
i=1
Substituting (8.8) into the equation above, we obtain
(1
⇢)
K
X
k
k=1
!
= (1
⇢)
K
X
⇢i µi ,
i=1
which is equivalent to
K
X
k
=
K
X
⇢i µi =
i=1
k=1
K
X
i.
i=1
So (8.9) holds for n = 0.
According to LemmaP
8.1.6, equality (8.9) means the global balance equation holds, so ⇡Q = 0.
It is easy to verify that x ⇡(x) = 1, so we can conclude that ⇡(x) is the stationary distribution
of the CTMC, which is in product form.
8.8.4
The Jackson Network
We consider a network where external arrivals into node i is a Poisson process with rate ri . A
customer moves from node i to node jPwith probability Pij after the service is completed, and
leaves the network with probability 1
j Pij = Pi . Therefore, the aggregated arrival to node i is
i
= ri +
X
j Pji .
j
We assume that there exists a unique solution to the equations above that satisfies i < µi for
all i. This can be guaranteed under a fairly general assumption that for every node j, there exists
node i such that Pi > 0 and a route (r1 , r2 , · · · , rk ) such that Pjr1 Pr1 r2 · · · Prk 1 rk Prk i > 0. This
assumption implies that every customer will eventually leave the network with probability one.
This network is called the Jackson network. A Jackson network with two nodes is illustrated in
Figure 8.11.
Theorem 8.8.4 The Jackson network has a stationary distribution and the distribution is in product form:
K
Y
⇡(n) =
⇢ni i (1 ⇢i ),
(8.10)
i=1
where n = (n1 , · · · , nk ), ni is the number of customers at node i, and ⇢i =
µi .
i
236
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Figure 8.11: A Jackson network with two nodes
Proof To prove this theorem, we first introduce the following notations:
Ti (n) = (n1 , · · · , ni
1, ni+1 , · · · , nk )
Tij (n) = (n1 , · · · , ni
1, · · · , nj + 1, · · · , nk ).
i
T (n) = (n1 , · · · , ni + 1, ni+1 , · · · , nk )
Note that state Ti (n) is the state after one customer left node i when the network was in state n,
and state T i (n) is the state after one customer arrived at node i when the network was in state
n, and state Tij (n) is the state after one customer moved from node i to node j when the network
was in state n.
We now check the global balance equation:
0
1
K
K
K
K
X
X
X
X
⇡(n) @
µi Pi Ini >0 +
ri +
µi Pij A
i=1
=
K
X
⇡(Ti (n))Ini >0 ri +
i=1
K
X
+
i=1
K
X
i=1 j=1,j6=i
⇡(T i (n))µi Pi
i=1
K
X
⇡(Tij (n))µj Pji Ini >0 .
i=1 j=1,j6=i
Substituting
⇡(n) in (8.10) in the equation above, we can verify the equality holds. Further
P
⇡(n)
=
1,
so the theorem holds.
n
⇤
We next present an alternative proof.
Proof Let Q̃ be the rate transition matrix of a Jackson network with external arrivals rate:
0
1
X
r̃i = i @1
Pij A ,
j
and assume the probability of moving from node i to node j is
P̃ij =
j Pji
i
.
8.9. INSENSITIVITY TO SERVICE-TIME DISTRIBUTIONS
237
We can verify that this is indeed the reversed chain and (8.10) is in fact the stationary distribution
based on Theorem 8.8.2.
⇤
8.9
Insensitivity to Service-Time Distributions
In previous sections, we studied the stationary distributions of FIFO queues with di↵erent arrival
and service models, and have seen that the stationary distributions depend on both the arrival
process and the service-time distribution. However, certain queueing models have stationary distributions that are insensitive to service-type distributions, i.e., the stationary distribution depends
only on the mean of the service and not on the entire distribution. In this section, we will study
one such queueing model.
The queue system that we consider in this section is the M/M/1 processor sharing model.
Under processor-sharing, if there are n customers in the queue, the capacity of the queue is equally
divided between the n customers. This is di↵erent from the M/M/1 queue studied earlier where
the queue is operated in FIFO (first-in, first-out) fashion. In a FIFO queue, the server devotes its
entire capacity to the first customer in the queue.
8.9.1
The M/M/1-Processor Sharing (PS) Queue
The Markov chain for the M/M/1-PS queue is the same as the Markov chain for the M/M/1-FIFO
queue because each packet departs at rate nµ in the PS queue but since there are n customers, the
total departure rate is µ.
Therefore, the stationary distribution of the M/M/1-PS queue is
⇡(n) = ⇢n (1
⇢).
Next, we consider M/GI/1-PS queue.
8.9.2
The M/GI/1-PS Queue
The Markov chains for M/GI/1-FIFO and M/GI/1-PS queues are di↵erent. We will first consider a
special case, where the service time are Erlang distributed, and show that the stationary distribution
of the number of customers in an M/GI/1-PS queue is insensitive to the parameters of the Erlang
distribution as long as the mean service time is the same.
Definition 8.9.1 (Erlang Distribution) X is an Erlang distributed random variable if
X = X1 + X2 + · · · + XK
⇤
where Xi ⇠ exp(Kµ) and Xi0 s are i.i.d.
Note that for an Erlang distributed random variable,
E[X] =
1
µ
and
V ar(X) =
1
.
Kµ2
238
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Further, as K ! 1, X approximates a constant service time µ1 . Let us consider a PS-queue with
service times that are Erlang with K phases. A packet with an Erlang-K service time distribution
can be thought as having K phases of service: the packet completes the ith phase and then moves
to the (i + 1)th phases.
Let (c1 , c2 , · · · , cn ) be the phases of the n packets in the system. When a new packet arrives, it
randomly picks a number {1, 2 · · · , n + 1} as its index. A packet departs when it has completed K
phases. If the ith packet departs, we relabel customers (i + 1, i + 2, · · · , n) as (i, i + 1, · · · , n 1)
respectively.
Theorem 8.9.1 The stationary distribution of the M/GI/1-PS queue satisfies
⇡(c1 , c2 , · · · , cn ) =
⇢n (1 ⇢)
,
Kn
(8.11)
where ⇢ = µ . Thus, the steady-state probability that there are n customers in the queue is
K X
K
X
c1 =1 c2 =1
···
K
X
cn =1
⇡(c1 , · · · , cn ) = ⇢n (1
⇢)
which is the same as the M/M/1-PS queue.
Proof To prove (8.11), we use the concept of a reversed chain. In the forward chain,
(c1 , · · · , ci , · · · , cn ) ! T i+ (c) , (c1 , · · · , ci + 1, · · · , cn )
at rate
Kµ
if ci < K,
n
th phase.
which occurs when the ith customer moves from the cth
i phase to the (ci + 1)
(c1 , · · · , ci
1 , K, ci+1 , · · ·
, cn ) ! Ti (c) , (c1 , · · · , ci
1 , ci+1 , · · ·
, cn )
at rate
Kµ
,
n
which occurs when the ith customer leaves the queue.
(c1 , · · · , ci , ci+1 , · · · , cn ) ! T i,1 (c) , (c1 , · · · , ci , 1, ci+1 , · · · , cn )
at rate
n+1
,
which occurs when a customer joins the system and is placed between the ith customer and (i + 1)th
customer. Note that there are n + 1 positions a joining customer can select, so the transmission
rate is /(n + 1).
In the reversed chain, we have
(c1 , · · · , ci
(c1 , · · · , ci , · · · , cn ) ! T i (c) , (c1 , · · · , ci
1 , 1, ci+1 , · · ·
, cn ) ! Ti (c) , (c1 , · · · , ci
(c1 , · · · , ci , ci+1 , · · · , cn ) ! T
i,K
1, · · · , cn )
1 , ci+1 , · · ·
, cn )
(c) , (c1 , · · · , ci , K, ci+1 , · · · , cn ).
Note that c in the reversed chain can be interpreted as the vector of remaining phases of each
packet. To verify that (8.11) is the stationary distribution, we utilize Theorem 8.8.2. We will
construct Q⇤ such that
⇡(c)
Q⇤c,ĉ = Qĉ,c
,
⇡(ĉ)
and show that Q⇤ is the rate transition matrix of the reverse chain and (8.11) is the stationary
distribution of the CTMC. We consider the following three cases:
8.9. INSENSITIVITY TO SERVICE-TIME DISTRIBUTIONS
239
(1) We know that
Qc,T i+ (c) =
Kµ
,
n
and
⇡(c) = ⇡(T i+ (c)) =
so
Q⇤T i+ (c),c = Qc,T i+ (c)
⇢n (1 ⇢)
,
Kn
⇡(c)
Kµ
=
.
⇡(T i+ (c))
n
(2) We know that
Qc,Ti (c) =
and
⇡(c) =
Kµ
,
n
⇢n (1 ⇢)
⇢n 1 (1
and
⇡(T
(c))
=
i
Kn
Kn
⇢)
1
,
so we have
Q⇤Ti (c),c = Qc,Ti (c)
⇡(c)
= .
⇡(Ti (c))
n
(3) We have that
Qc,T i,1 (c) =
n+1
,
and
⇡(c) =
⇢n (1 ⇢)
⇢n+1 (1 ⇢)
and ⇡(T i,1 (c)) =
,
n
K
K n+1
so we can obtain that
Q⇤T i,1 (c),c = Qc,T i,1 (c)
Next we show that
X
Qc,ĉ =
ĉ6=c
⇡(c)
Kµ
=
.
i,1
⇡(T (c))
n+1
X
Q⇤c,ĉ .
ĉ6=c
Consider the case where 1 < ci < K for all i. In this case, no departure will occur in the forward
chain because ci < K for all i; and no departure will occur in the reversed chain because ci > 1 for
all i. The equality above holds because
X
Qc,ĉ =
n
X
i=1
ĉ6=c
= n
=
Qc,T + (c) +
i
Kµ
+
n
+ Kµ,
n+1
X
i=1
Qc,T i,1 (c)
240
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
and
X
Q⇤c,ĉ =
n
X
Q⇤c,T (i,K) (c) +
i=1
ĉ6=c
=
=
n
X
Q⇤c,T i
(c)
i=1
Kµ
n
n
n
+ Kµ.
n+
P
P
Since ĉ6=c Qc,ĉ + Qcc = 0 and Qcc = Q⇤cc , we have that ĉ Q⇤c,ĉ = 0. The other cases can be
checked similarly.
We therefore have that Q⇤ is also a rate transition matrix. According to Theorem 8.8.2, we
conclude that Q⇤ is the rate transition matrix of the reversed chain and (8.11) is the stationary
distribution.
⇤
Next, let us consider an M/GI/1-PS queue with mixtures of Erlang distributions. In general, a
mixture of Erlang-K distributions is defined as follows.
Definition 8.9.2 (Mixture of Erlang-K Distribution) With probability pj , the service time
follows Erlang{Kj , µj }, such that
1
X
j 1
pj = 1
and
1
1 X pj
,
.
µ
µj
j=1
⇤
The following lemma states that any distribution can be approximated by a mixture of Erlang-K
distributions.
Lemma 8.9.2 Any random variable X taking value in on [0, 1) can be approximated by a mixture
of Erlang-K distributions. Specifically, given any
> 0, we can construct a mixture of Erlang-K
distributions that takes the value k with probability Pr (k  X  (k + 1) ) as K ! 1 for all
k 0.
Proof We present the basic intuition behind the statement of the theorem. Consider a distribution
with cumulative distribution function (CDF) F (x) as shown in Figure 8.12, and assume that Pr(y 
x  y + ) = py .
We then define a mixture of Erlang-K distributions such that with probability py , it is Erlang(K, y1 ),
y
i.e., K phases each with mean K
. It is easy to see that as K ! 1, we are approximating a random
variable which takes the value y (constant). Thus, as K becomes large, the random variable takes
the value y with probability py . Thus, for small delta, we are approximating the desired CDF.
⇤
Consider an M/GI/1-PS system with the above service time distribution. Suppose there are n
packets in the system. The state of the system is
x , {(t1 , c1 ), (t2 , c2 ), · · · , (tn , cn )},
8.9. INSENSITIVITY TO SERVICE-TIME DISTRIBUTIONS
241
Figure 8.12: A general CDF F (x)
where ti is the type of Erlang distribution chosen at arrival using the probabilities {p1 , p2 , · · · , }
and ci is the phase that the packet is currently in.
Using Theorem 8.8.2, we can show that
⇡
ˆ (t1 , c1 ) · · · ⇡
ˆ (tn , cn )
Z
⇡(x) =
is the stationary distribution of the CTMC, where
⇡
ˆ (ti , ci ) =
pt i
Kt i µ t i
and Z is a normalization constant so that
X
⇡(x) = 1.
x
To compute the steady-state probability of n packets in the system, note that
⇡(n) =
1
X
t1 =1
Since
Kti
1 X
X
ti =1 ci =1
···
Kt1
1 X
X
tn =1 c1 =1
···
Ktn
X
@(x).
cn =1
1
X
pt i
pt i
1
=
= ,
Kt i µ t i
µti
µ
ti =1
where the second equality holds due to the definition of µ, we obtain
✓ ◆n
1
⇡(n) =
,
Z µ
When n = 0 (i.e., x = ;) by Little’s law,
⇡(;) = 1
⇢,
242
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
which implies that
⇡(;) =
1
= (1
Z
⇢),
and
⇢)⇢n .
⇡(n) = (1
8.10
Connection-Level Arrivals and Departures in the Internet
In Chapter 2, we studied resource allocation in the Internet assuming that the set of flows in the
network is fixed. In reality, flows dynamically arrive and depart. We now use CTMCs to model
to study this connection level arrivals and departures, and characterize necessary and sufficient
conditions for connection-level stability.
Denote by nr the number of flows on route r. A flow is called a type-r flow if the flow is on
route r. We assume that each flow of type-r arrives according to a Poisson process of rate r , and
the file associated with the flow has an exponentially distributed number of bits with mean µ1r . A
flow departs when all of its bits are transferred. For example, if a flow of type-r and with B bits
arrives at time t, then it will depart at time T > t, where T is the smallest time such that
Z T
xr (s) ds = B,
t
where xr (t) is the transmission rate allocated to a flow on route r.
After the flow joins the network, the transmission rate of the flow is regulated by congestion
control. We assume a time-scale decomposition assumption such that congestion control occurs
instantaneously compared to file arrivals/departures. Specifically, we assume that the network
allocates xr to a flow on route r by solving the following optimization problem:
X
nr Ur (xr )
r
subject to
X
r:l2r
where Ur (·) is a concave function.
We assume that
X
r:l2r
r
µr
nr x r  c l
xr
0,
< cl or
X
8l
⇢r < c l ,
r:l2r
i.e., the aggregated workload on each link is less than the link capacity. Let nr (t) denote the
number of files of type r at time t. The question is is the network stable, or does n(t) has a welldefined stationary distribution? Note that arbitrary resource allocation rules may not fully utilize
resources, as the following example shows.
Example 35 Consider a tandem network with three nodes and two links. Assume there are three
types of flows in the networks, where type-1 flows only use link 1, type-2 flows only use link 2, and
8.10. CONNECTION-LEVEL ARRIVALS AND DEPARTURES IN THE INTERNET
243
Figure 8.13: A linear network with two nodes and three links
type-0 flows uses both link 1 and link 2 as shown in Figure 8.13. Further, assume that µr = 1 for
all r, and cl = 1 for both links. Note that any rate allocation among the three types of flows need to
satisfy:
n0 x 0 + n1 x 1  1
n0 x0 + n2 x2  1.
Consider the rate allocation scheme that gives absolute priority to flows of type-1 and -2 over
type-0 flows. So if n1 > 0, then the rate allocation (x0 , x1 , x2 ) satisfies
n0 x 0 = 0
and
n1 x1 = 1.
Similarly, if n2 > 0, the rate allocation satisfies
n0 x 0 = 0
and
n2 x2 = 1.
Clearly we need 1 < 1 and 2 < 1 for stability in this case. What about
type-0 can use the network only if n1 = n2 = 0, and
Pr(n1 = 0) = 1
1
and
Pr(n2 = 0) = 1
0?
Note that flows of
2.
Since n1 and n2 are independent, we have
Pr(n1 = n2 = 0) = (1
which implies that
0
1 )(1
2 ),
should satisfy
0
< (1
1 )(1
2)
for stability. This region is smaller than the following region
0
+
1
<1
and
0
+
1
< 1.
(8.12)
We will soon show that (8.12) is the capacity region under the utility maximization framework. In
other words, the network can be stabilized under the utility maximization framework if (8.12) is
satisfied while the resource allocation that gives priority to type-1 and type-2 flows cannot achieve
the full capacity region.
⇤
244
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Now we study connection-level stability under the utility maximization framework. Assume we
have R types of flows in the network. Define n to be a R-vector, where nr is the number of type-r
flows in the network. Further define R-vector er such that
er = (0, 0, . . . , 0, 1, 0, 0, . . . , 0)T ,
i.e., all entries of er is zero except the rth entry which is one.
Since file arrivals are Poisson and file sizes are exponential, n is a CTMC, and
n ! n + er
n !n
at rate
er
r
if nr > 0, at rate µr nr xr .
Assume that the utility functions are of the following form:
(
1 ↵
wr x1r ↵
if ↵ 6= 1, ↵ > 0
Ur (xr ) =
wr log xr if ↵ = 1.
Theorem 8.10.1 The CTMC is positive recurrent when
P
r:l2r
⇢r < cl for all l.
Proof We consider the following Lyapunov function:
X r
V (n) =
n1+↵
,
r
1
+
↵
r
where r will be chosen later. By the Foster-Lyapunov theorem (Theorem 8.1.8), for CTMCs, we
need to consider the drift
X
(V (j) V (i))Qij .
j:j6=i
For our model, the drift is
X
(V (j) V (i))Qij
j:j6=i
=
X r
(nr + 1)1+↵
1
+
↵
r
n1+↵
r
r
+
X r
(nr
1+↵
r
1)1+↵
n1+↵
µ r nr x r ,
r
where µr nr xr = 0 if nr = 0. Note according to the Taylor series and the mean-value theorem,
1
(nr + 1)1+↵
1+↵
1
n1+↵
= n↵r + ↵ñ↵r
r
2
1
,
nr  ñr  nr + 1
1
1
(nr 1)1+↵ n1+↵
= n↵r + ↵n̂↵r 1 , nr 1  n̂r  nr .
r
1+↵
2
After ignoring the n̂r and ñr terms, since they will be small compared to n↵r when nr is large,1 we
obtain
X
P
(V (j) V (i))Qij = r r n↵r ( r µr nr xr )
j:j6=i
=
1
P
r
r µr n↵r (⇢r
nr xr ).
This argument is straightforward but tedious. We leave the details to the reader.
8.10. CONNECTION-LEVEL ARRIVALS AND DEPARTURES IN THE INTERNET
Defining nr xr = Xr , we further have
X
X
(V (j) V (i))Qij =
r µr n↵r (⇢r
Xr )
r
j:j6=i
where {Xr } solves
245
X w r n r ✓ X r ◆1
max
1 ↵ nr
r
X
↵
such that
l:l2r
Xr  c l
8l,
which is equivalent to
X w r n↵
r
Xr1
0
1
↵
r
max
Xr
↵
X
such that
l:l2r
Xr  c l
8l.
Recall that for a convex function f (x) over a convex set C,
f (x) + rf T (x)(y
x)
f (y).
Suppose y = x⇤ , the maximizer, then the inequality above implies that
rf T (x)(x⇤
f (x⇤ )
x)
0 8x
f (x)
or
x⇤ )  0
rf T (x)(x
P
8x.
Since r:l2r ⇢r < cl implies that there exists ✏ > 0 such that {⇢r (1 + ✏)} is within the capacity
region, i.e., {⇢r (1 + ✏)} is a feasible value, we have
X
r
wr n↵r
(⇢r (1 + ✏)
(⇢r (1 + ✏))↵
Xr )  0,
which implies that
X w r n↵
r
r
Choosing r µr =
wr
⇢↵
r
⇢↵r
or r =
wr
,
µr ⇢↵
r
X
Xr ) 
(⇢r
✏
X w r n↵
r
⇢↵r
r
⇢r =
X w r n↵
r
r
↵ 1
⇢r
(⇢r
Xr )
✏
.
we obtain that
(V (j)
V (i))Qij
=
X w r n↵
r
⇢↵r
r
j:j6=i

✏
X w r n↵
r
r
↵ 1
⇢r
.
We therefore conclude that the CTMC is positive recurrent by the Foster-Lyapunov theorem.
⇤
From the theorem
above,
we
have
that
the
network
is
stable
under
the
utility
maximization
P
P
framework if
r:l2r ⇢r < cl for all l. It is not difficult to see that if
r:l2r ⇢r > cl , then the
network cannot be stable. Thus, the utility maximization framework achieves the largest possible
connection-level throughput.
246
8.11
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Distributed Admission Control
In this section, we consider the admission control in a communication network. Consider a single
server system with n sources, each generating packets at rate p according to a Poisson process.
The packet sizes of a source is exponentially distributed with mean 1/µp . So the system can be
modeled as an M/M/1 queue with arrive rate n p and average service time 1/µp (per packet).
Assume that the objective of the admission control is to guarantee that
Pr(q(1)
B)  ✏,
where q(1) denotes the queue length in steady state. We can use the M/M/1 formula to find
Nmax , the maximum number of sources that can be admitted in the network to satisfy the QoS
requirement. So the admission control should admit no more than Nmax sources, and a centralized
control should block a new source if there are already Nmax sources admitted in the network.
Further if we assume dynamic flow arrivals/departures, and assume that sources arrive according
to a Poisson process of rate and stay for an exponentially distributed amount of time with mean
1/µ. Then, the number of sources in the network under a centralized admission control can be
modeled as an M/M/Nmax /Nmax queue, and the blocking probability (the probability a source is
not admitted when it arrives at the system) can be obtained based on the M/M/Nmax /Nmax model.
We are interested in distributed admission control schemes which closely approximate the above
centralized admission control model. Assume that sources need to make their own decisions on
whether they should join the network.
Algorithm 11 Distributed Admission Control
1: Before joining the network, a source sends m probing packets.
2: A probing packet is marked when it arrives and finds at least B̃ packets in the queue.
3: if one of the probing packets is marked then
4:
the source does not join the network.
5: else
6:
the source joins the network.
7: end if
Consider the following distributed admission control scheme which is obtained using a time-scale
decomposition assumption by which the packet queue converges to the steady state instantaneously
compared to the time scale of source arrivals/departures. Let pmark (n) denote the probability a
probing packet is marked when there are n sources in the network. We assume the packets sizes
are exponential. So
pmark (n) = Pr(q (n) (1) B̃),
where q (n) (1) is the steady-state queue length of the M/M/1 queue with arrival
$ rate
% n
average service time 1/µp . Assume that pmark (n) = 1 if n
µ and define N̄ =
µ
p
and
. Typically,
we have Nmax < N̄ .
Further, we denote by p(n) the probability that a new source joins the network when there are
n other sources in the network. Since a new source joins the network only if none of its m probing
8.12. LOSS NETWORKS
247
packets is marked, we have
p(n) = 1
(1
pmark (n))m .
Figure 8.14: The CTMC for distributed admission control
The network can be modeled as a CTMC as shown in Figure 8.14. By studying this CTMC,
we can compute Pr(n Nmax ), and
pb = the fraction of rejected flows.
Note by Little’s Law,
E[n] = pb
so
pb =
1
= pb ⇢,
µ
E[n]
.
⇢
The parameters m (the number of probing packets), B̃ (the threshold for marking a probing packet)
can be used to tune the trade-o↵ between Pr(n Nmax ) and pb .
8.12
Loss Networks
Loss networks are models of circuit switched networks traditionally. Telephone networks are operated in the circuit switched mode. Each telephone call requests one unit of resource (traditionally,
it was 64 Kbps, but can be much less using compression techniques). Each link l in the network
has capacity cl , where cl is an integer. If cl = 10, it means that the link can carry 10 telephone
calls at a time. Assume that a route from a source S to destination D is fixed. We will denote a
route by r. It is simply a set of links connecting S to D.
Example 36 Consider the network shown in Figure 8.15. Links (1, 5) and (5, 6) can be a route
connecting source 1 and destination 6.
A call on route r can be admitted to the network only if there is at least one unit of capacity
on all the links on the route. Otherwise, the call is blocked and lost forever. We are interested in
the following question: given traffic statistics, how can we compute the fraction of calls lost? This
question will be studied in this section with the following assumptions:
• The traffic to route r is a Poisson process of rate
r.
• Calls stay in the network for an exponential amount of time, with mean 1/µ, and then depart.
248
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Figure 8.15: A circuit switched network.
• The arrival processes and service times are independent of each other.
Under the assumptions above, the circuit-switched network can be modeled as a loss network,
i.e., a network generalization of the M/M/s/s loss model. The following notations will be used
throughout this section:
• nr : the number of calls on route r,
• R : the set of all possible routes,
• n : the system state given by (n1 , n2 , . . . , nR ), where R = |R|, and
• ⇡(n) : the steady state distribution of state n.
In loss networks, the number of calls on a link can not exceed the link capacity, so we have the
constraints
X
nr  c l
(8.13)
r:l2r
for all link l. Note that l 2 r means link l is a part of route r. The network state n is a CTMC,
we will compute the steady-state distribution of this CTMC. We write the local balance equations
first:
r ⇡(n1 , . . . , nr , . . . , nR ) = (nr + 1)µ⇡(n1 , · · · , nr + 1, · · · , nR ).
Defining ⇢r =
r /µ,
we get
⇢r ⇡(n1 , · · · , nr , · · · , nR ) = (nr + 1)⇡(n1 , · · · , nr + 1, · · · , nR ).
It is easy to verify that
⇡(n) =
Q
r
⇢n
r
r2R nr !
Z
(8.14)
satisfies the above equation for a suitable normalization constant Z, where Z is computed by noting
that
X
⇡(n) = 1.
n:
P
r:l2r
nr cl 8l
8.12. LOSS NETWORKS
249
In other words,
Z=
n:
P
X
r:l2r
nr cl 8l
⇢nr r
.
nr !
Before we proceed further, it is useful to understand how (8.14) was derived. Suppose that
cl = 1 for all l. Then, the n0r s are independent and each nr behaves like an M/M/1 queue. In
nr
nr
Q
this case, ⇡(nr ) = ⇢nrr ! and ⇡(n) = r2R ⇢nrr ! . Therefore, (8.14) simply is a renormalization of the
above expression to account for the constraints on n. Luckily, such a guess satisfies the local balance
equations and hence is the correct steady-state distribution. This is an example of the “truncation
theorem” for reversible MCs (Exercise 8.10). If we can calculate Z, then the steady-state probability
of a call being blocked on route r can be calculated using the PASTA property (see Section 8.4) as
br = Pr (at least one link on route r is full) =
X
⇡(n),
n2Br
where
Br =
(
n:
X
s:l2s
ns = cl for some l 2 r
)
.
Except for small networks, this probability is difficult to compute since Z is very difficult to compute
for large networks and the cardinality of the set Br is very large. So we have to come up with an
approximation, which we do next.
8.12.1
Reduced-Load Approximation
Consider a regime where the arrival rates and link speeds are all scaled by a factor k, i.e., we
assume that cl = kc̃l and r = k ˜ r , where we will eventually let k ! 1. In other words, we want
to consider a large capacity network, with correspondingly large arrival rates. We further define
xr = nkr , so the constraints (8.13) can be written as:
X
r:l2r
xr  c̃l .
(8.15)
Further, we rewrite ⇡(n) as
⇡(n) =
1 Y enr log ⇢r
.
Z
nr !
r2R
Next, we use Striling’s formula
nr ! = enr log nr
So that
⇡(n) =
1 Pr2R nr log ⇢r
e
Z
nr +O(log nr )
.
nr log nr +nr O(log nr )
,
which implies that
⇡(x) =
1 Pr2R kxr log k ˜ /µ
e
Z
kxr log(kxr )+kxr O(log(kxr ))
.
250
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Note that
X
r2R
=
X
r2R
= k
⇣
⌘
kxr log k ˜ r /µ
⇣
⌘
kxr log ˜ r /µ
X
r2R
⇣
X
⇣
x1r log ˜ r /µ
r2R
⌘
kxr log xr + kxr
xr log ˜ r /µ
Given x1 and x2 such that
x1r log x1r + x1r
kxr log(kxr ) + kxr
⌘
!
xr log xr + xr
X
⇣
r2R
O(log(kxr ))
!
x2r log ˜ r /µ
O(log(kxr ))
O
⌘
✓
log(kxr )
k
◆
x2r log x2r + x2r
.
!
=
> 0,
as k ! 1, the steady state distribution of x2 is negligible compared to that of x1 because
⇡(x2 )
= lim e
k!1 ⇡(x1 )
k!1
k
lim
= 0.
Therefore, as k ! 1, the network will converge to a state x which solves the following optimization
problem:
P
max r2R xr log ˜ r /µ xr log xr + xr
P
subject to
r:l2r xr  c̃l 8l
xr
0.
Using the Lagrange multipliers, we obtain
L(x, p) =
X
xr log ˜ r /µ
X
xr log xr + xr
r2R
Note that
l
pl
X
r:l2r
xr
c̃l
!
.
@L
(x, p) = 0
@xr
implies that
log ˜ r /µ
log xr =
X
pl ,
X
pl
l2r
which further implies that
log xr = log ˜ r /µ
l2r
and
xr
=e
˜ r /µ
P
l2r
pl
.
According to the definitions of x and ˜ , we have
xr
nr
nr
=
=
,
˜ r /µ
⇢r
r /µ
(8.16)
8.12. LOSS NETWORKS
251
so that
P
nr
=e
⇢r
l2r
pl
.
(8.17)
By Little’s law
1
br ) ,
µ
br ) is the arrival rate of accepted calls on route r, which implies that
E[nr ] =
where
r (1
1
br =
r (1
E[nr ]
E[xr ]
=
⇡e
˜ r /µ
⇢r
P
l2r
pl
,
(8.18)
where the P
last approximation follows from the observation
that the network converges to state
P
˜r
p
p
l
l
l2r
l2r
xr = µ e
as k ! 1 (equation (8.16)). So e
provides an estimate of the blocking
probability if pl can be computed.
From the KKT conditions, pl satisfies
!
X
pl
xs c̃l = 0
s:l2s
for all l, in other words,
pl
X
ns
cl
s:l2s
!
=0
for all l. Replacing ns using equation (8.17), we obtain
pl
X
s:l2s
⇢s e
P
j2s pj
cl
!
=0
(8.19)
for all l, which provides a set of equations to compute pl .
Further, consider a route r = l, i.e., a single link route. From approximation (8.18), we have
1 br={l} ⇡ e pl , i.e., the probability that the calls on route r = {l} are blocked is approximately
e pl . Now assume all routes that use link l experience the same blocking probability, then bl , e pl
is an approximation of the blocking probability on link l.
Then (8.18) becomes
Y
1 br =
(1 bl ) ,
l2r
which suggests that the links behave independently in a large network. Further, (8.19) becomes
2
3
⇢
X
Y
= cl , if bl > 0
⇢s 4 (1 bj )5
 cl , if bl = 0.
s:l2s
j2s
This reinforces the blocking probability interpretation for bl because bl > 0 must mean that the
total number of calls on the link must be equal to cl in the large network limit. These statement
can be made precise, but we will not do so here. In summary, the reduced-load approximation
states that the blocking probabilities satisfy the following equations:
Q
1 br
= l2r (1 bl )
⇢
X
= cl , if bl > 0
⇢r (1 br )
.
 cl , if bl = 0.
r:l2r
252
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
8.12.2
Computing the Blocking Probability
Motivated by large system approximation, we present a heuristic to compute blocking probabilities,
whereby we compute the blocking probabilities at each link and then use them to compute the endto-end blocking probability on a route. The heuristic is presented as Algorithm 12.
Algorithm 12 A heuristic to compute blocking probabilities using he reduced-load approximation
1:
Assume b̃l are known. then, the arrival rate of route r traffic at link j (if j 2 r) is
⌘
Y ⇣
¯r =
1 b̃l r .
l:l2r,l6=j
Thus, b̃j can be calculated as
0
b̃j = EB @
2:
X ¯r
r:j2r
µ
1
, cj A ,
where EB is the Erlang-B loss formula.
The blocking probability on route r can be computed using the following equation:
⌘
Y⇣
1 br =
1 b̃l .
l2r
Note that step 1 is implemented iteratively, where
0
1
X ¯ r (k)
b̃j (k + 1) = E @
, cj A
µ
r:j2r
⌘
Y⇣
¯ r (k) = r
1 b̃l (k) .
l6=j
8.12.3
Alternate Routing
In the previous sections on loss networks, we assumed that a call is block if there is no available
capacity on its route. Thus, we implicitly assumed that each call is associated with only one route.
In general, a call between a source and its destination can be routed on one of many paths. Often,
there is a primary route which is attempted first and if there is no capacity available on the primary
route, then alternate routes are attempted according to some pre-specified rule. For simplicity, we
consider a fully connected network with N nodes and with all link capacities cl = c. Assume a call
tries the direct link to the destination first and if that is not available, then one of the 2-hop paths
is chosen at random and the call is routed on this path if the capacity is available. Else, the call
is rejected. Computing the blocking probability exactly in such a network is difficult. In fact, one
cannot even write closed form expressions in this case. We will use the reduced-load approximation
as a heuristic to calculate blocking probabilities under alternate routing. It is well-known that the
approximation computes the blocking probability fairly accurately. Assume r = for all r and
µ = 1. Let b̃ be blocking probability of a link. An alternately routed call from direct path r arrives
8.13. THE DOWNLOAD TIME IN BITTORRENT
a link l with probability
b̃ ⇥
1
N
2
253
⇣
⇥ 1
⌘
b̃ ,
where b̃ is the probability that the direct link is not available, 1/(N 2) is the probability of
choosing an alternative path that includes link l, and 1 b̃ is the probability the call is not blocked
by the other link in the alternate path. Thus, the total arrival rate at link l is
+
b̃(1
N
b̃)
2(N
2
2) =
⇣
1 + 2b̃(1
⌘
b̃) .
In the above calculation, we have used the fact that a link can be part of an alternate path for
2(N 2) direct paths. Thus, the fixed-point calculation becomes
⇣
⌘
b̃ = EB (1 + 2b̃(1 b̃)), c .
For this model if we plot b̃ as a function of for fixed c, we will see that two solutions exist (explored
in an exercise at the end of the chapter). Further the two solutions for b̃ may be far apart. In real
networks such phenomena arise, the network switches from low blocking to high blocking regimes
quickly. The reason is as follows: when a call is accepted on an alternate route, while we have
eliminated blocking for this call, we may block two calls (on the two links on the alternate path)
later. So it is not always good to accept calls on alternate paths.
In practice, a call is accepted on an alternate path only if the remaining capacity is no less than
ct on both links for some 0 < ct < c. Such a scheme is called trunk reservation and it reservers some
space for directly routed calls when the network traffic is high. We will see in the exercise that
such a trunk reservation scheme, with a properly chosen parameter ct , helps to reduce blocking
probabilities dramatically.
8.13
The Download Time in BitTorrent
In Chapter 7, we discussed the BitTorrent protocol for P2P file sharing. In this section, we use
a simple CTMC model to understand the performance of BitTorrent, in particular, the amount
of time a peer spends on downloading a file. Recall that in BitTorrent, seeds are peers who have
the complete copy of the file; and leechers are peers who have not downloaded the complete file.
Assume that the arrival process of leechers is a Poisson process with rate , the amount of time a
peer stays in the system after it becomes a seed is exponentially distributed with parameter . Each
node has an upload bandwidth µ and download bandwidth c. To model this system as a CTMC,
we assume that the download times are exponentially distributed, where the parameters depend
on the download speeds.
Let x denote the number of leechers and y denote the number of seeds. Then (x, y) is a CTMC
such that
• The transition rate from state (x, y) to state (x + 1, y) is , which occurs when a new leecher
joins the network.
• The transition rate from state (x, y) to state (x, y
the network.
1) is y, which occurs when a seed leaves
254
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
• The transition rate from state (x, y) to state (x 1, y + 1) is min{cx, µ(⌘x + y)}, which occurs
when a leecher becomes a seed. Note that cx is the total download capacity the leechers have
and µ(⌘x + y) is the maximum upload speed to the leechers. So min{cx, µ(⌘x + y)} is the
overall download speed. Note that leechers have a partial file so they can upload to other
leechers as well. But the upload capacity of a leecher can be used only if other leechers which
need the partial file can find the leecher. Thus the efficiency of file sharing is difficult to
capture exactly; the parameter is an approximation to capture file-sharing efficiency.
• The transition rate from state (x
joins the network.
1, y) to state (x, y) is , which occurs when a new leecher
• The transition rate from state (x, y + 1) to state (x, y) is y, which occurs when a seed leaves
the network.
• The transition rate from state (x + 1, y 1) to state (x, y) is min{cx, µ(⌘(x + 1) + (y
which occurs when a leecher becomes a seed.
1))},
The transitions in and out state (x, y) are presented in Figure 8.16.
Figure 8.16: Transitions in and out of state (x, y)
Let ⇡x,y denote the steady-state distribution. Since ⇡Q = 0 if the steady-state distribution
exists, the steady-state equation is
⇡x,y+1 (y + 1) + ⇡x
1,y
+ min{c(x + 1), µ(⌘(x + 1) + y
1)}⇡x+1,y
1
= ⇡x,y min{cx, µ(⌘x + y)} + ⇡x,y y + ⇡x,y .
We first analyze this steady-state equation assuming ! 1. Defining
x
y
x̄ = and ȳ = ,
and substituting them into the steady-state equation above, we have
⇡x,y+1 ( ȳ + 1) + ⇡x
=
1,y
+ min{c( x̄ + 1), µ(⌘( x̄ + 1) + ȳ
⇡x,y min{c x̄, µ(⌘ x̄ + ȳ)} + ⇡x,y
ȳ + ⇡x,y .
1)}⇡x+1,y
1
8.13. THE DOWNLOAD TIME IN BITTORRENT
Dividing
! 1, we obtain
at both sides and letting
⇡x,y+1 ȳ + ⇡x
=
255
1,y
+ min{cx̄, µ(⌘x̄ + ȳ)}⇡x+1,y
1
⇡x,y min{cx̄, µ(⌘x̄ + ȳ)} + ⇡x,y ȳ + ⇡x,y .
Further define
⇡
˜x̄,ȳ , ⇡
(8.20)
= ⇡x,y .
x̄, ȳ
Then a Taylor’s series approximation yields
˜x̄,ȳ
1 @⇡
@ ȳ
˜x̄,ȳ
1 @⇡
⇡ ⇡
˜x̄,ȳ
@ x̄
˜x̄,ȳ
1 @⇡
⇡ ⇡
˜x̄,ȳ +
@ x̄
⇡x,y+1 = ⇡
˜x̄,ȳ(1+ 1 ) ⇡ ⇡
˜x̄,ȳ +
⇡x
⇡x+1,y
1
=⇡
˜x̄(1
1,y
=⇡
˜x̄(1
1
1
),ȳ
),ȳ(1+ 1 )
˜x̄,ȳ
1 @⇡
.
@ ȳ
Substituting into equation (8.20), we have
@⇡
˜x̄,ȳ
ȳ
@ ȳ
@⇡
˜x̄,ȳ
+ min{cx̄, µ(⌘x̄ + ȳ)}
@ x̄
✓
@⇡
˜x̄,ȳ
@ x̄
@⇡
˜x̄,ȳ
@ ȳ
◆
= 0,
which implies
( ȳ
min{cx̄, µ(⌘x̄ + ȳ)})
@⇡
˜x̄,ȳ
= (min{cx̄, µ(⌘x̄ + ȳ)}
@ ȳ
1)
@⇡
˜x̄,ȳ
.
@ x̄
(8.21)
One solution to the equation above is to assume that ⇡
˜x̄,ȳ is a delta function at some point
(x̄⇤ , ȳ ⇤ ), i.e., ⇡
˜x̄,ȳ = 0 if (x̄, ȳ) 6= (x̄⇤ , ȳ ⇤ ). This means that (8.21) has to be satisfied only at (x̄⇤ , ȳ ⇤ ).
A possible choice of (x̄⇤ , ȳ ⇤ ) that satisfies (8.21) is
ȳ ⇤ = min{cx̄⇤ , µ(⌘ x̄⇤ + ȳ ⇤ )}
1 = min{cx̄⇤ , µ(⌘ x̄⇤ + ȳ ⇤ )},
(8.22)
i.e.,
ȳ ⇤ =
min {cx̄⇤ , µ (⌘ x̄⇤
1
+
(8.23)
ȳ ⇤ )}
= 1.
(8.24)
Equations (8.23) and (8.24) determine the values of ȳ ⇤ and x̄⇤ . According to the heuristic
argument above, the delta function ⇡x,y at point( x̄⇤ , ȳ ⇤ ) satisfies the global balance equation
when ! 1, so ⇡ is the stationary distribution of the CTMC. We next compute the value of x̄⇤
by considering two cases.
• For the first case, we assume
cx̄⇤  µ(⌘x̄⇤ + ȳ ⇤ ),
i.e.,
cx⇤  µ(⌘x⇤ + y ⇤ ),
(8.25)
256
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
where x⇤ = x̄⇤ and y ⇤ = ȳ ⇤ . So in steady state, the net download bandwidth is less than
or equal to the net upload bandwidth. In this case, from equation (8.24), we obtain
1
x̄⇤ = .
(8.26)
c
• For the second case, we assume
cx̄⇤ > µ (⌘x̄⇤ + ȳ ⇤ ) ,
so the upload bandwidth becomes the bottleneck. From equation (8.24), we obtain
✓
◆
1
1 1
⇤
x̄ =
.
µ
⌘
(8.27)
Summarizing the two cases above, we have
( 1
if cx̄⇤  µ(⌘ x̄⇤ + ȳ ⇤ )
⇣c ,
⌘
⇤
x̄ =
1
1 1
if cx̄⇤ > µ (⌘x̄⇤ + ȳ ⇤ ) .
µ
⌘,
Substituting the values of x̄⇤ and ȳ ⇤ into the conditions, we obtain
8
⇣
⌘
⌘
1
< 1,
if
1

µ
+
c
c
⇣
⌘
⇣
⌘
x̄⇤ =
1 1
1
: 1
, if c 1
> 1.
µ
⌘
⌘
µ
⇣
⌘
⇣
⌘
1 1
Note that 1  µ ⌘c + 1 is equivalent to µ1
⌘ 
⇣
⌘
1 1
1
> 1c . So (8.28) is equivalent to
⌘ µ
8
⇣
c 1
< 1,
if
⌘ ⇣µ
⇣c
⌘
x̄⇤ =
1 1
c 1
: 1
,
if
µ
⌘
⌘ µ
which can be rewritten as
⇤
x̄ = max
⇢
1
,
c
✓
1
µ
1
◆
1
c;
1
1
1
⌘
and
⌘
⌘
c
⌘
(8.28)
⇣
1
µ
1
⌘
> 1 is equivalent to
 1c .
> 1c ,
.
Thus, the number of peers in steady state (as ! 1) is given by
✓
⇢ ✓
◆
◆
1 1
1 1
1
⇤
⇤
⇤
⇤
x + y = (x̄ + ȳ ) =
max
,
+
.
c µ
⌘
According to Little’s law, the amount of time a peer spends on downloading a file is
⇢
✓
◆
x⇤
x̄⇤
1 1 1
1
=
= max
,
.
c ⌘ µ
The total time a peer stays in the system is
x⇤
+
1
= max
⇢
1 1
,
c ⌘
✓
1
µ
1
◆
+
1
.
Note that both the download and waiting times in the system are independent of , the arrival
rate. Normally, when the arrival rate increases, queueing systems get congested and therefore,
the waiting times get larger. However, in a P2P network, if the number of peers in the network
increases, then the total capacity available to upload files also increases proportionally. This is the
reason why the download and waiting times in the system do not increase with in a P2P network.
8.14. PROBLEMS
8.14
257
Problems
Exercise 8.1 Prove Result 7.2.1 (the sum of two independent Poisson process is a Poisson process)
and Result 7.2.2 (K random processes generated from a Poisson process are K independent Poisson
process).
Exercise 8.2 Consider an M/M/s/k queue with arrival rate and mean service time 1/µ. Find
the steady-state distribution of this queueing system. Are there conditions on and µ for the
steady-state distribution to exist?
Exercise 8.3 In this exercise, we will show a central-limit-theorem-like result for the M/M/s/s
loss model when the number of servers and the traffic intensity are both large and nearly equal to
each other.
Let B(s, ⇢) denote the Erlang-B formula for the blocking probability in an M/M/s/s system.
p
Let s = b⇢ +
⇢.c Show that
lim
⇢!1
p
⇢B(s, ⇢) = ( )/ ( ),
where (.) and (.) are the pdf and cdf of the standard normal random variable N (0, 1). Use the
following facts to show the above result:
• Central limit theorem for Poisson random variables: Let p(k, ⇢) = e
lim
b⇢+
⇢!1
• Stirling’s formula: limn!1
• lim⇢!1 e
facts.)
p
⇢ (1 +
p
/ ⇢)
⇢
p
n!
2⇡n( n
)n
e
=e
2 /2
⇢ ⇢k /k!.
Then,
p
X ⇢c
p(k, ⇢) = ( ).
k=0
= 1.
. (Prove this fact. You don’t have to prove the two previous
Exercise 8.4 Consider an M/GI/1 queue. Recall
A(z) = M ( (z 1))
⇡(0)A(z)(z 1)
Q(z) =
.
z A(z)
Please compute the closed-form expression of Q(z) and then derive the P-K formula.
Exercise 8.5 Consider an M/GI/1 queue with two classes of customers, with the class i arrival
rate, mean service time, variance of the service time being i , 1/µi and i2 , respectively. Let
us assume that the service discipline is non-preemptive priority with higher priority to class 1
customers, i.e., when the server becomes free, the first customer in the class 1 queue begins service.
However, a class 2 customer undergoing service cannot be interrupted even if a class 1 customer
258
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
arrives (hence, the terminology “non-preemptive” priority). Show that the average waiting time in
the queue for a class i customer is given by
P2
2
2
i=1 i 1/µi + i
i
Wq =
,
Pi
2(1 ⇢i 1 )(1
j=1 ⇢j )
where ⇢0 = 0 and ⇢i = i /µi , i = 1, 2.
Hint: Note the a class 2 customer has to wait for the service completion of all class 2 and class
1 customers that arrived prior to it, the service completion of all class 1 customers that arrived
while the class 2 customer is waiting in the queue and the remaining service time of the customer
in service when the class 2 customer arrived.
Exercise 8.6 For a GI/GI/1 queue with
< µ, show that
Wq 
( a2 + s2 )
,
2(1 ⇢)
where a2 and s2 are the inter-arrival time and service time variance, respectively, and ⇢ = /µ.
This result is called the Kingman bound and is the continuous-time analog of the result in Section
3.4.4 for discrete-time queues.
Hint: Let wk be the waiting time of the k th packet. Write wk+1 as
wk+1 = wk + Sk
Ak+1 + uk ,
where uk ensures that wk+1 does not become negative. Then, show that E[uk ] = E[Sk Ak+1 ] in
steady state. Finally, use the Lyapunov function wk2 and assume that the system is in steady-state
to derive the result.
Exercise 8.7
1. Consider an M/M/2 queue with arrival rate 2 and mean service time 1/µ.
Find the expected waiting time in steady-state of a packet in this queue.
2. Now consider a system consisting of two independent M/M/1 queues with mean service time
equal to 1/µ in each queue. Assume that packets arrive to this system according to a Poisson
process of rate 2 . When a packet arrives it joins queue 1 with probability 1/2 and queue 2
with probability 1/2. Compute the expected steady-state waiting time in this system.
Note: The mean waiting time in part (1) is smaller because it has a single queue so that no
server is idle when there is work to be done.
Exercise 8.8 Consider the M/G/s/s loss model, where the service-time distribution is Erlang with
K stages, each with mean 1/(Kµ). Show that the blocking probability depends only on the mean
of the service-time distribution 1/µ and not on the number of stages.
Hint: Proceed as in the insensitivity proof for M/GI/1-PS queue.
Note: this result can be extended to mixtures of Erlang distributions and more generally, to any
service-time distribution, to prove the insensitivity of the model to the distribution of service time
beyond the mean.
Exercise 8.9 Consider the M/M/1 queueing system, with arrival rate
1/µ.
and mean service-time
8.14. PROBLEMS
259
1. Show that the steady-state distribution is the Poisson distribution with mean ⇢ = µ.
2. Now consider an M/GI/1 model with P
the following service-time
distribution: the service
PK
K
time is 1/µi with probability pi , where i=1 pi = 1 and i=1 pi /µi = 1/µ. In other words,
the service time consists of a mixture K deterministic service times. Let (n1 , n2 , . . . , nK ) be
the state of this system, where ni is the number of customers with service time 1/µi in the
system. Find the steady-state distribution of (n1 , n2 , . . . , nK ).
P
3. In part (2) above, let n = i ni . Show that ⇡n , the steady-state distribution, only depends
on µ, and not on K, pi or µi explicitly.
Hint: You may have to use the multinomial theorem which is a generalization of the binomial
theorem. See wikipedia if you do not know the multinomial theorem.
4. Is the departure process of the model in part (2) Poisson? Clearly explain your answer.
Exercise 8.10 Let X be a CTMC over a state-space S and suppose that X is time-reversible in
steady-state. Now consider a CTMC Y which is a restriction of X to the state space A, where
A 2 S. By restriction, we mean that Y takes values only in A, has the same transition rates as X
but transitions out of A are not allowed. Show that Y is also time-reversible in steady-state and
its steady-state distribution is the same as the steady-state distribution of X restricted to A but
rescaled to add up to 1. This result is called the truncation theorem.
Exercise 8.11 Consider the following variation of the M/M/s/s loss model. Consider a single
link of capacity 50 with two call classes. Each call from each class requests one unit of capacity on
arrival. Assume both call classes have holding times that are exponentially distributed with unit
mean. The call arrival processes are independent Poisson processes, with a mean arrival rate of
40 for Class 1 calls and a mean arrival rate of 20 for Class 2 calls. Class 2 calls are blocked from
entering (and lost) if the available capacity is less than or equal to 5. Class 1 calls are blocked and
lost only if there is no available capacity. Thus, this is a loss model with some priority given to
Class 1 calls. This is an example of the trunk reservation introduced in Section 8.12.
1. Compute the blocking probabilities of Class 1 and Class 2 calls in this system.
Hint: Consider the Markov chain describing the total number of calls in the system.
2. What are the expected numbers of Class 1 and Class 2 calls in the system in steady-state?
Hint: Use Little’s law.
Exercise 8.12 Consider calls accessing a link with time-varying capacity. The call holding times
are independent and exponentially distributed with mean 1/µ and each call requires 1 unit of
bandwidth from the link. The link has an available capacity of nc0 in the time interval [0, ⌧1 ], an
available capacity of nc1 in [⌧1 , ⌧2 ] and an available capacity of nc2 in [⌧2 , 1), where ⌧2 > ⌧1 > 0.
We assume that nc0 > nc1 > nc2 , and n > 0. Suppose that there are n↵ calls in progress at time
t = 0 (assume ↵ 2 (0, c0 )) is an integer) and no further calls ever arrive at the system. Let N (t)
denote the number of calls in progress at time t. Thus, the bandwidth requested by the calls at
time t is N (t). We are interested in estimating the probability that the required bandwidth ever
260
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
exceeds the available capacity, up to a logarithmic equivalence. To this end, prove the following
fact for appropriate functions Ii (x) :
1
log Pr (there exists a t 2 [0, 1) such that N (t)
n!1 n
lim
c(t)) =
min Ii (ci ),
1,2
where c(t) is the available capacity at time t. Explicitly compute the functions Ii (x).
Exercise 8.13 Consider S ON-OFF sources which switch between an active (ON) state and an
inactive (OFF) state. Let us suppose that the sources access a bottleneck node with transmission
rate C bits/sec., and that each source accesses this node via an access link whose transmission rate
is 1 bit/sec. In other words, when the number of ON sources in the system is less than or equal to C,
each source is served at rate 1. Assume C < S. When the number of ON sources exceeds C, there is
some form of resource allocation mechanism that divides the available capacity C to all the sources
equally. In other words, if there are N (t) active sources in the system at time t, each source is that
is ON is served at rate C/N (t) bits/sec. We assume that the OFF times are exponential with mean
1/ . Once a source enters the ON state, it remains in this state till it receives an exponentially
distributed amount of service whose mean is 1/µ bits.
1. Let N (t) be the number of active sources in the system at time t. Draw the state transition
diagram of the Markov chain that describes the evolution of N (t).
2. Let ⇡i denote the steady-state probability that there are i active sources in the system. Write
down the equations needed to compute {⇡i }.
3. Derive an expression for the average amount of time spent in the ON state by a source, as a
function of {⇡i }.
Exercise 8.14 CSMA in continuous-time: Consider an ad hoc network whose scheduling algorithm
operates in continuous-time. Assume that if a link l interferes with the transmission of link j, then
j also interferes with l. Associate with each link l in the network a weight wl . Let S1 , S2 , · · · SK
be the independent sets in the interference graph of the network, i.e., each Sk represents a set of
links that can be scheduled simultaneously without interference. We include the empty set in the
set of independent sets. Consider the following MAC (medium access control algorithm): each link
can be either active or inactive at each time instant. At time 0, assume that the set of all active
links is an independent set. Further, at time 0, each link starts an exponentially distributed timer
with mean 1. When a link’s timer expires (call this link l), it checks to see if any other link in its
neighborhood (where the neighborhood is the set of other links with which it interferes) is active.
If there is an active link in its neighborhood, then link l maintains its current state (which must
be inactive), starts another timer and repeats the process. If no link in its neighborhood is active,
then the link chooses be active with probability ewl /(1 + ewl ), and chooses to be inactive with
probability 1/(1 + ewl ). Then, it starts a new timer and repeats the process. Note that the state of
a link does not change between two of its consecutive timer expiration moments.
1. Let X(t) be the set of links that are active at time t. Note that the set of active links must
be an independent set in the interference graph, and also note that X(t) is a CTMC. Find
the rate transition matrix of X(t).
8.14. PROBLEMS
261
2. Show that the steady-state distribution of X(t) is given by
P
Q
w
wl
e l2Sk l
l2Sk e
⇡Sk =
=
,
Z
Z
where Z is a normalization constant and ⇡; = 1/Z.
Exercise 8.15 Consider a loss model with two types of calls. Calls of type 1 arrive according to
a Poisson process of rate 1 = 50 calls per second. Calls of type 2 arrive at rate 2 = 1. Both
call types have holding times which are exponentially distributed with mean 1. Calls of type 2 are
admitted into the system only if the total number of calls (of both types) already in the system is
less than 45. Type 1 calls are admitted if the total number of calls is less than 50. If n(t) denotes
the total number of calls in the system, then n(t) is a CTMC whose transition diagram looks as in
Figure 8.17. Find the expected number of type 1 and type 2 calls in the system in steady-state.
Figure 8.17: The Markov chain for the system
Hint:
1. E [Type 2 calls] in steady-state is not equal to 45.
2. E [Type 1 + Type 2 calls] is E[n] which can be calculated after solving the steady-state distribution of n(t).
3. Compute the blocking probability of Type 1 calls and then use it to compute E [Type 1 calls] .
Exercise 8.16 Recall the Erlang-B blocking probability formula for the M/M/c/c loss model with
arrival rate and mean holding time µ:
⇢c
pb = c c! ,
X ⇢n
n!
n=1
where ⇢ = µ . Now suppose that ⇢ = ↵c and let c ! 1. Show that pb converges to
pb =
✓
↵
1
↵
◆+
.
Note: This suggests the following approximation for the blocking probability when c and ⇢ are
large:
✓
◆
⇢ c +
pb ⇡
.
c
262
CHAPTER 8. QUEUING THEORY IN CONTINUOUS TIME
Exercise 8.17 Recall the reduced local approximation for alternate routing without trunk reservation:
h
i
b̃ = EB (1 + 2b̃(1 b̃)), c ,
where b̃ is the blocking probability on link, EB (⇢, C) is the Erlang-B formula for a link with capacity
c and traffic intensity ⇢. Also, recall that the end-to-end blocking probability is given by
b = b̃(1
(1
b̃)2 ).
Let c = 1000 and ⇢ = 950. Compute b.
Note: There will be two values for b.
Exercise 8.18 Recall the reduced-load approximation for networks with alternate routing and
trunk reservation. Compute the blocking probability with c = 1000, µ = 1, r = 50, and compare
the result with the results of the previous problem.
Exercise 8.19 In the analysis of BitTorrent, we assume the system is stable (i.e., the CTMC is
in steady state). The evolution of the number of leechers and seeds can be described using the
following fluid model:
ẋ(t) =
min{cx(t), µ(⌘x(t) + y(t))}
ẏ(t) = min{cx(t), µ(⌘x(t) + y(t))}
y(t).
Please linearize the system and show
that
⇣
⌘ the linear system reaches its equilibrium starting from
1
1 1
1
any initial condition when c 6= ⌘ µ
.
⇣
⌘
⇣
⌘
1
1
Hint: Consider two separate cases: (i) 1c < ⌘1 µ1
and (ii) 1c > ⌘1 µ1
.
8.15
Notes
Continuous-time queueing theory is the subject of many books (see, for example, [5, 159]) and its
applications to communication networks has also been treated in many texts (see, for example,
[8, 46, 103, 151]). Reversible queueing networks and their many applications have been presented
in [59]. The heavy-traffic approximation to the Erlang-B formula was derived in [157].
Connection arrivals and departures in the Internet were studied in [9,28,131,139]. The approach
presented here is based on the analysis in [139, 140] which uses the fluid model presented in [9].
The connection-level model here assumes a time-scale separation between the connection arrival and
departure processes, and the congestion control algorithm. This assumption was removed in [91].
An interesting application of the connection-level analysis to wireless networks can be found in [90].
Distributed admission control was studied in [66]. Loss networks have been studied extensively as
models of telephone networks; [61, 103] present surveys which also contains an exhaustive list of
references. The Markov chain model of BitTorrent was developed in [161] and the fluid limit in
the large arrival rate regime was presented in [127]. CTMC models of bookahead systems were
considered in [43, 141].