Queueing Theory
Frank Y. S. Lin
Information Management Dept.
National Taiwan University
[email protected]
1
References


Leonard Kleinrock, “Queueing Systems
Volume I: Theory”, New York: Wiley, 1975-1976.
D. Gross and C. M. Harris, “Fundamentals of
Queueing Theory”, New York: Wiley, 1998.
2
Agenda








Introduction
Stochastic Process
General Concepts
M/M/1 Model
M/M/1/K Model
Discouraged Arrivals
M/M/∞ and M/M/m Models
M/M/m/m Model
3
Introduction
4
Queueing System

A queueing system can be described as customers
arriving for service, waiting for service if it is not
immediate, and if having waited for service, leaving
the system after being served.
5
Why Queueing Theory

Performance Measurement





Average waiting time of customer / distribution of
waiting time.
Average number of customers in the system /
distribution of queue length / current work backlog.
Measurement of the idle time of server / length of an
idle period.
Measurement of the busy time of server / length of a
busy period.
System utilization.
6
Why Queueing Theory (cont’d)

Delay Analysis
Network Delay =
Queueing Delay
+ Propagation Delay (depends on the distance)
+ Node Delay Processing Delay
(independent of packet length,
e.g. header CRC check)
Adapter Delay (constant)
7
Characteristics of Queueing Process

Arrival Pattern of Customers




Probability distribution
Patient / impatient (balked) arrival
Stationary / nonstationary
Service Patterns



Probability distribution
State dependent / independent service
Stationary / nonstationary
8
Characteristics of Queueing Process
(cont’d)

Queueing Disciplines






First come, first served (FCFS)
Last come, first served (LCFS)
Random selection for service (RSS)
Priority queue
Preemptive / nonpreemptive
System Capacity

Finite / infinite waiting room.
9
Characteristics of Queueing Process
(cont’d)

Number of Service Channels



Single channel / multiple channels
Single queue / multiple queues
Stages of Service



Single stage (e.g. hair-styling salon)
Multiple stages (e.g. manufacturing process)
Process recycling or feedback
10
Notation

A queueing process is described by A/B/X/Y/Z
11
Notation (cont’d)


For example, M/D/2/∞/FCFS indicates a
queueing process with exponential inter-arrival
time, deterministic service times, two parallel
servers, infinite capacity, and first-come, firstserved queueing discipline.
Y and Z can be omitted if Y = ∞ and Z = FCFS.
12
Stochastic Process
13
Stochastic Process

Stochastic process: any collection of random
variables Χ(t), t T, on a common probability
space where t is a subset of time.




Continuous / discrete time stochastic process
Example: Χ(t) denotes the temperature in the class on t
= 7:00, 8:00, 9:00, 10:00, … (discrete time)
We can regard a stochastic process as a family of
random variables which are “indexed” by time.
For a random process X(t), the PDF is denoted by
FX(x;t) = P[X(t) <= x]
14
Some Classifications of Stochastic
Process

Stationary Processes: independent of time
FX (x; t + τ) = FX (x; t)

Independent Processes: independent variables
FX (x; t) = FX1,…, Xn(x1,…, xn ; t1,…,tn)
= FX1(x1; t1) …FXn(xn; tn)

Markov Processes: the probability of the next state
depends only upon the current state and not upon any
previous states.
P[X(tn+1) = xn+1 | X(tn) = xn, …., X(t1) = x1]
= P[X(tn+1) = xn+1 | X(tn) = xn]
15
Some Classifications of Stochastic
Process (cont’d)

Birth-death Processes: state transitions take place
between neighboring states only.

Random Walks: the next position the process occupies
is equal to the previous position plus a random variable
whose value is drawn independently from an arbitrary
distribution.
16
General Concepts
17
Continuous-time Memoryless
Property
If X ~Exp(λ), for any a,b > 0,
P[X > a + b | X > a] = P[X > b]
Proof:
P[X > a + b | X > a]
P[ X  a  b 
X  a ]


 X  a  b   X  a 
P  X  a
P  X  a  b  1  Fx  a  b  e   ( a b )


   a  e   b  P ( X  b)
P  X  a
1  Fx  a 
e
18
Global Balance Equation

Define Pi = P[system is in state i]
Pij = P[get into state j right after leaving state i]

Pj   Pij 
i 0
(i  j )

P
i 0
(i  j )
i
 Pij
rate out of state j = rate into state j
19
General Balance Equation

Define S = a subset of the state space




 P P  P  P
j 0
( js )
j
i 0
( is )
ij
i 0
( is )
i
j 0
( js )
ij
S
j
rate in = rate out
20
General Equilibrium Solution

Notation:

Pk = the probability that the system contains k
customers (in state k)

P
k 0


k
1
λk= the arrival rate of customers when the system is in
state k.
μk= the service rate when the system is in state k.
21
General Equilibrium Solution (cont’d)

Consider state  k:
rate in  rate out
Pk  k  Pk 1  k 1

 Pk 1  k Pk
 k 1
k 1
 Pk 
Pk 1
k
.
.
.
k 1  k 2  0
 Pk 
P0 
k  k 1  1
λk
λk-1
k-1
k
μk
i
 P0

i 0 i 1
k+1
μk+1
k 1
#
22
General Equilibrium Solution (cont’d)

P
k 0
k
1
i
 
 P0  1
k 0 i 0 i 1

k 1
 P0 
1

   Pk k , T 
i
1  
k  0 i  0 i 1

k 1
waiting time w  T 
k 0
N

1

23
Little’s Result

N = average number of customers in the system
T = system time (service time + queueing time)
 λ= arrival rate
 N  T

Black box

N
Service time
…
Queueing time
System time T
24
M/M/1 Model
Single Server, Single Queue
(The Classical Queueing System)
25
M/M/1 Queue

Single server, single queue, infinite population:
k  
k  

Interarrival time distribution:
p (t )   e  t
Service time distribution

t0
p (t  t0 )    e t dt  1  e t0
0

Stability condition λ < μ
26
M/M/1 Queue (cont’d)

System utilization

  = P[system is busy], 1-  P[system is idel]


Define state Sn = n customers in the system
(n-1 in the queue and 1 in service)
S0 = empty system
rate out
S
rate in
27
M/M/1 Queue (cont’d)

Define pn = P[n customers in the system]
  pn    pn1 (rate in = rate out)

pn 1   pn    pn

n 1
p


 p0
 n 1

Since
 pi  1 
i 0


n
p


1
p

1

 0
0
n
i 0
i 0
n
p

1


,
p


 (1   )
n
 0
#
28
M/M/1 Queue (cont’d)

Average number of customers in the system
N   k  (1   )  k  (1   ) k   k
 (1   )     d  k / d 
d
k
 (1   )   


d
d  1 
 (1   )   


d

1






1 
N 

1 
#
29
M/M/1 Queue (cont’d)

Average system time
T


N


1 

(Little’s Result)

1/ 
1

1    
#
P[≧ k customers in the system]
k

  (1   )  i  (1   )
 k
1 
i k

30
M/M/1/K Model
Single Server, Finite Storage
31
M/M/1/K Model

The system can hold at most a total of K
customers (including the customer in service)
λk = λ
0
if k < K
if k  K
μk = μ
32
M/M/1/K Model (cont’d)


Pk  P0   P0  
i 0 

Pk  0
k 1
 P0 
k
kK
kK
1
1  / 

k
1   ( /  )   1  ( /  ) K 1
 k 1

0k  K
0
otherwise
K
33
Discouraged Arrivals
34
Discouraged Arrivals

Arrivals tend to get discouraged when more and
more people are present in the system.

k 
k 1
k  
35
Discouraged Arrivals (cont’d)
 /(i  1)
k 1
Pk  P0  
  /     P0

k!
i 0
k 1
P0 
1

1    /  
k 1
 Pk
 / 


k!
k
e



k
1

k!
e




N 

36
Discouraged Arrivals (cont’d)
( /  )  ( /  )
   k Pk  

e
k!
k 0
k 0 k  1



  1  e( /  ) 
T
N


(    ,   1  P0 )
/
 (1  e /  )
37
M/M/∞ and M/M/m
M/M/∞ - Infinite Servers, Single Queue
(Responsive Servers)
M/M/m - Multiple Servers, Single Queue
(The m-Server Case)
38
M/M/∞ Queue

There is always a new server available for each
arriving customer.
k  
k  k 
39
M/M/∞ Queue (cont’d)
( /  )k  / 
Pk  P0 

e
k!
i 0 (i  1) 
k 1


N

T 
1

(Little’s Result)
40
M/M/m Queue

The M/M/m queue



An M/M/m queue is shorthand for a single queue served
by multiple servers.
Suppose there are m servers waiting for a single line.
For each server, the waiting time for a queue is a
system with service rateμ and arrival rate λ/m.
The M/M/1 analysis has been done, at risk conclusion:
1
delay =    / n
throughput  
/n 


n
41
M/M/m Queue (cont’d)
λk = λ
μk = kμ if k  m
mμ if k > m
For k  m
For k > m

pk  p0


pk  p0



 k 1

 p0 ( )
2 k 
 k!



 k 1 1 k n


 p0 ( )
( )
2  n n
 n! n
42
M/M/n Queue (cont’d)

p
i 0
∴
i
1
p0 
1
n 1
(np)k (np)n 1
pi


k!
n ! (1   )
k 0
where  

n

P[queueing] =
p
k m
k
 (/  )n 

 p0
Total system time =
2
 (n  1)!(n   )
1
43
Comparisons (cont’d)

M/M/1 v.s M/M/4
If we have 4 M/M/1 systems: 4 parallel communication
links that can each handle 50 pps (μ), arrival rate λ = 25
pps per queue.
average delay = 40 ms.
Whereas for an M/M/4 system,
average delay = 21.7 ms.
44
Comparisons (cont’d)

Fast Server v.s A Set of Slow Servers #1
If we have an M/M/4 system with service rate μ=50 pps for
each server, and another M/M/1 system with service rate
4μ = 200 pps. Both arrival rate is λ = 100 pps
delay for M/M/4 = 21.7 ms
delay for M/M/1 = 10 ms
45
Comparisons (cont’d)

S1
Fast Server v.s A Set of Slow Servers #2
If we have n M/M/1 system with service rate μ pps for each
server, and another M/M/1 system with service rate nμ pps.
Both arrival rate is nλ pps
1/ 
T1 
S2
1 
μ
λ
μ
λ
…
λ
μ
1/ n 1/ n
T2 

n 1  
1
n
T1
 T2 
n
nλ
nμ
46
M/M/m/m
Multiple Servers, No Storage
(m-Server Loss Systems)
47
M/M/m/m

There are available m servers, each newly arriving
customers is given a server, if a customers arrives
when all servers are occupied, that customer is lost
e.g. telephony system.
 if k  m
k 
0 if k  m
k  k 
48
M/M/m/m (cont’d)
1
P0  ( /  )
k!
if k  m
0
if k  m
k
Pk 

k 1 
 P0   ( /  )

k
!
 k 0


1
49
M/M/m/m (cont’d)

Let pm describes the fraction of time that all m
servers are busy. The name given to this
probability expression is Erlang’s loss formula and
is given by
pm 
( /  ) m / m !
m
k
(

/

)
/ k!

k 0


This equation is also referred to as Erlang’s B
formula and is commonly denoted by B(m,λ/μ)
http://www.erlang.com
50