Lecture 7
 Poisson Processes (a reminder)
 Some simple facts about Poisson processes
 The Birth/Death Processes in General
 Differential-Difference Equations
 The Pure Birth Process (Generating our own Poisson
process from scratch)
 How are queues related to Markov chains
 The M/M/1 queue
 A Poisson arrival and Poisson departure queue
modelled using Markov chains
The Poisson Distribution (reminder)
 A random variable X is said to be Poisson if it
has the following distribution:
k e  k
beginner exercise –
P( X  k ) 
prove it IS a distribution
k!
 We can calculate the mean as follows:

k e
k 0
k!
E[ X ]   k

 
k 1
k 1e
(k  1)!

k e
k 0
k!
 

A Poisson Process
 A Poisson process is a process where the number of
arrivals in a time interval (size ) has a Poisson
distribution:
P{ A(t   )  A(t )  k}  e
 
( ) k
k!
 Where A(t) is the number of events (arrivals) up to time
t.
 Note that this is a Poisson distribution with mean t
 The parameter  is known as the rate of the process –
because in t time units, t arrivals will occur.
Poisson Interarrival Times
 tn is the time of packet n and n = tn+1 – tn
 How is n distributed? The probability n > s is the probability
that there are 0 arrivals in the period tn to tn+s
P{ n  s}  P{ A(tn  s)  A(t n )  0}  e
P{ n  s}  1  e
s
(s) 0
s
e
0!
 s
 Note that a similar derivation proves the “memoryless” property
of the Poisson process. The distribution of the time to next
arrival starting from any time t where tn < t < tn+1 would be just
the same as if we start counting from the previous arrival.
Approximating a Poisson Process
 For every t  0 and   0:
P{ A(t   )  A(t )  0}  1    o( )
P{ A(t   )  A(t )  1}    o( )
P{ A(t   )  A(t )  2}  o( )
P{ A(t   )  A(t )  0}  e
 
P{ A(t   )  A(t )  1}  e

( )
 e   1    o( )
0! 1
( )

 e    o( )
1!
0
The third property follows from the first two.
The Birth Death Process
 A birth-death process is a Markov process in which
transitions from state k can only be made from the
adjacent states k-1 and k+1
 Think of a transition from k to k+1 as a birth and in the
reverse direction as a death.
 More importantly, we could consider it as arrivals and
departures from a queue where the arrivals and
departures are Poisson processes.
 Firstly, we must consider why a Markov chain is
appropriate to the modelling.
Continuous Time Markov Chains
 Note that the Markov chains we have talked about
before were “discrete time” – there were discrete steps
which occurred at given times.
 Here we need to think about continuous time Markov
chains – those where transitions between states could
occur at any time.
 The technicalities of continuous time Markov chains are
beyond the scope of this course.
 Therefore, we will ignore this technicality and pretend
we are dealing with discrete time Markov chains with
very small times between states.
The General Birth-Death Process
 When the pop. = k, births and deaths happen as Poisson
processes: birth rate k and death rate k (0 =0)
 B(t,t ) is the number of births in the period (t,t+t )
 D(t,t ) is the number of deaths in the period (t,t+t )
P{B(t,t )= 0 | Pop. = k} = 1 - k t +o (t )
P{B(t,t )= 1 | Pop. = k} = k t +o (t )
P{B(t,t )> 1 | Pop. = k} = o (t )
P{D(t,t )= 0 | Pop. = k} = 1 - k t +o (t )
P{D(t,t )= 1 | Pop. = k} = k t +o (t )
P{D(t,t )> 1 | Pop. = k} = o (t )
Differential Difference Equations
 Define the probability that the pop. is k at time t
as Pk(t). Now, for k > 0 we have:
Pk (t  t )  Pk (t )  (k   k )tPk (t )  k 1tPk 1 (t ) 
 k 1tPk 1 (t )  o(t )
Pk (t  t )  Pk (t )
o(t )
 (k   k ) Pk (t )  k 1 Pk 1 (t )   k 1 Pk 1 (t ) 
t
t
Now, taking the limit as t0
dPk (t )
 (k   k ) Pk (t )  k 1 Pk 1 (t )   k 1 Pk 1 (t )
dt
These are known as
dP0 (t )
 0 P0 (t )  1 P1 (t )
differential difference equations
dt
A Quick Aside – The Pure Birth
Process
 Consider process k= and k=0
dP0 (t )
 P0 (t )
dt
P0 (t )  e
 t
P1 (t )  tet
(t ) 2 e  t
P2 (t ) 
2
dPk (t )
 Pk (t )  Pk 1 (t )
dt
dP1 (t )
 P1 (t )  e t
dt
dP2 (t )
 P2 (t )  tet
dt
(t ) k e  t
Pk (t ) 
k!
Which is the original Poisson process we started with (no surprise)!
The General Birth-Death Process as
a Markov Chain
0
0
0
1  0
 
1
0
 1 1  1  1
P 0
2
1   2  2
 2

0
3
1   3  3
 0
 



0
1
k-1
2
0
1
1
2
...
2
2
k
k






k
...
k+1
Note that we number the states from 0 so that the state
number is the same as the population.
Equilibrium Probabilities
 We are often interested in questions of the form:
“What is the average size of the population?” or
“What is the probability that the population is of
size k at time t?”
 We are therefore interested in the equilibrium
probabilities.
 Recall our balance equations:

 i   p ji j
j 0


i 0
i
1
Equilibrium Probabilities(2)
 In the case of our Birth-Death process these are:
 k  (1  k   k ) k  k 1 k 1   k 1 k 1
k 0
 0  (1  0 ) 0  1 1
k 0
(k   k ) k  k 1 k 1   k 1 k 1
k 0
0 0  1 1
k 0
rearrange to:
compare with: dPk (t )  (   ) P (t )   P (t )   P (t )
k
k
k
k 1 k 1
k 1 k 1
dt
dP0 (t )
 0 P0 (t )  1 P1 (t )
dt
Solving the problem
(k   k ) k  k 1 k 1   k 1 k 1
k  0 (1)
0 0  1 1
Rearrange (2):
Substitute into (1) with k=1
Rearrange:
1 
k 0
(2)
0
0
1
0
(1  1 )  0  0 0   2 2
1
10
2 
0
 2 1
We suspect (correctly) the following relation:
(proving this is part of your coursework)
k
k  0
i 1
i 1
i
Completing the Birth-Death Process
i 1
k  0
i 1 i
k

from other balance equation:

i 0
i
1
i 1
 0  1   0 
k 1 i 1 i

0 
Rearranging
k
1
i 1
1  
k 1 i 1  i

k

 0  1   i
i 1
Finally, the M/M/1 process
 The M/M/1 queue is simply a birth death process
with k=  and k=. Substituting into our
previous equations we get:
 k   k 0
where =/ is known
0 
1

1   k
k 1
as the utilisation factor for
a stable system this is < 1
1
 1 
From the geometric series:  0 
1   /(1   )
k


(
1


)

Therefore:
k
The M/M/1 Process Solved
 We now want to get the expected queue length:


k 0
k 0
E[ N ]   k k  k (1   )  k
we use a familiar trick to get:
k 0
d k 0
d  1   
  k  (1   )   
E[ N ]  (1   )   k k 1  (1   ) 
d

E[N ] 

d  1 
1 

From Little’s Theorem the average delay:

T
(1   )
Average Queue Length in M/M/1
E[N]
E[N ] 
1 

Utilisation