Playout
Buffer
Management
NUS.SOC.CS5248
OOI WEI TSANG
1
You are Here
Encoder
Decoder
Middlebox
Receiver
Sender
Network
NUS.SOC.CS5248
OOI WEI TSANG
2
How to recv and play?
open socket
while not done
if socket is readable
read packet from socket
remove RTP header
decode
play back
NUS.SOC.CS5248
OOI WEI TSANG
3
What’s Wrong?
 packet ordering
 packet loss
 next packet arrive in-time?
Especially bad for audio applications
NUS.SOC.CS5248
OOI WEI TSANG
4
Detour: A Brief
Intro to Audio
Conferencing
NUS.SOC.CS5248
OOI WEI TSANG
5
Audio Compression
 Normally uncompressed
 Telephone quality:
 8-bit
audio
 8Khz
 20-30ms per packet
NUS.SOC.CS5248
OOI WEI TSANG
6
Common Technique
 Silence Detection
 No
need to send if there is no
sound at the input
 Talkspurt
 Sequence
of consecutive audio
packets (in between silence)
NUS.SOC.CS5248
OOI WEI TSANG
7
Detecting Talkspurt in
RTP
 Marker bit
 Not
reliable as packet with marker
bit could be lost
 Deduce from timestamp and
sequence number
NUS.SOC.CS5248
OOI WEI TSANG
8
Delay Jitter
NUS.SOC.CS5248
OOI WEI TSANG
9
What causes Jitter?
 Network delay =
Propagation Delay (fixed) +
Queueing Delay (variable)
 Delay jitter is caused by variable
queueing delay
NUS.SOC.CS5248
OOI WEI TSANG
10
Delay Jitter
Transit
Time
small jitter
large jitter
Time
NUS.SOC.CS5248
OOI WEI TSANG
11
Spike
Transit
Time
Time
NUS.SOC.CS5248
OOI WEI TSANG
12
Buffer: The
Jitter
Absorber
NUS.SOC.CS5248
OOI WEI TSANG
13
If Zero Jitter
Time
NUS.SOC.CS5248
OOI WEI TSANG
14
With Jitter
LOSS
NUS.SOC.CS5248
OOI WEI TSANG
Time
15
With Jitter + Buffer
Buffer Size
Playout Delay
NUS.SOC.CS5248
OOI WEI TSANG
Time
16
Q: How to set
playout
delay?
NUS.SOC.CS5248
OOI WEI TSANG
17
Adapting Playout Delay
 When jitter is low, reduce delay
 When jitter is high, increase delay
NUS.SOC.CS5248
OOI WEI TSANG
18
Fundamental Trade-off
Latency vs Packet Loss
NUS.SOC.CS5248
OOI WEI TSANG
19
Playout Delay
 Once you set the playout delay,
cannot change!
 NOT true: can change at beginning
of talkspurt
NUS.SOC.CS5248
OOI WEI TSANG
20
Adapting Playout Delay
SEND
RECV
PLAY
NUS.SOC.CS5248
OOI WEI TSANG
21
Adaptation
Algorithm
NUS.SOC.CS5248
OOI WEI TSANG
22
Variables and Notations
Tnet(i)
Tbuffer(i)
Tarrive(i)
Tplay(i)
Tdelay(i)
Tsend(i)
NUS.SOC.CS5248
OOI WEI TSANG
23
First Packet in the
Talkspurt
Tdelay(i) = Enet(i) + 4 Vnet(i)
Tplay(i) = Tsend(i) + Tdelay(i)
Tnet(i)
Tbuffer(i)
Tarrive(i)
Tplay(i)
Tdelay(i)
Tsend(i)
NUS.SOC.CS5248
OOI WEI TSANG
24
Subsequent Packets
Tplay(j) = Tplay(i) + Tsend(j) – Tsend(i)
Tnet(i)
Tbuffer(i)
Tarrive(i)
Tplay(i)
Tdelay(i)
Tsend(i)
NUS.SOC.CS5248
OOI WEI TSANG
25
How to
estimate
Enet(i)
NUS.SOC.CS5248
OOI WEI TSANG
26
Algorithm 1
(Jacobson’s)
Enet(i) = aEnet(i-1) + (1-a)Tnet(i)
NUS.SOC.CS5248
OOI WEI TSANG
27
Algorithm 2
if Tnet(i) > Enet(i)
Enet(i) = bEnet(i-1) + (1-b)Tnet(i)
else
Enet(i) = aEnet(i-1) + (1-a)Tnet(i)
NUS.SOC.CS5248
OOI WEI TSANG
28
Algorithm 3
Enet(i) = min { Tnet(j) }
(over all packets in previous
talkspurt)
NUS.SOC.CS5248
OOI WEI TSANG
29
Algorithm 4 (Ramjee’s)
 Ramjee’s Proposal
 Observation: Algorithm 1-3 take
too long to react to spike.
NUS.SOC.CS5248
OOI WEI TSANG
30
Spike
NUS.SOC.CS5248
OOI WEI TSANG
31
Ramjee’s Idea
Works in 2 modes
SPIKE
NUS.SOC.CS5248
OOI WEI TSANG
NORMAL
32
Three Questions
 When to switch from normal to
spike mode?
 When to switch from spike back to
normal?
 How to estimate during spike
mode?
NUS.SOC.CS5248
OOI WEI TSANG
33
Normal to Spike
 if difference in delays is large
NUS.SOC.CS5248
OOI WEI TSANG
34
Normal to Spike
 if difference in delays is large
 difference in delays:
Tnet(i) – Tnet(i-1)
 large:
800 + 2Vnet(i)
NUS.SOC.CS5248
OOI WEI TSANG
35
Another View of Spike
(ZOOM)
Tnet(i)
Tarrive(i)
NUS.SOC.CS5248
OOI WEI TSANG
36
Spike to Normal
 slope = slope/2 +
|2Tnet(i) – Tnet(i-1) – Tnet(i-2)|/8
 if slope < 64
switch to normal
NUS.SOC.CS5248
OOI WEI TSANG
37
Estimation in Spike
Mode
Enet(i) = Enet(i-1) + Tnet(i) – Tnet(i-1)
NUS.SOC.CS5248
OOI WEI TSANG
38
First Packet in the
Talkspurt
Tdelay(i) = Enet(i) + 4 Vnet(i)
Tplay(i) = Tsend(i) + Tdelay(i)
NUS.SOC.CS5248
OOI WEI TSANG
39
First Packet in the
Talkspurt
Tdelay(i) = Enet(i) + 4 Vnet(i)
Tplay(i) = Tsend(i) + Tdelay(i)
Tprop(i)Tq(i)
Tnet(i)
Tbuffer(i)
Tarrive(i)
Tplay(i)
Tdelay(i)
Tsend(i)
NUS.SOC.CS5248
OOI WEI TSANG
40
How to
estimate
Vnet(i)
NUS.SOC.CS5248
OOI WEI TSANG
41
Variation of Delay
 Vnet(i) =
aVnet(i-1) + (1-a)|Enet(i) – Tnet(i)|
NUS.SOC.CS5248
OOI WEI TSANG
42
Comparisons
of 4
Algorithms
NUS.SOC.CS5248
OOI WEI TSANG
43
Playout Delay vs Loss
Rate
Loss Rate
Tdelay(i)
NUS.SOC.CS5248
OOI WEI TSANG
44
Algorithm 5
(Moon’s)
NUS.SOC.CS5248
OOI WEI TSANG
45
Problems of Existing
Algorithms
 Jacobson’s react too slowly
 Ramjee’s follow the delay too
closely
NUS.SOC.CS5248
OOI WEI TSANG
46
Moon’s Idea
 Collect statistics on packets that
have arrived
 Find t such that q% of last w
packets has Tnet(i) < t
 Tdelay(i) = t
NUS.SOC.CS5248
OOI WEI TSANG
47
Example (w =50, q =
90%)
num of
packets
1
2
3
4
5
6
7
8
9
10
11 12
delay
NUS.SOC.CS5248
OOI WEI TSANG
48
Spike Mode
 Tdelay(i) = Enet(1)
SPIKE
NUS.SOC.CS5248
OOI WEI TSANG
NORMAL
49
Spike Mode
NUS.SOC.CS5248
OOI WEI TSANG
50
Switching Mode
 Normal to Spike
if (Tnet(i) > k*Tdelay(i))
 Spike to Normal
if (Tnet(i) < k’ * OLDTdelay(i))
NUS.SOC.CS5248
OOI WEI TSANG
51
Loss Rate vs. Delay
Tdelay(i)
Loss Rate
NUS.SOC.CS5248
OOI WEI TSANG
52
Theoretical
Lower Bound
NUS.SOC.CS5248
OOI WEI TSANG
53
Dynamic
Programming
NUS.SOC.CS1102
OOI WEI TSANG
54
Fibonacci Numbers
i0
0

Fi  1
i 1
 F  F otherwise
i 2
 i 1
0
1
0
1
NUS.SOC.CS1102
OOI WEI TSANG
+
…
2
…
i-2
i-1
i
55
Fibonacci Numbers
fib(n)
x[0] = 0
x[1] = 1
for (i = 2; i < n; i++)
x[i] = x[i-1] + x[i-2]
return x[i]
NUS.SOC.CS1102
OOI WEI TSANG
56
Binomial Coefficient
k
 n  1  n  1
 n  
  

    k  1  k 
 k  1
if k  0 or n

1
1
1
n
1
1
1
1
1
1
NUS.SOC.CS1102
OOI WEI TSANG
57
Change-Making Problem

[Weiss] 7.6
For a currency with coin C1, C2, .. Cn
(cents), what is the min number of coins
needed to make K cents of change?
NUS.SOC.CS1102
OOI WEI TSANG
58
Example
C = {1, 5, 10, 20, 50}
 K = 76 cents
 Give 4 coins 50 + 20 + 5 + 1 = 76

NUS.SOC.CS1102
OOI WEI TSANG
59
Formulation

To make a change of K cents, either






make
make
make
make
make
a
a
a
a
a
change
change
change
change
change
of
of
of
of
of
(K-50) cents, or
(K-20) cents, or
(K-10) cents, or
(K-5) cents, or
(K-1) cents
Number of coins for K =
1 + minimum of all the above choices
NUS.SOC.CS1102
OOI WEI TSANG
60
Dynamic Programming
coinUsed ( K )  1  min {coinUsed ( K  Ci )}
i
K-20
K-10
…
K-5
K
…
min
NUS.SOC.CS1102
OOI WEI TSANG
+1
61
Theoretical
Lower Bound
NUS.SOC.CS5248
OOI WEI TSANG
62
Goal
 Input:
A
packet trace
 A number of packet losses
 What is the minimum average
playout delay Tdelay(.) ?
NUS.SOC.CS5248
OOI WEI TSANG
63
Trace
k
M talkspurts
1,k
NUS.SOC.CS5248
OOI WEI TSANG
2,k
3,k
j,k
nk,k
64
New Notations
 M:

Number of Talkspurt
 Npacket(k) or nk
 Number
of packets in talkspurt k
 Ntotal
 Total
NUS.SOC.CS5248
OOI WEI TSANG
number of packets
65
Define..
d(k, i)
minimum average playout delay for choosing i
packets to be played out from k-th talkspurt
..
1,k
NUS.SOC.CS5248
OOI WEI TSANG
2,k
..
k
3,k
j,k
M
nk,k
66
Define..
D(k, i)
minimum average playout delay for choosing i
packets to be played out from k-th to M-th
talkspurt
..
NUS.SOC.CS5248
OOI WEI TSANG
k
..
M
67
Dynamic Programming
0
1
2
N
1
2
3
M
NUS.SOC.CS5248
OOI WEI TSANG
68
Formulation for Base
Case
if i = 0 then
D(k,i) = 0
if k = M
if i ≤ Npacket(M) then
D(k,i) = d(k,i)
else
D(k,i) = ∞
NUS.SOC.CS5248
OOI WEI TSANG
69
Formulation for Recursive
Case
..
k
..
M
 (i  j ) D(k  1, i  j )  jd (k , j ) 
D(k , i ) min 

0 j i
i


NUS.SOC.CS5248
OOI WEI TSANG
70
Loss Rate vs. Delay
Tdelay(i)
1%
NUS.SOC.CS5248
OOI WEI TSANG
Loss Rate
71
Video Playout
NUS.SOC.CS5248
OOI WEI TSANG
72
Two Methods
 I-Policy
 Display
at fixed playout delay
 Drop late frames
 E-Policy
 Display
late frame at next period
 Playout delay increase
NUS.SOC.CS5248
OOI WEI TSANG
73
I-Policy
a
b
c
d
e
f
g
h
1
2
3
4
5
6
7
8
a
NUS.SOC.CS5248
OOI WEI TSANG
c
e
9
10
f
h
74
E-Policy
a
b
c
d
e
f
g
h
1
2
3
4
5
6
7
8
9
10
11
b
c
e
f
g
h
a
NUS.SOC.CS5248
OOI WEI TSANG
d
75
I-Policy
a
b
c
d
e
f
g
h
1
2
3
4
5
6
7
8
9
10
a
b
e
f
g
h
NUS.SOC.CS5248
OOI WEI TSANG
76
E-Policy
a
b
c
d
e
f
g
h
1
2
3
4
5
6
7
8
9
10
a
b
c
d
e
f
NUS.SOC.CS5248
OOI WEI TSANG
77
Pro and Cons
 I-Policy: Fixed latency
 E-Policy: No Loss Frame or Gap
NUS.SOC.CS5248
OOI WEI TSANG
78
Idea
if queue length > L for T seconds
drop incoming frames
long queue -> small T
short queue -> large T
NUS.SOC.CS5248
OOI WEI TSANG
79
Queue Monitoring
NUS.SOC.CS5248
OOI WEI TSANG
0
0
0
0
0
0
0
waiting time
3
3
3
2
2
2
2
threshold
2
3
4
5
6
7
8
queue length
80
Queue Monitoring
NUS.SOC.CS5248
OOI WEI TSANG
0
0
0
0
0
0
0
waiting time
3
3
3
2
2
2
2
threshold
2
3
4
5
6
7
8
queue length
81
Queue Monitoring
NUS.SOC.CS5248
OOI WEI TSANG
1
0
0
0
0
0
0
waiting time
3
3
3
2
2
2
2
threshold
2
3
4
5
6
7
8
queue length
82
Queue Monitoring
NUS.SOC.CS5248
OOI WEI TSANG
2
1
0
0
0
0
0
waiting time
3
3
3
2
2
2
2
threshold
2
3
4
5
6
7
8
queue length
83
Queue Monitoring
NUS.SOC.CS5248
OOI WEI TSANG
3
2
1
1
0
0
0
waiting time
3
3
3
2
2
2
2
threshold
2
3
4
5
6
7
8
queue length
84
Summary
 Preliminary work by Stone and
Jeffay on Video Playout
 Ramjee’s and Moon’s adaptive
audio playout algorithm
NUS.SOC.CS5248
OOI WEI TSANG
85
Possible
Survey and
Project Topic
NUS.SOC.CS5248
OOI WEI TSANG
86
Adapting Playback
 Besides adapting buffering time,
we can adapt playback time
 Estimating and eliminating clock
drift
NUS.SOC.CS5248
OOI WEI TSANG
87