Adaptive Playout
1
NUS.SOC.CS5248
Ooi Wei Tsang
You are Here
Encoder
Decoder
Middlebox
Receiver
Sender
Network
2
NUS.SOC.CS5248
Ooi Wei Tsang
How to recv and play?
open socket
while not done
if socket is readable
read packet from socket
remove RTP header
decode
play back
3
NUS.SOC.CS5248
Ooi Wei Tsang
What’s Wrong?
 packet ordering
 packet loss
 next packet arrive in-time?
Especially bad for audio applications
4
NUS.SOC.CS5248
Ooi Wei Tsang
Overview
Decode
RTP
Classifier
Network
5
NUS.SOC.CS5248
Ooi Wei Tsang
Implementation
 Single Thread
 using
select()
 Multi-Threads
6
NUS.SOC.CS5248
Ooi Wei Tsang
Packet Buffer
 Sorted by sequence number
 When ADU is complete, send to
decoder
RTP
Classifier
7
NUS.SOC.CS5248
Ooi Wei Tsang
Playout Buffer
 Stored decoded data in playout
order
 Post-processing/Mixing may
happens
Decode
8
NUS.SOC.CS5248
Ooi Wei Tsang
Why Buffer?
9
NUS.SOC.CS5248
Ooi Wei Tsang
Sending Packets
Packet
Time
10
NUS.SOC.CS5248
Ooi Wei Tsang
Receiving Packets
Packet
Time
11
NUS.SOC.CS5248
Ooi Wei Tsang
With Jitter
Packet
Time
12
NUS.SOC.CS5248
Ooi Wei Tsang
With Jitter
Packet
Time
13
NUS.SOC.CS5248
Ooi Wei Tsang
What causes Jitter?
 Network delay =
Transmission Delay (fixed) +
Propagation Delay (fixed) +
Queuing Delay (variable)
 Delay jitter is caused by variable
queuing delay
14
NUS.SOC.CS5248
Ooi Wei Tsang
Delay Jitter
Transit
Time
small jitter
large jitter
Time
15
NUS.SOC.CS5248
Ooi Wei Tsang
Spike
Transit
Time
Time
16
NUS.SOC.CS5248
Ooi Wei Tsang
Today’s Question
 How big is the playout buffer?
 When to playback?
17
NUS.SOC.CS5248
Ooi Wei Tsang
Types of Applications
 Non-interactive
 Buffer
can be large
 Interactive
 As
small as possible
18
NUS.SOC.CS5248
Ooi Wei Tsang
Types of Applications
 Video
 Frames
are discrete (easier problem)
 Audio
 Samples
are “continuous”
19
NUS.SOC.CS5248
Ooi Wei Tsang
Naive Answer
 How big is a buffer?
 Fixed
at a small value, to reduce
latency
 When to playback?
 Playback
as soon as possible, to
reduce latency
20
NUS.SOC.CS5248
Ooi Wei Tsang
A Brief Introduction to
Audio Conferencing
21
NUS.SOC.CS5248
Ooi Wei Tsang
Audio Conferencing
 Live, interactive application
 Latency
is important
 Each packet 20-30ms of audio
22
NUS.SOC.CS5248
Ooi Wei Tsang
Silence Suppression
 Silence Detection
 if
no sound, no need to send
 Talk spurt
 consecutive
audio packets (between
silence)
 hundreds of ms
23
NUS.SOC.CS5248
Ooi Wei Tsang
Demo
24
NUS.SOC.CS5248
Ooi Wei Tsang
Recall: RTP Header
 marker bit:
 depends on payload
 e.g. beginning of frame
25
NUS.SOC.CS5248
Ooi Wei Tsang
RTP and Talkspurt
 First packet of a talkspurt will have
marker bit set to 1
26
NUS.SOC.CS5248
Ooi Wei Tsang
RTP and Talkspurt
 Deduce talkspurt from sequence
number and timestamp
1
20
2
40
3
60
5
190
SeqNo
TimeStamp
27
NUS.SOC.CS5248
Ooi Wei Tsang
Consequences of Talkspurt
 Opportunity to adjust playout delay
 if
jitter is large, increase delay
 if jitter is small, decrease delay
28
NUS.SOC.CS5248
Ooi Wei Tsang
Fixed Playout Delay
SEND
RECV
PLAY
29
NUS.SOC.CS5248
Ooi Wei Tsang
Adaptive Playout Delay
SEND
RECV
PLAY
30
NUS.SOC.CS5248
Ooi Wei Tsang
Adaptive Playout Delay
SEND
RECV
PLAY
31
NUS.SOC.CS5248
Ooi Wei Tsang
Trade-Off
Latency vs. Packet Loss
32
NUS.SOC.CS5248
Ooi Wei Tsang
Latency vs Loss-Rate
Loss
Rate
Latency
33
NUS.SOC.CS5248
Ooi Wei Tsang
Adaptive Playout Mechanisms
for Packetized Audio
Applications in WAN
R. Ramjee, J. Kurose, D. Towsley,
H. Schulzrinne
INFOCOM 1995
34
NUS.SOC.CS5248
Ooi Wei Tsang
Variables and Notations
35
NUS.SOC.CS5248
Ooi Wei Tsang
Variables and Notations
Tnet(i)
Tbuf(i)
Tarrive(i)
Tplay(i)
Tdelay(i)
Tsend(i)
36
NUS.SOC.CS5248
Ooi Wei Tsang
1st Packet in Talkspurt
We can estimate as
37
NUS.SOC.CS5248
Ooi Wei Tsang
How to estimate Vnet(i)
38
NUS.SOC.CS5248
Ooi Wei Tsang
How to estimate Enet(i)
 Method 1: Jacobson’s Method
39
NUS.SOC.CS5248
Ooi Wei Tsang
Spike
Tnet
Time
40
NUS.SOC.CS5248
Ooi Wei Tsang
Problems
 Does not react to spike fast enough
41
NUS.SOC.CS5248
Ooi Wei Tsang
How to estimate Enet(i)
 Ramjee’s Method
SPIKE
NORMAL
42
NUS.SOC.CS5248
Ooi Wei Tsang
Ramjee’s Idea
if Tnet(i) suddenly increase
SPIKE
NORMAL
if “slope” is small enough
43
NUS.SOC.CS5248
Ooi Wei Tsang
In Spike Mode
SPIKE
Tnet
i
44
NUS.SOC.CS5248
Ooi Wei Tsang
In Normal Mode
NORMAL
Tnet
i
45
NUS.SOC.CS5248
Ooi Wei Tsang
Evaluations
Loss
Rate
Delay
NUS.SOC.CS5248
Ooi Wei Tsang
46
Problems with Ramjee’s Method
Transit
Time
Time
47
NUS.SOC.CS5248
Ooi Wei Tsang
Packet Audio Playout Delay
Adjustment: Performance
Bounds and Algorithms
S. Moon, J Kurose, D. Towsley
Multimedia Systems 1998
48
NUS.SOC.CS5248
Ooi Wei Tsang
Recall Previous Methods
49
NUS.SOC.CS5248
Ooi Wei Tsang
How to Set Tdelay(i)
 Moon’s Method
 Collect statistics on packets that
have arrived.
 Find t such that q% of last w packets
have Tnet(i) < t.
50
NUS.SOC.CS5248
Ooi Wei Tsang
Example (w =50, q = 90%)
num of
packets
1
2
3
4
5
6
7
8
9
10
11 12
delay
51
NUS.SOC.CS5248
Ooi Wei Tsang
Setting Tdelay(i)
NORMAL
SPIKE
52
NUS.SOC.CS5248
Ooi Wei Tsang
Setting Tdelay(i)
Transit
Time
Time
53
NUS.SOC.CS5248
Ooi Wei Tsang
Performance Bound
 Given a trace of packets, and a loss
rate, find the minimum average
playout delay.
 Use Dynamic Programming
54
NUS.SOC.CS5248
Ooi Wei Tsang
A Packet Trace
k
M talkspurts
1,k
Tnet
2,k
13 15
3,k
j,k
nk,k
10 …
55
NUS.SOC.CS5248
Ooi Wei Tsang
More Notations
 M:

Number of Talkspurt
 Npacket(k) or nk
 Number
of packets in talkspurt k
 Ntotal
 Total
number of packets
56
NUS.SOC.CS5248
Ooi Wei Tsang
Definition
minimum average playout delay for
choosing i packets to be played out
from k-th talkspurt
k
M talkspurts
57
NUS.SOC.CS5248
Ooi Wei Tsang
How to find
58
NUS.SOC.CS5248
Ooi Wei Tsang
Definition
minimum average playout delay for
choosing i packets to be played out
from k-th to M-th talkspurt
k
..
M
M talkspurts
59
NUS.SOC.CS5248
Ooi Wei Tsang
Base Case
D(k, 0) =
D(M, i) =
minimum average playout delay for
choosing i packets to be played out
from k-th to M-th talkspurt
60
NUS.SOC.CS5248
Ooi Wei Tsang
Recursive Case
..
1,k
2,k
..
k
..
3,k
M
nk,k
j
61
NUS.SOC.CS5248
Ooi Wei Tsang
Performance Bound
 Given a trace of M talkspurts and n
packets, and a loss rate e, find the
minimum average playout delay.
 Answer: Minimum possible average
playout delay is D(1, (1-e)n)
minimum average playout delay for
choosing i packets to be played out
from k-th to M-th talkspurt
62
NUS.SOC.CS5248
Ooi Wei Tsang
Evaluations
Loss
Rate
Delay
NUS.SOC.CS5248
Ooi Wei Tsang
63
Summary
 Playout Adjustment for Audio
Conferencing
 Weighted Average Methods vs.
Statistical Methods
 An Analysis of Minimum Playout
Delay
64
NUS.SOC.CS5248
Ooi Wei Tsang
Practical Complications
 Clock Drifts
 Route Change
65
NUS.SOC.CS5248
Ooi Wei Tsang
Advanced Techniques
 Speed-up Playback
66
NUS.SOC.CS5248
Ooi Wei Tsang