Coupling Discrete Time Events to Continuous Time
in RMCAT
(a.k.a. The Anatomy of a RMCAT RTT)
and Reasonable Bounds on the Time Rate
of Change in Available Capacity
November 5, 2014
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
1
Talk Outline
1. Motivation (Prior Work On Test Plan Capacity Change Design)
2. RMCAT Discrete Time RTT Formalized
•
Fluid-flow (continuous-time) model and rigorous RMCAT RTT definition.
3. Infinitely Fast Capacity Change Downward
•
•
Unavoidable delay spike caused by infinitely fast capacity change.
How to avoid the delay spike (proposal could be presented to IETF).
4. How Quickly ANY RMCAT Design Can Track Capacity Changes
•
Result is independent of algorithm type (“self-clocked” or “rate-based”).
5. Reasonable Assumptions on Time-Rate-of-Change of Capacity
•
•
•
•
Worse-case RTT defines “tracking responsiveness” (w/o predictive component).
Squelching mechanisms required (self-clocked schemes do this automatically).
TCP Dynamics as a function of their RTT.
A reasonable bound on RTCP feed back intervals.
6. Implications for Adaptation with Wireless (WiFi/LTE/etc).
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
2
Motivation (Discussion at IETF 90)
Colin Perkins (Last Call on rmcat-cc-requirements):
• However, as I noted at IETF 90, I think the draft should also
include a secondary requirement to keep delay variation (jitter)
down, where possible, since larger delay variation needs larger
receiver-side buffers to compensate, increasing overall latency.
Large
Delay
Spikes
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Colin (and others) believed these
might be artifacts of the algorithms
NADAv2
Cisco Confidential
3
A RMCAT Lab Discrete Time RTT (for Seq. No. = Z)
Note: Per-packet Feedback (no RTCP yet)
Step 3: ACK of Z sent
Step 4: Records Reception of ACK of Z
= ACK Z Receive Timestamp
Earliest
Possible
Feedback
= ACK Z Send Timestamp
dr= Reverse Propagation Delay
Direction of Feedback/ACKs
Router Before
Bottleneck
RMCAT
Source
Router After
Bottleneck
Gig 0/0.11
Bottleneck Link
Step 1: Source Sends Z
RMCAT
Receiver
192.168.11.2
Step 2: Receiver Records
Reception of Z
= Z Receive Timestamp
= Z Send Timestamp
Bottleneck Direction
tn-2 tn-1 tn
time
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
df1= Forward
Propagation Delay
Prior to
Bottleneck
d(t) = Queue
Delay at
Continuous
Time t
df2= Forward
Propagation Delay
After
Bottleneck
tk-2 tk-1 tk
Modelled
zero
receiver
processing
delay
dproc= 0
time
Cisco Confidential
4
RMCAT LAB: Measurement Framework (Seq. No. = Z)
Sequence number Z sent.
Sequence number Z received (forward delay calculated here).
df1
df1
tn-2 tn-1 tn
Sequence number Z-1 sent.
tk-2 tk-1 tk
t
time
ack,Z
t ack,Z is the earliest time that the queue
measurement is available at source
(new rate change can occur here).
The time the queue was actually sampled is tqueue,Z = (tn+df1).
The effect of the rate change on the queue occurs df1 after the source
made the rate change (a full RTT after the queue was sampled)!
Let’s define continuous time t for fluid-flow modelling of the
rate adaption component. That is, let (t-tOFFSET) represet times offset from time t.
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
5
RMCAT: Continuous Time Modeling for Control Loop
dr
RMCAT Sender:
Receiver
RMCAT
Reciever
d(t)
dsend_proc
=0
RMCAT Sender:
Generates
New Rate, r(t)
df1
(≈ d0 at equilibrium)
(no RTCP)
Queue
rsend(t)
For small
;
q(t) = max [ q( t - ) + (rat_queue(t) – C) , 0 ]
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
df2
dpcv_proc
=0
Queue increases/decreases as difference in rates
When queue not emptied, it
integrates the difference in
rates (1/s in Laplace Domain).
Also note:
• rat_queue(t) = rsend ( t – df1)
• d(t) = q(t)/C
Cisco Confidential
6
Control Loop: Laplace Domain (linearize about equilibrium)
e-s(dr)
RMCAT Receiver
e-s(0)=1
RMCAT
Reciever
RMCAT Sender
R(s) = XR(s)PQ(s)
e-s(df1)
e-s(df2)
Queue
1/[sC]
PQ(s) = Queue Prediction Part
XR(s) = Rate Mechanics Part
e-s(0)=1
(could
model
a RTCP
mean
delay of
100ms)1
Key Control Loop Observations At Equilibrium:
•
LTI System => Doesn’t matter where prediction or rate control is (sender or receiver).
• Wherever it is, it WILL take a minimum of a RTT1 to make a difference at the queue.
1
– If the RTCP reporting interval is >> (df1+df2+dr), it will dominate. If not, it will look like delay noise to individual delay samples.
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
7
So … What is the Discrete-Time RMCAT RTT Definition?
dr
RMCAT Sender:
Receiver
RMCAT
Reciever
d(t)
dsend_proc
=0
RMCAT Sender:
Generates
New Rate, r(t)
df1
(≈ d0 at equilibrium)
dpcv_proc
=0
df2
Queue
Forward Delay = df1 + { d(t)| t = t
}+ df2
SAMP
RMCAT RTT (ti) |
ti = tACK for seq Z received
= df1 + { d(t)| t = t
}+ df2 + dr
SAMP (seq no z)
RMCAT RTT ( ti ) | ti = tACK for seq z rcveived = d f1 + df2 + dr + d(t – [dx + df2 + dr] ) ,
where dx = d(t)| t = t
SAMP (seq no z)
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
8
Queue Delay Variation During Downward Capacity Change
Fastest Possible Rate Adaptation Example (imprudently quick)
Assume rate control estimates C (i+1) via the
queue sample immediately after t0 and then
sends at C(i+1) (or less) as soon as possible.
Assume r(t) = Ci before t0
Minimum response time
to affect queue.
Ci
Example in IETF RMCAT Test Plan:
Capacity: 2500 kbps -> 600kbps
Assume RTT: 0.200 seconds (100 ms one-way)
Minimum possible delay
spike is caused by
(C (i+1)- C i ) * RTT
too many bits on queue.
Minimum ADDITIONAL bits on queue before
we can do anything about it 380000 bits.
Queue must empty at 600 kbps rate.
Thus minimum delay spike is 633 ms!
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
t0
t0 + RTT
Note: If we assumed one-way delay of 50 ms, 316 ms is minimum.
C (i+1)
Cisco Confidential
9
Queue Delay Variation During Downward Capacity Change
Fastest Possible Rate Adaptation Example (imprudently quick)
Ci
r(t) = Ci
(queue at equilibrium,
delay at equilibrium is d0)
t0
Delay
at
Queue,
d(t)
dacc =
[Ci - C
(i+1)]
C (i+1)
t0 + RTT
Minimum delay spike NOT caused by candidates!
* RTT
m
Accumulated bits delay
d0
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
C (i+1)
To lower
dmin_spike,
decrease
rate of
change
to new
rate
1
dmin_spike = (dacc + d0)
Different
Candidate
Responses
Cisco Confidential
10
Talk Outline
1. Motivation (Prior Work On Test Plan Capacity Change Design)
2. RMCAT Discrete Time RTT Formalized
•
Fluid-flow (continuous-time) model and rigorous RMCAT RTT definition.
3. Infinitely Fast Capacity Change Downward
•
•
Unavoidable delay spike caused by infinitely fast capacity change.
How to avoid the delay spike (proposal could be presented to IETF).
4. How Quickly ANY RMCAT Design Can Track Capacity Changes
•
Result is independent of algorithm type (“self-clocked” or “rate-based”).
5. Reasonable Assumptions on Time-Rate-of-Change of Capacity
•
•
•
•
Worse-case RTT defines “tracking responsiveness” (w/o predictive component).
Squelching mechanisms required (self-clocked schemes do this automatically).
TCP Dynamics as a function of their RTT.
A reasonable bound on RTCP feed back intervals.
6. Implications for Adaptation with Wireless (WiFi/LTE/etc).
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
11
How Quickly Can RMCAT Track Available Capacity Changes?
Answer: It Depends on the RTT, Duh!
Ci
t0
t0 + RTT
C (i+1)
• The quickest time a RMCAT flow can possibly “measure” (and “react to”)
changes in capacity is bounded by ITS round-trip time.
• Thus the quickest time it can influence it’s contribution to the queue after
a change in capacity is thus bounded by ITS round-trip time.
• Corollary: RMCAT flows with different RTTs can react to changes on different
time scales (which correspond to their individual RTTs). Just like TCP!
• A reasonable ASSUMPTION for the fastest time-rate-of-change in available
capacity is one which could be tracked (i.e., measured and reacted to) by
a RMCAT flow with an assumed worst-case RTT and RTCP interval.
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
12
A reasonable assumption bound for RMCAT
time rate of change in available capacity
CLocal_Max
C(t)
CMax_Change
t?
CLocal_Min
t? + RTT
• Assume worst-case RMCAT RTT of ~250 ms (¼ sec).
• Highest capacity change “frequency” that is actionable is fMAX ≈ 4 Hz.
•
Capacity changes faster than this CANNOT be “seen/sensed/measured” and then
“actionable” by a RMCAT flow with the “worst-case RTT”.1
• Ditto for ACK-based, “self-clocked”, protocols like TCP, SCReAM too!
•
•
TCP can’t know to stop within this time either; as they only stop after their ACKs stop.
TCP thus “pounds the queue” causing gross overflow events on long RTT connections too!
• Corollary: “Fast Response” is a matter of the worst-case capacity
change assumption - not a fundamental property of “self-clocked” protocols.2
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
1 – If other information was known (e.g., form), predictive components could react quicker.
Cisco Confidential
2 - Rebuttal to: http://conferences.sigcomm.org/sigcomm/2014/doc/slides/150.pdf
13
TCP Dynamics as a Function of the RTT
1 TCP: Voice Delay, 50 ms BE Queue
RTT Estimate ~ 50ms,
fast reaction per unit time
-Serialization delay for TCP packet ~ 11 ms
Queue emptied only 4 times*
Voice Only
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Near 100% Link Utilization (delay 30 ~ 80 ms)
1 TCP, Delay seen by Voice Packets
Cisco Confidential
14
14
TCP Dynamics as a Function of the RTT
1 TCP: Voice Delay, 500 ms BE Queue
RTT Estimate now
includes queuing delay ...
subsequent rate increase
reaction times are slowed.
Queue
Never
Emptied
100% Link Utilization (delay 100 ~ 500 ms)
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
1 TCP, Delay seen by Voice Packets
Cisco Confidential
15
15
ACK-Gated RMCAT Proposals (a la Ericsson)
CLocal_Max
C(t)
CMax_Change
t0
CLocal_Min
t0 + RTT
• On long-RTT connections, ACK-gated protocols will also hit the queue too
hard and build up excess delay. They will “stop” quickly – but that is a
matter of degree (by comparison to how rate-based designs) – not
because of some more beneficial property of ACK-gated protocols.*
• Once a worst-case RTT assumption has been made, it imposes a theoretical
constraint on how quickly ANY RMCAT FLOW can adapt to it.
•
Thus imposing a “hidden assumption” on the time-rate-of-change of capacity ANY
RMCAT DESIGN on the worst-case RTT can adapt to.
• This, in turn, imposes a minimum RTCP spacing constraint:
• For 250 ms RTT, < π/2 spacing (@4 Hz) implies TRTCP ≤ ~ 62.5 ms.
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
16
Implications for WiFi/LTE/Wireless Adaptation
CLocal_Max
C(t)
CMax_Change
t0
CLocal_Min
t0 + RTT
Repeated/paraphrased from last slide:
On long-RTT connections, ACK-gated protocols will also hit the queue too hard and
build up excess delay. They will “stop” quickly – but that is a matter of degree
(by comparison to rate-based approaches) – not because of some more
beneficial property of “self-clocked” protocols over rate-controlled protocols.
Summary:
• We will need to develop better “squelching conditions” in future enhancements to our
present RMCAT designs (i.e., when feedback stops and/or becomes irregular).
• Complete squelch is only prudent when there is no impairment in feedback path.
• Wireless challenges now become a known second-order problem; unless we want
to limit RTT (not possible) or go to per-packet feedback (non RTCP approaches).
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
17
Summary
1. RMCAT Discrete Time RTT Formalized
•
Fluid-flow (continuous-time) model and rigorous RMCAT RTT definition.
2. How Quick Can ANY RMCAT Design Can Track Capacity Changes
•
Result is independent of algorithm type (“self-clocked” or “rate-based”).
3. Reasonable Assumptions on Time-Rate-of-Change of Capacity
•
•
•
•
Worse-case RTT defines “tracking responsiveness” (w/o predictive component).
Like TCP, dynamics of RMCAT solution will be a function of their RTT.
The feedback intervals and flow RTT will determine capacity tracking ability.
Bounding feedback intervals and worst-case RTT effectively bounds best-case
tracking.
4. Implications for Adaptation with Wireless (WiFi/LTE/etc).
•
Squelching mechanisms required (self-clocked schemes do this automatically).
© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
18