QoS Controllers for the Internet
Ibrahim Matta
Azer Bestavros
[email protected]
www.cs.bu.edu/faculty/matta
[email protected]
www.cs.bu.edu/faculty/best
Computer Science Department
Boston University
Boston, MA 02215
Internet QoS
 Quality of Service (QoS)
 Smarter bandwidth management
 Eliminate cost of over-provisioning
 Enhance commercial competitiveness
 Traffic controllers placed in strategic places
 Fast packet classification
 Intelligent transmission control of packets
 Keep rest of the Internet simple
Placement and Functionality of
QoS Controllers
• Aggregate Control
• Isolation Control
• Proxy Control
Improved User and
Network Performance
Flow
Flows controlled s
isola
as a bundle
ted
Wireless
proxy
General Architecture
TCP Control
On every ACK
if (window < threshold)
window += 1
else
window += 1 / window
// slow start phase
// double window every RTT
// congestion avoidance
// increment by 1 every RTT
On timeout
threshold = window / 2
window = 1
On duplicate acknowledgments
threshold = window = window / 2
// fast recovery
Aggregate TCP Control
• Control congestion for collections of TCP flows that share the same bottleneck
1 MB transfers, 32 concurrent TCP flows (no cross traffic)
Why Aggregate TCP Control?
Clients
•Lightly Loaded:
Bottleneck Bandwidth = 5Mbps
(delay-bandwidth product of 114
packets)
Total Transfer = 64K packets
•Heavily Congested:
Delay-bandwidth product of 1
packet
Server
1
n 25ms
32Mbps
25ms
RED
Lightly Loaded Conditions
conlink BW (n=1)
conglink BW (n=8)
Bottleneck Bandwidth (n=8)
5000000
5000000
4000000
4000000
3000000
3000000
bits
bits
Bottleneck Bandwidth (n=1)
2000000
2000000
1000000
1000000
0
0
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73
time
time
conglink BW (n=16)
conglink BW (n=32)
5000000
5000000
4000000
4000000
3000000
3000000
bits
bits
Bottleneck Bandwidth (n=16)
2000000
Bottleneck Bandwidth
(n=32)
2000000
1000000
1000000
0
0
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67
time
time
Heavily Congested Conditions
MW (n=8 Bandwidth
delay*bw= 1 pkt) (n=8)
Bottleneck
60000
60000
50000
50000
40000
40000
bits
30000
30000
20000
20000
10000
10000
0
0
1 45 89 133 177 221 265 309 353 397 441 485 529 573 617
1 57 113 169 225 281 337 393 449 505 561 617 673 729 785 841
time
time
BW (n=16
delay*bw= 1 pkt) (n=16)
Bottleneck
Bandwidth
BW (n=32
delay*bw= 1 pkt) (n=32)
Bottleneck
Bandwidth
60000
60000
50000
bits
40000
30000
40000
30000
20000
10000
20000
10000
time
4497
4216
3935
3654
3373
3092
2811
2530
1968
1687
1406
1125
844
563
1
1537
1441
1345
1249
1153
1057
961
865
673
577
481
385
289
193
97
769
time
282
0
0
1
bits
50000
2249
bits
BW (delay*bw=1
pkt, n=1)
Bottleneck
Bandwidth
(n=1)
Aggregate versus Individual TCP Control
 Under individual TCP congestion control, each flow can
alternate between sending packets at its minimum possible
congestion window of 1 and timing out
 Under heavily congested conditions, congestion window of 1
can be more than the flow’s fair share of the congested link
 Bad utilization of network resources
 Aggregate TCP control allows individual flows to have
congestion windows of values less than 1
 Under heavy load, a flow’s fair share of the available
bandwidth can actually be less than 1
 Improved stability and throughput
Implementation
 A common control (virtual socket) can keep track of
- a count of active flows, n
- an aggregate congestion window
(in addition to other parameters to manage constituent flows)
 The virtual socket would make sure that
- the available bandwidth (in terms of aggregate congestion
window packets) is divided evenly among active flows
- under high congestion (timeout) conditions, if needed, the
aggregate sending rate is reduced below n/RTT
Size-Based Isolation Control
 Routing aggregates of packet flows with divergent
characteristics on (logically) separate communication paths
 Length (lifetime and size) of TCP flows follows a heavy-tailed
distribution
- few long-lived TCP flows carry most of the bytes
 Long-lived flows have a less bursty arrival/departure process
 Isolate long flows from short ones
Isolating Burstiness of Short Flows
Load balanced assignment policy of UDPShort
flows flows on Path 1, and long
flows on Path 2
Isolating Window Dynamics of Short Flows
8 TCP flows (light load)
200 long and 400 short TCP fl
60% of bandwidth taken by lo
More Predictability
Long TCP flows with 60% of total resources
Short TCP flows with 40% of total re
More Fairness
• RED does not improve fairness among short TCP flows
Implementation
 Increase predictability by isolation and aggregate control
 Only traffic controllers at the edges do per-flow management (Diffserv-like
architecture)
 Increase predictability for long flows by isolating burstiness and window
dynamics of short flows
 Short flows also benefit by not being shut off --- or unable to get their fair
share before they end!
 Isolation improves fairness for both short and long TCP flows
 Traffic controller marks packets as belonging to long-lived flow
 Direct long-lived flow to another label-switched path using MPLS
 Gradually introduce re-routed TCP flow, possibly by purposely dropping
packets to force it into slow-start phase
QoS-Based Isolation Control
 Short flows are usually interactive, delay-sensitive
 Long flows are usually bulk, throughput-sensitive
 Allocate more bandwidth to short flows
 Response time is reduced for the majority of flows (that
are short)
 Fairness is also maintained for both short and long flows
Proxy Control
 Mask variability over shorter time scales to avoid disrupting
control loops operating over longer time scales
 Estimate length and characteristics of the control loops
 Must not compromise the end-to-end semantics
Wireless Proxy
• Hiding local recovery times at
a wireless base station from TCP
• Transient wireless losses do
not infiltrate into the estimation
of Round Trip Time used to
detect congestion (buffer overflow)
losses over the (wired) Internet
A Control-Theoretic Approach
W/d if r(n-1) d <=
B+C d
r(n) =
1/d otherwise
r : sending rate
W : maximum window
size
d : round trip delay
B : buffer space at
bottleneck
C : capacity of bottleneck
 Describe the behavior of the system by a discrete-time model
B + C d : pipe size
 State changes at discrete instants of time that correspond to the arrival
of feedback signals (acknowledgments, timeouts)
 Evaluate stability, convergence, fairness and performance
Programmable Traffic Controller
 Lucent’s NEPPI (Network Element for Programmable Packet Injection)
 Control programs can instruct dispatcher to forward certain packets to them
 Or instruct dispatcher to create a fast path for packets
Packet Processing in NEPPI
 E.g. flow isolation program may instruct dispatcher to count a certain
number of packets from a flow, then mark subsequent packets as long-lived
 Or, may request to receive TCP packets with SYN/FIN/ACK flags to detect
beginning of a TCP connection, then classify the flow based on transfer size
and network state
Conclusions
 Traffic controllers can be placed in strategic places in the
Internet to provide efficient QoS support
- in front of clients/servers, or
- at exchange/peering points between administrative domains
 Reduce cost and enhance commercial competitiveness of
Internet service providers and carriers
 Basic research in the control of complex dynamical systems,
that of the Internet
 Experimental research in the implementation of a
programmable traffic controller
- programming interface to soft services, i.e. capabilities can
be turned on/off and control parameters dynamically adjusted