Adaptive Transmission
Protocols for the Future
Internet
Hari Balakrishnan
MIT Lab for Computer Science
http://www.sds.lcs.mit.edu/~hari
Internet Service Model
Internet
Router
A best-effort network: losses & reordering can occur
• Congestion due to overload causes
losses
• Transmission protocols provide endto-end data transport
– Loss recovery (if reliability is important)
– Congestion management (to reduce
Transmission Protocols
• User Datagram Protocol (UDP)
– Simple datagram delivery
– Other protocols built on top (e.g., RTP for
video)
• Transmission Control Protocol (TCP)
– Reliable, in-order byte stream delivery
– Loss recovery & congestion control
• TCP is dominant today, and is tuned for:
– Long-running transfers
– Wired links and symmetric topologies
Problem #1: The Web!
r1
r2
r3
Internet
Server
r-n
Client
• Multiple reliable streams
• Individual objects are small
• So what?
Far too inefficient!
Far too aggressive!
Problem #2: Application
Heterogeneity
Server
u1 r1
u2
r2
u3
r3
u-m r-n
Internet
Client
• New applications (e.g., real-time streams)
– The world isn’t only about HTTP or even
TCP!
• So what?
Applications do not adapt to congestion
Problem #3: Technology
Heterogeneity
Metricom Ricochet Lucent WaveLAN
Regional-Area
+
Asymmetry
Metro-Area
Cellular Digital
IBM Infrared
Packet Data (CDPD)
• Tremendous diversity
• So what?
Awful performance
Mobility-related inefficiencies
Campus-Area
Packet Radio
In-Building
•
•
•
•
•
•
•
•
Why is Efficient Transport
Hard?
Congestion
Channel errors
Asymmetry
Latency variability
Packet reordering
Mobility
Large network “pipes”
Small network “pipes”
Solution: Adaptive
Transmissions
• A framework to adapt to various network
conditions
• Guiding principle: the end-to-end
argument
– Do only the “right” amount inside the
network
– Expose important information to
applications
• Algorithms to adapt to different
This Talk
•
•
•
•
•
•
•
•
Congestion
Channel errors
Asymmetry
Latency variability
Packet reordering
Mobility
Large network “pipes”
Small network “pipes”
TCP Overview
• Loss
10
recovery
9
7
8
6
5
4
3
1
0
1
1
1
1
2
0
lost
Timeouts based on mean round-trip time (RTT) and deviation
Fast retransmissions based on duplicate ACKs
• Congestion control
– Window-based algorithm to
determine sustainable rate
– Upon congestion, reduce window
– “ACK clocking” sends data
Congestion Management
Challenges
•
•
•
•
•
Heterogeneous traffic mix
Multiple concurrent streams
Variety of applications and transports
Control algorithms must be stable
Clean separation from other tasks like
loss recovery
“Solution” #1: Persistent
Connections
r1
r2
r3
Server
r-n
Put everyone on same
ordered byte stream
Client
While this fixes some of the problems of independent
connections, it really is a step in the wrong direction!
1. Far too much coupling between objects
2. Far too application-specific
3. Does not enable application adaptation
“Solution” #2: Web
Accelerators
• Is your Web experience too slow?
• Chances are, it’s because of pesky
TCP congestion control and those
annoying timeouts
• Web accelerators will greatly speed up
your transfers…
• By just “adjusting” TCP’s congestion
control!
“Solution” #3: Integrated TCP
Sessions
r1
r2
r3
Server
r-n
Client
• Independent TCP connections, but shared control parame
[BPS+98, Touch98]
• Shared congestion windows, round-trip estimates
• But, this approach doesn’t accomodate non-TCP traffic
What is the World Heading
Toward?
Server
u1 r1
u2
r2
u3
r3
Internet
u-m r-n
Client
• The world won’t be just HTTP
• The world won’t be just TCP
Logically different streams (objects) should be kept
separate, yet efficient congestion management must be
performed.
What We Really Need…
HTTP
TCP1
Congestion
Manager
Audio
Video1
TCP2
Video2
UDP
IP
An integrated approach to end-to-end
congestion management for the Internet
using the CM
CM: Some Salient Features
• Shared learning
– Maintains host- and domain-specific
information
• Heterogeneous application support
• Simple application interfaces to CM
• Robust and stable rate control
algorithms
• Flexible bandwidth-apportioning using
receiver hints
• Enables application adaptation to
The CM API
• A simple but powerful application-to-CM
API
• Three classes of functions
– Query
– Control
– Application callback
• Design principle: Application-Level
Framing (ALF)
– Feed information up to application
– Application decides what to send; CM tells it
How the API Works
CM does not buffer any data; request/callback/notify API
Preliminary Results
• Simulation results show significant
improvements in performance
predictability
– E.g., TCP with CM reduces timeouts and
shares bandwidth well between connections
• CM’s internal congestion algorithm is
rate-based
– Great platform for experimenting with new
control schemes
• Experiments with scheduling algorithms
Summary & Status
• The CM provides a simple API to make
applications adaptive and networkaware
– Enables all traffic to adhere to basic
congestion control principles
– Improves performance predictability
– Enables shared state learning
• ns-2 experiments in progress
• Linux implementation coming soon
(including rate-adaptive applications)
This Talk
•
•
•
•
•
•
•
•
Congestion
Channel errors
Asymmetry
Latency variability
Packet reordering
Mobility
Large network “pipes”
Small network “pipes”
TCP/Wireless Performance
Today
Technology
Rated
Bandwidth
1 Mbps
IBM
Infrared
Lucent
2 Mbps
WaveLAN
Metricom
100 Kbps
Ricochet
Hybrid wireless 10 Mbps
cable
Typical TCP
Throughput
100-800 Kbps
50 Kbps-1.5 Mbps
10-35 Kbps
0.5-3.0 Mbps
Goal: To bridge the gap between perceived and rated performance
Channel Errors
Internet
Router
Loss  Congestion
Burst losses lead to coarse-grained timeouts
23
2121
Loss ==> Congestion
Result: Low throughput
0
Performance Degradation
Sequence number (bytes)
2.0E+06
Best possible
TCP with no errors
(1.30 Mbps)
1.5E+06
TCP Reno
(280 Kbps)
1.0E+06
5.0E+05
0.0E+00
0
10
20
30
40
50
60
Time (s)
2 MB wide-area TCP transfer over 2 Mbps Lucent WaveLAN
Our Solution: Snoop
Protocol
• Shield TCP sender from wireless vagaries
– Eliminate adverse interactions between protocol
layers
– Congestion control only when congestion occurs
• The End-to-End Argument [SRC84]
– Preserve TCP/IP service model: end-to-end
semantics
– Is connection splitting fundamentally important?
• Eliminate non-TCP protocol messages
–Fixed
Is link-layer
messaging
fundamentally
to mobile:
transport-aware
linkimportant?
protocol
Mobile to fixed: link-aware transport protocol
Snoop Protocol: FH to MH
6
4 3 2
1
Snoop agent
5
Base Station
FH Sender
1
Snoop agent: active interposition agent
– Snoops on TCP segments and ACKs
– Detects losses by duplicate ACKs and
timers
– Suppresses duplicate ACKs from FH
sender
Cross-layer protocol design: snoop agent
Mobile Host
Snoop Protocol: FH to MH
1
Snoop Agent
Base Station
FH Sender
Mobile Host
5
Snoop Protocol: FH to MH
4
3
2
1
Base Station
FH Sender
Mobile Host
Snoop Protocol: FH to MH
6
4 3 2
1
5
Base Station
FH Sender
1
Mobile Host
6
Snoop Protocol: FH to MH
4 3 2
5
Sender
1
Base Station
3
2
21
Mobile Host
Snoop Protocol: FH to MH
5 4 3 2
1
6
Base Station
4
3
Sender
2
Duplicate ACK
ack 0
Mobile Host
1
Snoop Protocol: FH to MH
6 5 4 3 2
1
6
Base
5 Station
1
Sender
Retransmit from cache
at higher priority
ack 0
4 3 2
ack 0
ack 0
Mobile Host
1
Snoop Protocol: FH to MH
6 5 4 3 2
1
Base Station
5
Sender
ack 0
Suppress
Duplicate Acks
1 4 3 2
ack 4
Mobile Host
1
Snoop Protocol: FH to MH
6 5
Clean cache on new ACK
Base Station
6
Sender
ack 4
5 1 4 3 2
ack 5
Snoop Protocol: FH to MH
6
Base Station
Sender
ack 4
ack 5
6
1 5 4 3 2
ack 6
Mobile Host
Snoop Protocol: FH to MH
7
9
8
Base Station
Sender
ack 5
ack 6
6
Active soft state agent at base station
Transport-aware reliable link protocol
Preserves end-to-end semantics
1 5 4 3 2
Mobile Host
Snoop Performance
Improvement
Sequence number (bytes)
2.0E+06
Best
possible
TCP
(1.30
Mbps)
1.5E+06
Snoop (1.11 Mbps)
TCP Reno
(280 Kbps)
1.0E+06
5.0E+05
0.0E+00
0
10
20
30
40
50
60
Time (s)
2 MB wide-area TCP transfer over 2 Mbps Lucent WaveLAN
Congestion Window (bytes)
Benefits of TCP-Awareness
Snoop
60000
50000
40000
30000
20000
10000
0
LL (no duplicate ack suppression)
0
10
20
30
40
50
60
70
80
Time (sec)
• 30-35% improvement for Snoop: LL congestion window
is small (but no coarse timeouts occur)
• Connection bandwidth-delay product = 25 KB
Suppressing duplicate acknowledgments and TCP-awareness
leads to better utilization of link bandwidth and performance
Snoop Protocol Status
• BSD/OS implementation
– Integrated with Daedalus low-latency handoff
software
• Version 1 released 1996; Version 2 released
1998
• In daily production use at Berkeley and UC
Santa Cruz
• Several hundred downloads
– Ports to Linux, FreeBSD, NetBSD
Summary: Wireless BitErrors
• Problem: wireless corruption mistaken for
congestion
• Solution: Snoop Protocol
• General lessons
– Lightweight soft-state agent in network infrastructure
• Guided by the End-to-End Argument
• Fully conforms to the IP service model
– Cross-layer protocolTransport
design & optimizations
Link-aware transport
(Explicit Loss Notification)
Network
Transport-aware link
(Snoop agent at BS)
Link
Physical
Conclusions
• Efficient data transport is a hard
problem: congestion, errors,
asymmetry,...
• Adaptive transmission schemes are
essential in the future Internet
• Architectural components should
include
– Congestion Manager (CM)
– Error-handlers (e.g., Snoop protocol)
– (And many other features)
• Wanted: a grand unified transmission