3G Meets the Internet: Understanding
the Performance of Hierarchical
Routing in 3G Networks
Wei Dong (U T Austin)
Joint work with
Zihui Ge, Seungjoon Lee
(AT&T Labs - Research)
ITC 2011, San Francisco, USA
September 2011
Outline
Background
Hierarchical routing vs. flat routing
Hierarchical routing and replicated service
Possible interaction with application layer
Summary
2
Why do we care about 3G performance
User’s expectation on 3G performance is higher than
3G networks now carry more traffic than ever before
ever before
3
Simplified 3G Architecture
SGSN
Node B
GGSN
User Device
RNC
A
B
C
Node B
4
Destination
Outline
Background
Hierarchical routing vs. flat routing
Hierarchical routing and replicated service
Possible interaction with application layer
Summary
5
Hierarchical routing vs. flat routing
Node B
SGSN
GGSN
User Device
RNC
A
B
C
Node B
6
Destination
Air Mile to GGSN
How long average packet travels (i.e., air mile)?
Metric: distance from RNC to SGSN to GGSN
Weighted average using traffic volume at RNC for a week
How average air mile changes as number of GGSNs varies
More GGSNs make the network increasingly flat
Incremental (start from 4 most populated cities) vs. from-the-scratch
Use all RNCs as candidate locations
Heuristics for placement
Greedy: iteratively choose the best location one by one
K-means: Clustering based on K initial points
We use the best of 10 runs with different random seeds
7
Placement Result
1400
1200
18% worse than optimal
Greedy from existing
Greedy
K−means
Lower bound
Average Distance (km)
1000
The benefit of adding more
GGSNs slows down
800
600
400
200
0
1
8
Having more GGSN locations reduces the air
miles significantly
2
3
4
5
6
7
# of GGSN Locations
8
Number of GGSN VS Weighted average distance
9
10
How does distance translate to delay?
Curve fitting using periodic probe data
Probe devices are at about 250 different locations across the
nation
~70 3G devices
~180 HSPA devices
One or two ping measurements per hour
We use the min for each (probe, server) pair for a day
Consider detour routing through GGSN when calculating
distance
Probe-> SGSN -> GGSN -> Server (external or internal)
9
Distance vs. RTT (HSPA)
RTT = 0.017 d + 59.9
(e.g., increase by 300km => ~5ms latency
increase)
10
Outline
Background
Hierarchical routing vs. flat routing
Hierarchical routing and replicated service
Possible interaction with application layer
Summary
11
Hierarchical routing and replicated
service
Node B
SGSN
User Device
A
GGSN
RNC
Node B
B
C
With CDN, relative
performance is even worse!
D
12
Destination
Hierarchical routing and replicated
service
Air mile to CDN server (weighted average)
EAG (Exit-at-GGSN): Current routing
RNC – SGSN – GGSN – CDN server
EAS, EAR (Exit at SGSN/RNC): idealized routing
RNC – SGSN – CDN or RNC – CDN
CDN server selection
Normal: closest to exit point
DNS caching can cause suboptimal selection (discussed later)
Location information
RNC (hundreds of different locations), SGSN (tens of different
locations), GGSN (tens of different locations)
Location of CDN servers (tens of different locations)
13
Air Mile vs. # CDN Locations
Current routing
Idealized routing
Suboptimal
server
selection
14
Relative performance
degrades
with more CDN
servers!
Air Mile vs. # CDN Locations
Current routing
Idealized routing
Suboptimal
server
selection
15
Benefit of idealized routing
may outweigh the benefit of
having more CDN locations
Distance Distribution (tens of CDN
Locations)
1
0.9
Difference of median is
851 km which translates to
23.7% difference in RTT
0.8
0.7
CDF
0.6
0.5
0.4
0.3
Current
0.2
Exit at SGSN
Exit at RNC
0.1
0
16
0
500
1000
1500
2000
2500
Distance (km)
3000
CDF of Distance
3500
4000
4500
Outline
Background
Hierarchical routing vs. flat routing
Hierarchical routing and replicated service
Possible interaction with application layer
Summary
17
Interaction with DNS Caching
Most CDNs use DNS to direct users to different servers
Browsers manage their own DNS cache
May not follow TTL set by DNS server
Browser
Timeout value
(min)
Market share (%)
IE
30
60.74
Safari
5
5.09
Firefox
1
22.91
User mobility may cause switching between WiFi and 3G interfaces
UE may use cached DNS entry and continue to go to old, suboptimal server
18
Switching from WiFi to 3G
Node B
SGSN
User Device
Normal scenario
after switching
A
GGSN
RNC
B
Before switching
C
D
19
DNS caching leads to
suboptimal
server
Destination
Measurement Setup
Measurement done on laptop PC with wifi card and USB 3G
card.
Browser: Internet Explorer
Sites: Akamai customers
Manually switch between WiFi and 3G to emulate mobility
Measure the download throughput of video (several minutes long)
Four scenarios
On WiFi, using WiFi CDN server (returned by WiFI DNS server)
On WiFi, using 3G CDN server (returned by 3G DNS server)
On 3G, using WiFi CDN server
On 3G, using 3G CDN server
20
Measurement: Akamai Customers from
NJ
Mbps
25
20
0.4
13 times
difference
0.35
0.3
15
0.25
10
Wifi to
0.2
3G-CDN
0.15
5
0
•
•
•
3G to
Wifi-CDN
Wifi to
0.1
Wifi-CDN
Larger throughput
gap with WiFi
0.05
Smaller difference with 3G, maybe due to bottleneck in
radio link => Likely to change
0 in LTE
More frequent switching possibleA1-1A1-2A2-1A2-2A3-1A3-2
with increasing WiFi
hotspots
WiFi
21
3G to 3GCDN
3G
Summary
Compared between idealized routing and current detour
routing in 3G architecture
Flat routing reduces air mile significantly but the difference in
end-to-end delay is only modest
Relative performance gap grows with replicated service
Interaction between routing change and DNS caching can cause
up to an order of magnitude throughput degradation
Our findings not only apply to current 3G networks
The difference in end-to-end delay can grow as wireless
technology improves further
The use of aggregation points still applies to recent cellular
architectures such as EPC
22
Q&A
Thanks!
23
Backup Slides
24
Switch from 3G to wifi
Node B
SGSN
Before switch
User Device
NJ
GGSN
IL
RNC
Normal
IN
NY
25
After switch
Destination
Considering routing change…
1
0.9
0.8
Over 1600 km!
0.7
CDF
0.6
0.5
0.4
0.3
Current
Exit at SGSN
0.2
Exit at RNC
Current − RNC CDN
0.1
0
26
Current − SGSN CDN
0
1000
2000
3000
Distance (km)
4000
CDF of Distance
5000
6000
Cost of triangular routing
1
0.9
0.8
0.7
CDF
0.6
0.5
0.4
0.3
Current
0.2
Exit at SGSN
Exit at RNC
0.1
0
27
0
500
1000
1500
2000
2500
Distance (km)
3000
CDF of distance based on RNCs
3500
4000
4500
Considering routing change…
1
0.9
0.8
0.7
CDF
0.6
0.5
0.4
Current
0.3
Exit at SGSN
Exit at RNC
0.2
Current − RNC CDN
Current − SGSN CDN
0.1
0
28
0
1000
2000
3000
Distance (km)
4000
CDF of distance based on RNCs
5000
6000
Measurement on Akamai
customers from MA
Mbps 30
25
20
3G to 3G-CDN
3G to Wifi-CDN
Wifi to 3G-CDN
Wifi to Wifi-CDN
15
10
5
0
MLB
29
MLB2
NBA
NBA2
Location: MIT
Audi
Audi2
Measurement on Limelight
customers from NJ
Mbps 18
16
14
12
10
8
6
4
2
0
30
3G to 3G-CDN
3G to Wifi-CDN
Wifi to 3G-CDN
Wifi to Wifi-CDN
Limelight has:
•Fewer locations
•Better
connectivity
last mile
Inefficiency
is less to
obvious.
Why?
providers via a global fiber-optic
network Location: AT&T Labs
Summary of measurement result
Inefficiency is more obvious when switch from 3G to Wifi
For 3G the air interface is the dominant part
Inefficiency is less obvious when there are fewer locations to
choose from
Akamai VS Limelight
Can become bigger issue in the future
Advances in wireless technology
Vertical handoff
31
Measurement on Akamai customers
from NJ – RTT
ms
400
350
300
250
3G to 3G-CDN
3G to Wifi-CDN
Wifi to 3G-CDN
Wifi to Wifi-CDN
200
150
100
50
0
MLB
32
NBA
Toyota