IP Reachability Differences: Myths and Reali<es Dr. Dan Massey
Colorado State University
He Yan, Ben Say, Dave Oko, Brendan Sheridan
Colorado State University
Dr. Christos Papadopoulos
Colorado State University
04/15/2011 IP Reachability: Myths and Reality Dr. Dan Pei
AT&T Research
1
Ideally, all ISPs provide you with reachability to same Internet In practice: Different ISPs can reach different portions of the Internet.
In other words, there are some prefixes that can be reached via ISP A
but cannot be reached by ISP B (a vice versa).
The reasons why this occur are well known:
Bogon filtering causes reachability differences for newly allocated prefixes.
Bogon filtering matters, but the fact is that regional differences matter
most and these differences may be political, economic,
technical, or a combination of the above
Bogon filtering and regional issues matter, but the real difference is due to
various customer only services.
Even if there are differences, there are very little (if any) “production” services
on networks that are not globally reachable from most ISPs.
04/15/2011 IP Reachability: Myths and Reality 2
Our Objec<ves 1.  Demonstrate reachability differences exist and quan<fy the extent of these differences. 2.  Examine which conjectures (if any) provide a significant contribu<on to the differences. 04/15/2011 IP Reachability: Myths and Reality 3
Methodology •  Compare BGP Rou<ng Tables from 25 ISPs –  Data collected from Oregon RouteViews –  All peers send full tables of over 310K prefixes –  Studied from April 1 to April 20, 2010 •  Study Uses ISP0 As a Central Point of View –  ISP0 is a <er-­‐1 ISP –  Have addi<onal data on ISP0 opera<ons –  Compare each of the other 24 ISPs to ISP0 04/15/2011 IP Reachability: Myths and Reality 4
Types of Table Differences •  Exact Matches: ISP0 prefix appears ISPX table –  Ex: Both ISP0 and ISP2 have a route for 129.82/16 •  Covered Prefixes: ISP0 prefix does not appear in ISPX, but ISPX can reach all IP addresses –  Less Specific: 131.179.0/17 does not appear in ISPX, but 131.179/16 does appear in ISPX –  More specifics: 131.179.0/17 does not appear in ISPX, but 131.179.0/18 and 131.179.64/18 appear in ISP X •  Remainder represents poten<al reachability differences. 04/15/2011 IP Reachability: Myths and Reality 5
Exact Matches 313000
Number of Prefixes
312000
311000
310000
309000
308000
98.5% of ISP0 prefixes also appear in ISPX
307000
306000
0
5
10
15
ISP Index
20
25
(not shown here, but 97.25% of ISPX prefixes also appear in ISP0)
04/15/2011 IP Reachability: Myths and Reality 6
The BGP Table Differences 4500
4000
On average, 3250 prefixes appeared in
ISP0 tables, but did not appear in ISP22
tables
Number of Prefixes
3500
3000
2500
2000
1500
1000
500
0
04/15/2011 0
5
10
15
ISP Index
IP Reachability: Myths and Reality 20
25
7
Covered Prefixes 7,000
Number of Prefixes
6,000
5,000
Not Covered
Covered
4,000
3,000
Nearly all ISP0 prefixes that don’t appear in
ISPX are covered by other table entries in ISPX
2,000
1,000
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
ISP Index
04/15/2011 IP Reachability: Myths and Reality 8
Overall Reachability Differences (Prefixes) 10,000
Not Reachable
Partially Reachable
Number of Prefixes
1,000
100
10
1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
ISP Index
04/15/2011 IP Reachability: Myths and Reality 9
Overall Reachability Differences (IP Addresses) 1e+08
Number of Destinations (IPs)
1e+07
1e+06
100,000
10,000
1,000
100
1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
10
ISP Index
04/15/2011 IP Reachability: Myths and Reality 10
Our Objec<ves 1.  Reachability differences do exist. –  Small percentage (less than 3%) of total routes. –  Can be up to 1000 prefixes and millions of IP addresses 2.  Examine which conjectures (if any) provide a significant contribu<on to the differences. 04/15/2011 IP Reachability: Myths and Reality 11
Transient Changes •  Short Lived BGP Table Differences –  Fast learners: IPS0 learns the route first, eventually other ISPs catch up and also learn the route –  Slow to react: ISPX learned the prefix is unreachable, but it takes longer for ISP0 to withdraw the route. •  Long Term BGP Table Differences. –  For mul<ple days, ISP0 could reach the prefix but ISPX could not. 04/15/2011 IP Reachability: Myths and Reality 12
Transient Behaviors 2,500
1/3 of reachability differences persisted for
entire study period.
Number of Prefixes
2,000
1,500
1,000
500
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
Number of Days Appeared
1/3 of reachability differences are short lived
04/15/2011 IP Reachability: Myths and Reality 13
Bogon Routes •  ISPs Apply Policies to Filter Invalid Routes –  No universal agreement on what is “invalid” •  New Alloca<ons and Bogons –  Filter unallocated prefixes and update filter ager alloca<on –  Going back 7 months accounts for 43 prefixes. –  Going back years yields 295 prefixes. •  Management Strategies and RADb –  92 prefixes with no entry in RADb –  199 prefixes with no origin in RADb. •  Does produce reachability differences, but only explains a small percent of differences. 04/15/2011 IP Reachability: Myths and Reality 14
Regional Differences 2,500
1895 prefixes from India, only unreachable
from ISP 22. ISP 22 added these routes on
day 13 of study.
Number of Prefixes
2,000
1,500
1,000
500
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
Number of Days Appeared
Bulgaria accounted for most prefixes (509)
Great Britain was second highest (168)
04/15/2011 IP Reachability: Myths and Reality 15
Customer Services and Prefix Usage •  ISPs Provide Specialized Services to Customers –  Confirmed ISP0 has several such prefixes –  Most are covered prefixes, but did find some uncovered prefixes –  Accounts for 9 prefixes in our study. •  Are Services Running on the Impacted Prefixes? –  Scanned prefixes looking for common services –  Limited scan to /19 or more specifics –  Responding prefixes further checked for DNS entries •  Contacted Prefix Owners –  Responses include reports of reachability problems, blacklist issues, network tes<ng, and so forth •  Valid Global Services Running on Clear Majority of Prefixes 04/15/2011 IP Reachability: Myths and Reality 16
RouteViews Data in Real-­‐Time 04/15/2011 IP Reachability: Myths and Reality 17
Conclusions •  Reachability Issues Do Exist –  Over 97% of prefixes appeared as exact matches –  But differences do occur and impact both research and opera<ons •  Common Conjectures: –  Transient issues, Bogons, Management, Regional Policies, Configura<on Errors, Special Services all proposed as explaina<ons –  All are valid, but none are dominant. 04/15/2011 IP Reachability: Myths and Reality 18
Ques<ons [email protected] 04/15/2011 IP Reachability: Myths and Reality 19