An Internet-Wide View of
Internet-Wide Scanning
Zakir Durumeric, Michael Bailey, J. Alex Halderman
University of Michigan
USENIX Security ’14 ·∙ San Diego, CA
August 20, 2014
Internet-Wide Scanning
We released ZMap at USENIX Security last year • TCP Scan of full IPv4 in < 45 minutes !
Internet-­‐Wide scanning appears to be useful • 15 studies based on ZMap data masscan
An Internet-­‐Wide View of Internet-­‐Wide Scanning
2
Internet-Wide Scanning
We released ZMap at USENIX Security last year • TCP Scan of full IPv4 in < 45 minutes !
Internet-­‐Wide scanning appears to be useful • 15 studies based on ZMap data !
Who is using ZMap? !
Did ZMap alter the scanning landscape? !
Are operators now blocking Internet scans?
An Internet-­‐Wide View of Internet-­‐Wide Scanning
masscan
3
Talk Outline
1. Broad Overview of Scanning Landscape !
2. Case Studies: Scanning triggered by backdoors in home routers, Heartbleed, and NTP vulnerabiliWes !
3. Defensive reacWons against scanning !
An Internet-­‐Wide View of Internet-­‐Wide Scanning
4
Detecting Internet-Wide Scan Traffic
Data Collec:on !
•
!
•
!
•
Collected background traffic from a large network telescope at Merit Network during 2013–2014 Darknet does not host any services — probes are likely part of Internet-­‐wide scans Approach will likely miss targeted scanning 0.15%&of&IPv4&
Merit&
PCAP&
An Internet-­‐Wide View of Internet-­‐Wide Scanning
5
Detecting Internet-Wide Scan Traffic
Data Collec:on !
•
!
•
!
!
•
Collected background traffic from a large network telescope at Merit Network during 2013–2014 Darknet does not host any services — probes are likely part of Internet-­‐wide scans Approach will likely miss targeted scanning 0.15%&of&IPv4&
Merit&
Es:ma:ng Actual Scans !
•
Assume that scan targets are ordered by a uniform random distribuWon !
•
EsWmate coverage and scan rate using binomial distribuWon
An Internet-­‐Wide View of Internet-­‐Wide Scanning
PCAP&
6
How do we define a “scan”?
An Internet-­‐Wide View of Internet-­‐Wide Scanning
7
Defining a Scan
Des:na:on Source Rate Size
targeWng a single protocol on a single port set of conWguous IPs within a single AS sending at an esWmated rate of 10 pps reaching ≥100 hosts in our darknet
1.2.3.4 - 1.2.3.20
…
SYN Scan — TCP/443
…
Merit Darknet
An Internet-­‐Wide View of Internet-­‐Wide Scanning
8
Fingerprinting Scanners
We invesWgated open source network scanners and created fingerprints for ZMap and masscan !
ZMap !
• IP ID staWcally set to 54321 !
Masscan !
• IP ID = dest addr ⊕ dest port ⊕ tcp seqnum An Internet-­‐Wide View of Internet-­‐Wide Scanning
masscan
9
Network Telescope Traffic Overview
January 2014 !
• Darknet received an average 1.4 billion packets (55 GB) per day !
• Detected 10.8 million scans from 1.8 million unique hosts !
• 2,013 ZMap scans and 1,326 masscan scans
An Internet-­‐Wide View of Internet-­‐Wide Scanning
10
Targeted Services
6e+12
Probe Packets
5e+12
4e+12
3e+12
2e+12
1e+12
MP
SN s
ksy
Lin
TP
SM
RE
P
HE
NT YW
AN
pc
C
VL L
SQ
My
C
VN
N
SIP GE
AR
CH S
TP
HT t
lne
L
Te
SQ
MS TTP
-H
Alt
S
DN
P
RD
TP
HT
H
SS P
ICM MB
5/S
44
An Internet-­‐Wide View of Internet-­‐Wide Scanning
11
Targeted Services
6e+12
Probe Packets
5e+12
48.9% of scan traffic is from
small scans on port 445 —
Conficker Traffic
4e+12
3e+12
2e+12
1e+12
MP
SN s
ksy
Lin
TP
SM
RE
P
HE
NT YW
AN
pc
C
VL L
SQ
My
C
VN
N
SIP GE
AR
CH S
TP
HT t
lne
L
Te
SQ
MS TTP
-H
Alt
S
DN
P
RD
TP
HT
H
SS P
ICM MB
5/S
44
An Internet-­‐Wide View of Internet-­‐Wide Scanning
12
4e+11
3e+11
2e+11
Probe Packets
Targeted Services
6e+11
5e+11
1e+11
S
IO
etB
9/N
13
MP
SN s
ksy
Lin
TP
RE
SM
HE
P
NT YW
AN
pc
C
VL L
SQ
My
C
VN
N
SIP GE
AR
CH S
TP
HT t
lne
0)
Te QL (808
S
MS TTP
-H
Alt
S
DN
P
RD
TP
HT
H
SS P
ICM
13
An Internet-­‐Wide View of Internet-­‐Wide Scanning
Targeted Services
6e+11
<.1%
.1-1%
1-10%
10-100%
Probe Packets
5e+11
4e+11
3e+11
2e+11
1e+11
S
IO
etB
9/N
13
MP
SN s
ksy
Lin
TP
RE
SM
HE
P
NT YW
AN
pc
C
VL L
SQ
My
C
VN
N
SIP GE
AR
CH S
TP
HT t
lne
0)
Te QL (808
S
MS TTP
-H
Alt
S
DN
P
RD
TP
HT
H
SS P
ICM
An Internet-­‐Wide View of Internet-­‐Wide Scanning
14
Large Scans
1
Non-Conficker Scanning
0.9
CDF of Probe Packets
0.8
0.7
0.6
0.5
0.4
0.3
78% of traffic is from scans targeting >1% of IPv4
0.2
0.1
0
0.0001%
0.001%
0.01%
0.1%
1%
10%
100%
Estimated Scan Coverage
An Internet-­‐Wide View of Internet-­‐Wide Scanning
15
Large Scans
1
Non-Conficker Scanning
0.9
CDF of Probe Packets
0.8
0.7
0.6
62% of traffic is from scans targeting >10% of IPv4
0.5
0.4
0.3
0.2
78% of traffic is from scans targeting >1% of IPv4
0.1
0
0.0001%
0.001%
0.01%
0.1%
1%
10%
100%
Estimated Scan Coverage
An Internet-­‐Wide View of Internet-­‐Wide Scanning
16
Scan Dynamics
January 2014 !
• 18,000 scans (0.28%) targeted ≥1% of the IPv4 address space !
• 2,700 scans (0.04%) targeted ≥10% of the IPv4 address space !
• 100 ASes responsible for 85% of this scan traffic An Internet-­‐Wide View of Internet-­‐Wide Scanning
17
Scan Dynamics
January 2014 !
• 18,000 scans (0.28%) targeted ≥1% of the IPv4 address space !
• 2,700 scans (0.04%) targeted ≥10% of the IPv4 address space !
• 100 ASes responsible for 85% of this scan traffic !
Four types of scanning stand out:
• Academic and industry research groups !
• Regularly scheduled scans from Chinese ASes !
!
• UnidenWfiable scans from bullet-­‐proof hosWng providers • ShodanHQ Search Engine
An Internet-­‐Wide View of Internet-­‐Wide Scanning
18
Research Groups and Security Consultants
Many of the networks responsible for the most scan traffic are academic insWtuWons and consultants performing regular scans !
Primarily focused on amplificaWon alacks (NTP, DNS) and cryptographic ecosystems (SSH, HTTPS) !
In almost all cases, studies appear to be conducted responsibly and allowed easy exclusion An Internet-­‐Wide View of Internet-­‐Wide Scanning
19
Regular Chinese Scans
500
Regular daily scans of ICMP, SSH, SQL Server, and TCP/0 TCP/0 — non-­‐standard-­‐compliant port frequently used to fingerprint network stacks and bypass firewalls !
Responsible for the majority of ICMP, SQL Server, MySQL, and ICMP traffic — far more than other countries
400
January 2014 Scans
!
China
United States
Netherlands
Others
300
200
100
0
L
SQ
MS
TP
PP L
SQ
My
P/0
TC -alt
TP
HT
P
RD L
SQ
MS P
ICM
H
SS
An Internet-­‐Wide View of Internet-­‐Wide Scanning
20
Large Hosting Providers
50% of the top 100 ASes responsible for scan traffic were large hosWng providers !
Many were bullet-­‐proof hosWng providers !
Bullet-­‐Proof Hos:ng Providers !
!
!
• AdverWse turning a blind-­‐eye to malicious behavior • Scanning for almost every common protocol • Very rarely any idenWfiable informaWon about owners
An Internet-­‐Wide View of Internet-­‐Wide Scanning
Top Scanning Providers
Ecatel Network (NL)
Plus Server (DE)
Slask Data Center (PL)
Single Hop (US)
CariNet, Inc. (US)
Server4You (DE)
OVH Systems (UK)
Thor Data Center (IS)
Psychz Networks (US)
21
Talk Outline
1. Broad Overview of Scanning Landscape !
2. Case Studies: Scanning triggered by backdoors in home routers, Heartbleed, and NTP vulnerabili:es !
3. Defensive reacWons against scanning !
An Internet-­‐Wide View of Internet-­‐Wide Scanning
22
Linksys Router Backdoor
Linksys Backdoor Probes
1e+10
1e+09
<1% Scans
>=1% Scans
Public Disclosure
1e+08
1e+07
1e+06
12/14
12/21
12/28
01/04
01/11
01/18
01/25
Date
An Internet-­‐Wide View of Internet-­‐Wide Scanning
23
Linksys Router Backdoor
Linksys Backdoor Probes
1e+10
1e+09
<1% Scans
>=1% Scans
Public Disclosure
All non-Shodan scans used ZMap (71%) or masscan (29%)
1e+08
1e+07
1e+06
12/14
12/21
12/28
01/04
01/11
01/18
01/25
Date
An Internet-­‐Wide View of Internet-­‐Wide Scanning
24
Open NTP Resolvers
97.3% of probe traffic is part of large scans (targeWng >1% of IPv4) !
Primarily scanned from bullet-­‐proof hosWng providers. !
50% of scans used ZMap or Masscan “#yolo”
“#lulz”
“Openbomb
Drone Project”
!
Not certain that scanners are malicious, but absolutely appear so An Internet-­‐Wide View of Internet-­‐Wide Scanning
http://ra.pe
25
Heartbleed Vulnerability
Scans began <24 hours aver disclosure !
53 scans from 27 hosts in the week following disclosure !
38% of scans originated from China !
Scans occurring from bulletproof hosWng providers !
95% of scans used ZMap or Masscan
An Internet-­‐Wide View of Internet-­‐Wide Scanning
26
Heartbleed Vulnerability
Scans began <24 hours aver disclosure !
53 scans from 27 hosts in the week following disclosure !
Matter of Heartbleed
38% of scans originated from China !
IMC’14, Vancouver
Scans occurring from bulletproof hosWng providers !
95% of scans used ZMap or Masscan
An Internet-­‐Wide View of Internet-­‐Wide Scanning
27
So what about ZMap?
The majority of scan traffic is not generated by ZMap 7e+08
!
!
Evidence that alackers are starWng to take advantage of ZMap and Masscan !
An Internet-­‐Wide View of Internet-­‐Wide Scanning
4e+08
3e+08
2e+08
1e+08
0
ot
yB
Sp d
ui
Sq Bot
ra
Aid QL
S
MS AP
M
EP in
dm
Ra P
T
PP P
uln
M
SN ys V
ks
Lin P
T
SM
C
VL
P
ere
NT ywh
An
pc QL
S
My
SIP
N
C
VN RGE
A
CH PS
T
HT P-alt
T
HT
P
ICM QL
S
MS
P
RD P
T
HT
S
DN
H
SS
UlWmately lowers the barrier of entry for both groups 5e+08
Scan Probes
Research groups are using ZMap responsibly 6e+08
ZMap
Masscan
Other
28
Talk Outline
1. Broad Overview of Scanning Landscape !
2. Case Studies: Scanning triggered by backdoors in home routers, Heartbleed, and NTP vulnerabiliWes !
3. Defensive reac:ons against scanning !
An Internet-­‐Wide View of Internet-­‐Wide Scanning
29
Do networks drop scan traffic?
Michigan Engineering AS is responsible for 3rd most scan traffic !
Performed simultaneous scans from Georgia Tech and Michigan to detect blocked traffic !
Scanned using same randomizaWon seed
—reduce hosts lost due to churn
An Internet-­‐Wide View of Internet-­‐Wide Scanning
30
Do networks drop scan traffic?
EsWmated 0.05% of IPv4 address space is no longer accessible !
208 exclusion requests — 0.15% of IPv4 address space !
Dropped traffic and excluded networks have a minuscule impact
An Internet-­‐Wide View of Internet-­‐Wide Scanning
31
When do networks drop scan traffic?
6e+06
Total Blacklisted Address Space
Est. Inaccessible Address Space
IP Addresses
5e+06
4e+06
3e+06
2e+06
1e+06
0
/1
01
/1
10
/1
07
/1
04
/1
01
/1
10
/1
07
/1
04
/1
01
4
3
3
3
3
2
2
2
2
An Internet-­‐Wide View of Internet-­‐Wide Scanning
32
How are organizations noticing?
Detec:on Mechanism
Organiza:ons
Firewall Logs
22 (34%)
Web Server Logs
14 (22%)
IDS Logs
10 (16%)
Invalid SSH or OpenVPN Handhshake
10 (16%)
Public Blacklists
2
(3%)
Other
6
(9%)
An Internet-­‐Wide View of Internet-­‐Wide Scanning
33
Future Work
Exclusion standard !
Understand defensive reacWons !
!
CorrelaWng distributed scanners !
Determining scan intent An Internet-­‐Wide View of Internet-­‐Wide Scanning
34
Conclusion
Scanning landscape has shived — large horizontal scans are now common !
Internet-­‐Wide scanning is a combinaWon of both researchers and alackers taking advantage of new tools !
Network operators have been slow to respond to scanning despite scanning being easy to detect !
Internet-­‐Wide scanning remains a valid methodology An Internet-­‐Wide View of Internet-­‐Wide Scanning
35
Questions?
An Internet-Wide View of Internet-Wide Scanning
Zakir Durumeric, Michael Bailey, J. Alex Halderman
University of Michigan
[email protected]
USENIX Security ’14 ·∙ San Diego, CA
August 20, 2014