Shield: Vulnerability-Driven EndHost Firewall for Preventing Known
Vulnerability Attacks
Sigcomm ’04
Software patching not an effective first-line
defense
• Sasser, MSBlast, CodeRed, Slammer, Nimda,
Slapper all exploited known vulnerabilities
whose patches were released months or weeks
before
• 90+% of attacks exploit known vulnerabilities
[Arbaugh2002]
• People don’t patch immediately
Why don’t people patch?
• Disruption
– Service or machine reboot
• Unreliability
– Software patches inherently hard to test
• Irreversibility
– Most patches are not designed to be easily
reversible
• Unawareness
Firewall also not an effective
first line defense
• Traditional firewalls
– Typically in the network
• One-size-fits-all solution, lack application-awareness, miss end-to-end
encrypted traffic
– Course-grained
• High false positive rate
• Exploit-driven firewalls (or 1st gen intrusion detection)
– Filter according to exploit (attack) signatures
• Attack code obfuscation, e.g., polymorphism, metamorphism, can
evade the firewall
– Worms spread fast (in minutes or seconds!)
• Real-time signature generation and distribution difficult
Shields: End-host Vulnerability-Driven
Network Filters
• Goal: Protect the time window between vulnerability
disclosure and patch application.
• Approach: Characterize the vulnerability instead of its exploits
and use the vulnerability signature for end-host firewalling
• Shields combine the best features of
– Patches: vulnerability-specific, code level, executable
– Firewall: exploit-specific, network level, data-driven
• Advantages of Shield:
– Protection as good as patches (resilient to attack variations), unlike
exploit-driven firewalls
– Easier to test and deploy, more reliable than patches
Vulnerability vs. Exploit (1:M)
• Many exploits against a single vulnerability
– E.g., many different strings can overrun a vulnerable buffer
• Vulnerability signatures generated at vulnerability
discovery time
– E.g., sizeof (msg.buffer) > legalLimit
• Exploit signatures generated at attack time
– E.g., Snort signature for Slammer:
alert udp $EXTERNAL_NET any -> $HOME_NET 1434 (msg:"MS-SQL
Worm propagation attempt"; content:"|04|"; depth:1;
content:"|81 F1 03 01 04 9B 81 F1 01|"; content:"sock";
content:"send";
Overview of Shield Usage
New Shield Policy
Incoming or
Outgoing
Network Traffic
Shield
Policies
Shielded Traffic
to Processes or
Remote Hosts
End-Host Shield
• Shield intercepts vulnerable application traffic above the
transport layer.
• Policy distribution very much like anti-virus signature
model – automatic, non-disruptive, reversible
Vulnerability Modeling
Vulnerability
State
Machine
S0
S0
Message
S1
S2
V4
S4
S3
S2
Exploit
Event
Protocol State Machine
S5
V4
S5
Application Functionality
in S2
Shield Policy (Vulnerability Signature):
Vulnerability state machine + how to recognize and react
to exploits in the vulnerable state
Protocol analysis is the key for vulnerability-driven filtering
Shield Architecture: Goals
• Flexibility: support protocol analysis for any
application level protocols
• Fidelity: protocol analysis consistent with that
of the application
• DoS resilience: hold less state than that of the
application
Flexibility: separate mechanism from
policy
• Mechanism: protocol analysis – reconstruct
message and session semantics: e.g., parsing,
state machine operations
– GAPA: generic application-level protocol analyzer
• Policy: a language that describes protocol state
machine, message formats, and specific
vulnerabilities
– GAPAL: GAPA language
• Shield policy: a GAPAL script that blocks
according to a vulnerability
Achieving Shield Fidelity
• Infidelity results in evasion or false positives
• Sources of inconsistencies:
– Misunderstanding of the protocol or message
format
• Test suites or trace-driven debugging
– Event dispatching logic:
• Session as an abstraction independent of socket or host
pair
– Scattered message arrivals:
• Message as an abstraction independent of the packet
Achieve DoS-resilience:
• Session state:
– Current protocol state
– Parsing state
– Handler continuation
• Parsing:
– Exploit-checking only -- much streamlined parsing
– Aggressive byte skipping
– Save the partial field only (instead of partial
message)
Achieving Safety: GAPAL
Protocol <protoName> {
uses <lowerLayerName>
transport = { TCP|UDP/<port> }
// session-local vars
<baseType> <varName>;
grammar {
// msg-local vars
<baseType> <varName>;
NonTerminal  <name>:<type>
{ <code>}
….
};
State-machine <name> {
(<state>, IN|OUT|Timeout)  handler;
initial-state = <stateName>;
final-state = <stateName>;
};
Session-identifier (<startNonTerminal>) {
<code>
return <session ID>;
};
Handler <name> (<startNonTerminal>) {
// handler-local vars
<baseType> <varName>;
<grammar visitor>
<post-parsing code>
return “<nextState>;
};
}; // protocol
Key Properties of a GAPAL
• Completeness
– Binary as well as text-based protocols
– Layering
• Ease of authoring protocol descriptions
– Payload parsing grammar similar to BNF
– E.g., HTTP RFC spec - text ~= GPA policy for HTTP
• Safety
–
–
–
–
Strong typing
No dynamic memory allocation
No general-purpose loops or iterators
Semantic checking and optimization at compile time
GAPA as a General Facility
• Rapid protocol analysis enabler for IDSes,
firewalls, and network monitors; and allow
flexible customization
• Easy authoring of Shield vulnerability signature
– Vulnerability signature authoring as refinement of
previously specified protocol
– Merging vulnerability signatures of the same
application becomes trivial
Shield Architecture
Policy
Loader
Per-App
Spec
New
Policies
Exe->Spec ID
How to parse message
How to identify a session
Raw bytes
Port #
Application
Dispatcher
ParsePayload
Drop
TearDownSession
Raw bytes
Spec ID
HandlerAt(State, Event)
Session
Dispatcher
Shield
Interpreter
Event for
Session i
State
Machine
Engine
Interpret (Handler)
CurState
SetNextState
Session
Session
Session
State
i
State
State
Shield Implementation and Evaluation
• First prototype implemented as Windows Layered Service
Provider (LSP)
– Working shields for vulnerabilities behind Blaster, Slammer, and
CodeRed
– Near-zero false positives
– Performance and scalability results promising:
• Negligible overhead for end user machines
• 14-30% throughput overhead for an artificial scenario stressing Shield
• Second prototype based on GAPAL
– 48Mbps for CodeRed, 72Mbps for host header, 8-18Mbps for Blaster
• MSRC 2003 Bulletin study (49 bulletins)
– All 12 worm-able vulnerabilities are easily shield-able
– Some of the other 37 may also be shield-able
Comparison Across Defenses
Software Patch
Firewall
Traditional IDS
Shield
Time Window
After patching
After vul
discovery
After vul
discovery
After vul
discovery
Vul-specific?
Yes
Service-specific Exploit-specific
Yes
Working Layer
Application
Network
Transport
Transport to
Application
Easy to Deploy
No
Yes
Yes
Yes
False Alarming
No
High
Medium
Low
Limitation
• Low throughput
– Parallelization across protocols?
• Difficult to write application-level protocol
– Automatic protocol generation?
• Protocol parser is coupled with signature,
resulting in poor extensibility.
– Decouple protocol parser with signature engine.
NetShield: Massive Semantics-Based
Vulnerability Signature Matching for
High-Speed Networks
Sigcomm ’10
Speed
High
Motivation of NetShield
State of the
art regex Sig
IDSes
NetShield
Theoretical accuracy
limitation of regex
Low
Existing
Vulnerability
Sig IDS
Low
Accuracy
High
21
Matching Problem Formulation
• Suppose we have n signatures, defined on k matching
dimensions (matchers)
– A matcher is a two-tuple (field, operation) or a four-tuple
for the associative array elements
– Translate the n signatures to a n by k table
– This translation unlocks the potential of matching multiple
signatures simultaneously
Rule 4: URI.Filename=“fp40reg.dll” && len(Headers[“host”])>300
RuleID Method == Filename == Header == LEN
1
DELETE
*
*
2
POST
Header.php
*
3
*
awstats.pl
*
4
*
fp40reg.dll
name==“host”; len(value)>300
5
*
*
name==“User-Agent”; len(value)>544
22
Matching Algorithms
Candidate Selection Algorithm
1.Pre-computation: Decides the rule order and
matcher order
2.Runtime: Decomposition. Match each matcher
separately and iteratively combine the results
efficiently
23
Step 2: Iterative Matching
PDU={Method=POST, Filename=fp40reg.dll,
Header: name=“host”, len(value)=450}
S1={2} Candidates after match Column 1 (method==)
S2= S1 A2 +B2 ={2}
{}+{4}={}+{4}={4}
S3=S2
A3+B3 ={4}
{4}+{}={4}+{}={4}
Si  Ai 1
Don’t care
RuleID Method == Filename
== Header == LEN
R1
R2
R3
1
2
DELETE
SiPOST
*
matcher i+1
*
Header.php
*
*
3
*
awstats.pl
4
*
fp40reg.dll
5
*
*
Si  Ai 1
require
In Ai+1 len(value)>300
name==“host”;
matcher i+1
name==“User-Agent”; len(value)>544
24
24
High Speed Parsing
Tree-based vs. Stream Parsers
Keep the whole parse
Parsing and matching
VS.
tree in memory
on the fly
Parse all the nodes
in the tree
Only signature related
VS.
fields (leaf nodes)
• NetShield uses stream parser to save memory
and running overhead.
25
Parsing Results
Trace
TH
DNS
TH
NU
TH
WINRPC WINRPC HTTP
Avg flow len (B)
77
879
596
6.6K 55K 2.1K
Throughput
(Gbps)
Binpac
Our parser
0.31
3.43
1.41
16.2
1.11
12.9
2.10 14.2 1.69
7.46 44.4 6.67
11.2
Max. memory per 16
11.5
15
11.6
15
3.6
14
Speed up ratio
NU
HTTP
3.1
14
DARPA
HTTP
3.9
14
connection
(bytes)
26
Parsing+Matching Results
8-core 11.0
Trace
TH
NU
TH
WINRPC WINRPC HTTP
NU
HTTP
DARPA
HTTP
Avg flow length (B)
879
596
6.6K
55K
2.1K
10.68
14.37
4
9.23
10.61
1.8
0.34
2.63
11.3
2.37 0.28
17.63 1.85
11.7 8.8
1.48
32
0.033 0.038 0.0023
28
28
28
Throughput (Gbps)
Sequential
CS Matching
Matching only time
speedup ratio
Avg # of Candidates 1.16
Avg. memory per
connection (bytes)
32
27
Scalability Results
Throughput (Gbps)
0
1
2
3
4
Performance
decrease
gracefully
0
200
400
600
# of rules used
800
28
Contribution
• Significantly speed up vul sig matching,
making vul-based IDS practical.
Limitation
• No support for stateful vul. compared with
Shield.
• Speed-up is only effective for symbolic
constraint signature.
– “HttpMethod=POST && HttpFilename=header.php”