Configuration Fuzzing for
Software Vulnerability Detection
Huning Dai, Chris Murphy, Gail Kaiser
Columbia University
Observation
Most vulnerabilities only reveal themselves
under three conditions:
1. particular inputs
Fuzz Testing (Miller et al., 1988 )
A. Randomly generated inputs may fail to satisfy syntactic constraints.
B. It is hard to evaluate how much of the input/configuration
space is explored
C. Limited information about the "failure"
2
Observation
Most vulnerabilities only reveal themselves
under three conditions:
2. particular configurations of the software
Configuration Testing (Memon and Porter et al., 2004)
A. Didn’t apply to security testing.
B. Provide little information other than pass/fail
3
Observation
Most vulnerabilities only reveal themselves
under three conditions:
3. particular runtime environment.
Fault Injection (Hsueh et al., 1997)
A. Permutes the external environment.
B. Relies on the faults being injected.
C. Considerable false postives.
4
Our Solution

Configuration Fuzzing
A. Instead of generating random inputs, Configuration
Fuzzing mutates the application configuration.
B. To increase effectiveness, Configuration Fuzzing tests
are carried out “In Vivo” after a software is released, with
real-world inputs and runtime environment.
C. Instead of only checking for failure, surveillance functions
are run throughout the tests; these functions check for
violations of “security invariants” and log detailed
information.
5
Overview
Background
 Model
 ConFu Framework
 Case Studies
 Limitations and Conclusion

6
Background

In Vivo Testing (Murphy et al., 2009)
Executes tests in the context of the running
program after the software is released without
affecting the main process.

Security Invariants (Biskup, 2009)
Not merely
const int security;
const char secure;
But rules once broken indicates …
7
Approach

Configuration Fuzzing
Configuration Fuzzing mutates the application configuration
under predefined configuration constraints of the softwareunder-test to look for potential vulnerabilities.
Surveillance functions using security invariants are executed
throughout the test in order to detect vulnerabilities.
Tests are executed in the deployment process while the
application is running, “in vivoly”.
8
Model
9
Introduction to ConFu
ConFu: CONfiguration FUzzing testing
framework
 Steps:
1. Identifying the configuration variables
2. Generating fuzzing code
3. Identifying functions to test
4. Generating test code
5. Executing tests

10
STEP 1

Identifying the configuration variables
X11Forwarding yes
#[options.x11_forwarding]@{0,1}
TCPKeepAlive yes
#[options.tcp_keep_alive]@{0,1}
UseLogin
no
#[options.use_login]@{0,1}
Protocol
1
#[options.permit_root_login]@{1,2,3}
…
…
Part of the annotated configuration file of OpenSSH
11
STEP 2

Generating fuzzing code
typedef struct {
int x11_forward;
int tcp_keep_alive;
…
} result;
void fuzz_config()
{
/* generate a set of values */
result r=covering_array();
options.x11_forward = r.x11_forward;
options.tcp_keep_alive = r.tcp_keep_alive;
...
}
An example fuzzer for OpenSSH
12
STEP 3 & STEP 4

Identifying functions to test
do_child()  ConFu_do_child()

Generating test code
void ConFu_test_do_child(…)
{
fuzz_config(); /*Fuzz configuration*/
ConFu_do_child(…); /*Call the original
function*/
check_invariants();
}
Test function for do_child()
13
STEP 5

Executing tests
void do_child(…)
{
/*Create new process*/
int pid = fork();
if(pid == 0){
/*Test function*/
ConFu_test_do_child(…);
exit(0);
}
/*Original function*/
return ConFu_do_child();
}
do_child(Wrapper)
fork()
test_do_child(test)
exit
_do_child(original)
continue
Wrapper function for do_child()
14
Case Studies: Feasibility

Reproduce known vulnerabilities and use ConFu to
detect them.
CVE-2000-0525: early versions of OpenSSH do not properly drop
privileges when the UseLogin option is enabled, which allow local
users to execute arbitrary commands by providing the command to
the ssh daemon.
CVE-2009-2958: The tftp_request function in tftp.c in dnsmasq
before 2.50, when --enable-tftp is used, allows remote attackers to
cause a denial of service (NULL pointer dereference and daemon
crash) via a TFTP read (aka RRQ) request.
15
Case Studies: Performance

Target program: OpenSSH 2.1.0

Chosen function: do_child()

Configuration: permit root login, ignore rhosts,
ignore user known hosts, strict modes, x11
forwarding … a total of 15 configuration variables.

Environment: Intel Core2Quad Q6600 server with
2.40GHz and 2GB of RAM running Ubuntu 8.04.3
16
Case Studies: Performance

Results
#
of
tests
Overhead
introduced by
fuzz_config
Per test
Overhead
introduced by
_do_child
Per test
Overhead
introduced by
Check_invariants
Per test
Total Avg.
Additional
Time
Per test
100
0.034
0.0027
0.00001
0.037
1000
0.042
0.0024
0.00001
0.045
10000
0.038
0.0029
0.00001
0.041
100000
0.037
0.0023
0.00001
0.039
Overhead of instrumented do_child()(in seconds) with varying number of tests
17
Limitations and Future Work

Testers’ intervention is required to identify
the functions to test

A priori knowledge of the potential
exploitation behavior is required
18
Conclusion

Our contribution is an approach that checks
for software vulnerability after the software is
released and developed a testing framework
based on this approach.

Useful in helping developers build more
secure software and improve the security of
existing software systems.
19
Configuration Fuzzing for
Software Vulnerability Detection
Huning Dai
[email protected]
20
What is Covering Array?
A B C
0 0 0
0 1 1
1 0 1
1 1 0
A 2-way covering array for three variables
We notice that whichever two columns out of the
three columns are chosen, all possible pairs of
values appear. Specifically, the pairs 00, 01, 10 and
11 all appear in the rows when we look at the
columns of AB only, AC only and BC only.
21