QVM
An Efficient Runtime for Detecting Defects
in Deployed Systems
Matthew Arnold
Martin Vechev
Eran Yahav
Motivation:
Dynamic analysis for Debugging
Testing
Production
 High overhead
 Low overhead is
tolerable
 Deep properties
relating to program
correctness
mandatory
 Very limited
information
Motivation:
Dynamic analysis for Debugging
Testing
Production
 High overhead
 Low overhead is
tolerable
 Deep properties
relating to program
correctness
mandatory
 Very limited
information
QVM Approach
 Virtual Machines are mainstream
 Java, C#, Perl, Python, PHP, etc…
 Runtime support to improve program correctness
 Array bounds checking, GC, etc
 VM implementations focus on performance
 Advanced dynamic optimization technology
 QVM vision
 Start using VM technology to improving program
correctness
 More advanced checking
 Efficient analyses
 Ubiquitous: simply turn on a VM flag
But Why Modify the VM?
 VM Disadvantages
 Portability
 Complexity
 Why not use:
 Aspects, JVMTI, java.lang.Instrument, bytecode inst. ???
 VM Advantages
 VM only information:
 Free bits in object header
 Can walk the heap if we desire (GC)
 Performance
 Exploit dynamic optimization
 Dynamic updating of instrumentation (via OSR, etc)
 Ease of deployment
New Overhead Philosophy
 Traditional dynamic analysis
 If I use X, how much overhead will it cost me?
 QVM: reverse the question
 I am willing to give you X% overhead
 What can you do for me?
 User specifies an overhead budget
 Goal: give user as much useful information as possible
 May miss errors, or report “I don’t know”
 But enables some checking in scenarios where it is
currently infeasible
Contributions
 Overhead manager (OHM)
 Adapts analyses to meet user specified target overhead
 Dynamic analyses checking correctness properties
 Typestate checking
 Object centric sampling
 Heap Assertions
 QVMI
 Overhead aware interface for medium-granularity
VM events
 Allocations, method calls, class loads,…
QVM Architecture
Application
typestate
specs
typestate
client
Clients
heap probes
client
assertions
client
QVMI
observed overhead
specified
overhead
OHM
VM Core
QVM
event
filters
Execution
Engine
adjust sampling rates
event
callbacks
violations
report
QVMI: The QVM Interface
 Profiling interface
 Similar to JVMPI/JVMTI
 Method calls, allocations, etc
 Key Difference: filtering on the VM side
agent
event
filters
JVMTI
event
callbacks
VM
Execution
Engine
QVMI: The QVM Interface
 Profiling interface
 Similar to JVMPI/JVMTI
 Method calls, allocations, etc
 Key Difference: filtering on the VM side
agent
event
filters
event
filters
JVMTI
QVMI
event
callbacks
VM
Execution
Engine
event
callbacks
Execution
Engine
QVMI: The QVM Interface
 At VM startup
 QVM profiling agents register with VM
 When compiling a method
 JIT queries QVM agents
 “Does invocation of method foo() require a call back?”
 If not, no callback is compiled into code
 Ensures no overhead for uninteresting events
 At runtime
 Callbacks go through QVM overhead can be measured
Overhead Manager (OHM)
 Monitoring: measure overhead incurred by clients
 Sampling strategy: events callbacks have adjustable sample rate
 Controller: adjusts sample rate based on measured overhead
QVMI
observed overhead
specified
overhead
OHM
event
filters
Execution
Engine
adjust sampling rates
VM Core
event
callbacks
Overhead Manager Challenges
 Fine grained timers critical
 Large number of small timings is difficult
 Must have notion of “total application time”
 We use Linux getrusage
 Multi-threaded apps have issues
 See paper
 Analyses must be able to be “turned off”
 OK to miss bugs
 But must not produce meaningless results
Origin-specific sampling
Randomly distributed sampling
produces undesirable results
Execution frequency
Code
eventA (…)
eventB (…)
eventC(…)
Origin-specific sampling
Origin-specific sampling
Maximizes code coverage
Execution frequency
Sample Counter
Counter
Reset
0
1
Code
eventA (…)
0
1
eventB (…)
37
100
eventC(…)
QVM Clients Analyses
 Typestate property checker
 Ordering of API calls
 Heap Probes
 Query properties of the heap
 Java Assertions
Overhead of all clients controlled
by Overhead Manager
Client 1: Typestate Property
Checker
*
dispose*
| release*
else
Object
allocation
undisposed
disposed
b
Object
death
err
*
Typestate Property Checker
 Simple to implement via QVMI
 Events used
 Object Allocation, method invocation, object death
 Sampling typestate is problematic
 Ex: File Open  Close
 High problem of sampling close but not open
 Solution: object-centric sampling
Object Centric Sampling
assert (…)
tracked
tracked
T t = new T()
assert(…)
 Tracked objects marked using
bit in object header
 Bit checked before executing
callbacks
Use Case Example: Azureus
Over 160 million downloads
Azureus Resource Leaks
 Typestate checker for undisposed GDI
resources
 Actual QVM report:
QVM ERROR: [Resource_not_disposed] object [0x98837030]
of class [org/eclipse/swt/graphics/Image]
allocated at site ID 2742 in method
[com/aelitis/azureus/.../ListView.handleResize(Z)V]
died in state [UNDISPOSED]
with last QVM method invoked [org/.../Image.isDisposed()Z].
Client 2: Heap Probes
 Heap Probes
 Allow programmer to query properties of the heap
 isShared(Object o1)
 Do two or heap objects point to o1
 isThreadOwned(Thread t, Object o)
 Is o reachable from only thread t only
 Uses components of a parallel GC to evaluate heap
queries
 Worst case: requires traversal of entire heap
 Probe sites automatically sampled by overhead manager
Azureus Example
class ListView extends ... {
private Image imgView = null;
// ...
protected void handleResize(boolean bForce) {
// ...
if (imgView == null || bForce) {
imgView = new Image(listCanvas.getDisplay(), clientArea);
lastBounds = new Rectangle(0, 0, 0, 0);
bNeedsRefresh = true;
} else {
// ...
}
// ...
}
}
OS
Resources
imgView
OS
Resources
Possible Fix
protected void handleResize(boolean bForce) {
// ...
if (imgView == null || bForce) {
if(imgView != null && !imgView.isDisposed()) {
OS
Resources
imgView
assert(!QVM.isShared (imgView));
imgView.dispose();
}
imgView = new Image(listCanvas.getDisplay(), clientArea);
lastBounds = new Rectangle(0, 0, 0, 0);
bNeedsRefresh = true;
} else {
// ...
}
// ...
}
OS
Resources
Experimental Evaluation
970.7
40
Base overhead
5% Budget
60.1 114.5261.7142.4 206.1189.7 41.2
54.3 180
272 166.24
35
30
Overhead
25
20
15
Global10
sampling
100
90
5
80
0
Percent of allocation sites
sampled
0.5
70
javac
jess
0.2
jack
db
60
50
40
30
20
10
0
eclipsedbluindex
mpegjess jackhsqldb
javacchartfopbloat
comp.
antlrmtrtpmd Average
0.7
luindex
bloat
hsqldb
chart Average
Overhead Manager: stabilization
Overhead Manager
Base overhead
40
970.7
60.1
114.5
261.7
5% Budget
142.4
10% Budget
206.1
189.7
20% Budget
41.2
54.3
Exhaustive
272
180
166.24
35
30
Overhead
25
20
15
10
5
0.5
0.2
0.7
0
javac
jess
jack
db
luindex
bloat
hsqldb
chart
Average
Leak Detection Results
Application
SWT
Resources
IOStreams
High
Frequency
Fixed
Azureus
11
0
4
5
Etrader
17
0
2
0
Feednread
1
7
0
0
Goim
3
0
1
3
IBMapp1
0
0
0
0
IBM app2
3
2
0
0
Jcommander
9
0
0
0
Juploader
0
1
0
0
nomadpim
2
0
0
0
Rssowl
8
3
0
0
Tvbrowser
0
5
0
0
Tvla
0
4
0
0
Virgoftp
6
0
0
6
Total
60
22
7
14
Sampling coverage (5% budget)
Global sampling
Origin-centric sampling
100
90
Percent of allocation sites sampled
80
70
60
50
40
30
20
10
0
eclipse
db
luindex mpeg
jess
jack hsqldb javac chart
fop
bloat comp. antlr
mtrt
pmd
Average
Summary
 Recap




Adaptive overhead controller
Clients: typestate, assertions, heap probes
QVMI
Found and fixed bugs several real applications
 Future Work
 Improve efficiency of heap assertions
 Concurrent or incremental evaluation
 Overhead manager
 Tighter overhead guarantees
The End