RClassify: Classifying Race
Conditions in Web Applications via
Deterministic Replay
Lu Zhang
Chao Wang
ICSE, May 24, 2017
Client-side Web Application
2
HTML/CSS + JavaScript
• HTML
• CSS
• JavaScript
 content
 look
 code (UI events, etc.)
3
Event Execution Order
ev1: parsing(image1)
ev2: parsing(script1)
ev3: parsing(outputField)
ev4: firing(image1.onload)
image1Loaded
Write
ev2: parsing(script1)
Read
ev4: firing(image1.onload)
4
Event Execution Order
ev1: parsing(image1)
ev2: parsing(script1)
ev3: parsing(outputField)
ev4: firing(image1.onload)
image1Loaded
Write
ev2: parsing(script1)
outputField
Read
ev4: firing(image1.onload)
Read
Write
ev3: parsing(outputField)
ev4: firing(image1.onload)
5
Execution Order
ev1: parsing(image1)
ev2: parsing(script1)
ev4: firing(image1.onload)
ev3: parsing(outputField)
image1Loaded
Write
ev2: parsing(script1)
outputField
Read
ev4: firing(image1.onload)
Read
Write
ev3: parsing(outputField)
ev4: firing(image1.onload)
6
Race Conditions
• Common in client-side web applications
– Hard to diagnose
• Existing tools report too many false positives
– Overwhelm the developers
– Counter-productive
7
Our Contribution
• Classifying race-condition warnings
– Bogus
– Real
– Harmless
– Harmful
… based on evaluating the actual impact
8
Outline
• Motivation
• Background
• Our Method
– Part 1: Deterministic replay
– Part 2: State comparison
• Evaluation
• Conclusion
Must-Happen-Before
ev1: parsing(image1)
ev2: parsing(script1)
ev3: parsing(outputField)
ev4: firing(image1.onload)
Ev 4: undefined
outputField
Ev 4: undefined
image1Loaded
Bad run#1: onload before
parsing script1
Ev 1: Parsing
image1
Bad run#2: onload before
parsing outputField
Ev 2: Parsing
script1
Ev 3: Parsing
outputField
Ev 4: Done!
Good run: onload
after parsing outputField
10
Main Challenge
• Many events & many event types
–
–
–
–
–
–
Parsing
User actions
Asynchronous resource loading
AJAX callbacks
Timers
…
11
Async JS
HTML
onload
Interleaving
of Events
Ajax
Timer
onclick
f1()
<image src=… onload = “f1()”>
a.js
<script src=“a.js” async = “true”>
f2()
<script>
Ajax(url , f2)
</script>
<script>
setTimeOut(f3, 1000);
</script>
f3()
f4()
<button onclick= “f4()”>
…
12
Erroneous Interleaving
f1()
<image src=… onload = “f1()”>
a.js:
function f4(){…}
<script src=“a.js” async = “true”>
f2()
<script>
Ajax(url , f2)
</script>
<script>
setTimeOut(f3, 1000);
</script>
f3()
f4()
<button onclick= “f4()”>
…
Unexpected
behavior
13
Outline
• Motivation
• Background
• Our Method
– Part 1: Deterministic replay
– Part 2: State comparison
• Evaluation
• Conclusion
Our Method
URL of Web
Application
Race-condition
Warnings
Compare the
Program States
Harmful or
Harmless
Static Analysis
of HTML files
Execution 1
Instrumented
Web Application
Execution 2
Replay the Racing
Event Pair
15
Intercepting the Events
• We need to intercept the racing events at run time
– Statically instrument the HTML files
– Instrument the APIs that create these events
– Periodically scan the DOM to detect added events
• We need to postpone them whenever needed
16
Postponing the Events
• Race condition: A → B
B
Waitlist:
Empty
Waitlist:
A
A
B
Notify
Notify
A
Time
• Wait until all events in the Waitlist are executed
17
Outline
• Motivation
• Background
• Our Method
– Part 1: Deterministic replay
– Part 2: State comparison
• Evaluation
• Conclusion
Recording the Program State
A
Executing:
A→B
B
Program
State
• Record the state after each execution, including:
–
–
–
–
HTML DOM tree
List of event handlers added dynamically
JavaScript variables
Environment variables
19
Filtering the Irrelevant Fields
• Certain fields may have different values due to
reasons other than race conditions
•
•
•
•
Date and time
Unique session id
Nondeterministic response from the server
Other factors
• 3-way comparison:
• A→B
• A→B
• B→A
20
Outline
• Motivation
• Background
• Our Method
– Part 1: Deterministic replay
– Part 2: State comparison
• Evaluation
• Conclusion
Implementation
• Race Detection tool
– EventRacer
[Raychev et al., OOPSLA’13] [Petrov et al., PLDI’12]
• HTML Instrumentation
– Jsoup 1.8.1 with 2.4K lines of Java code
• Event Monitoring/Control
– Our JavaScript libraries (Rclassify.js, PreRClassify.js)
– 2.1K lines of JavaScript code
• Execution Environment
– Teleport Ultra 1.69 – download the content of websites
– Firefox (35.0 – 47.0) – the web browser
22
Benchmarks
• Group #1
– Small web application benchmarks from recent publications
• WebRacer, EventRacer, WAVE, etc.
• Written for illustrating tricky race conditions
– Goal: to confirm that we can correctly classify them
• Group #2
– 70 real websites randomly chosen from Fortune-500 companies
• 1,903 warnings reported by EventRacer
• Most of them are bogus
– Goal: to confirm that we can identify the harmful ones
23
Results (Group #1)
[Mutlu et al., FSE’15]
[Jensen et al., OOPSLA’15]
[FSE’15]
R4 [oopsla’15]
24
Results (Group #2)
[Jensen et al., OOPSLA’15]
R4 [oopsla’15]
25
Results (Group #2)
• EventRacer detected
1903 warnings
• Our method detected
132 harmful races (7%)
• The R4 tool detected
33 harmful races
26
Results (Group #2)
[Raychev et al., OOPSLA’13]
• Compared to EventRacer’s heuristic filtering method
• We identified
– 38 harmful race conditions in EventRacer’s high-risk races
– 40 harmful race conditions in EventRacer’s benign races
– 54 harmful race conditions in EventRacer’s the-rest
27
Example Race Condition (classified as harmful)
28
Conclusions
• The first evidence-based method for classifying racecondition warnings in client-side web applications
– Replay + State comparison
• JavaScript-based platform-agnostic replay framework
– Self-monitoring and control
• Outperformed existing techniques
– On both small benchmarks and real websites
29