Why Events Are a Bad Idea
(for high concurrency servers)
CS533
Winter 2007
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
The Stage
Highly concurrent applications
What makes concurrency
hard?
Race conditions
Scalability (no O(n) operations)
Overload -Scheduling &
resource contention
Ideal
Performance
Internet servers (Flash, Ninja,
SEDA)
Operate “near the knee”
Avoid thrashing!
Peak: some
resource at max
Overload: some
resource thrashing
Load (concurrent tasks)
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
The Debate
Performance vs.
Programmability
Questions
Threads vs. Events?
Ideal
Ease of Programming
Current threads pick one
Events somewhat better
Current
Threads
Current Events
Current Threads
Performance
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
Our Position
Thread-event duality still holds
But threads are better anyway
More natural abstraction
Better fit with tools and hardware
Compiler-runtime integration is key
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
“Why Did we use Events”
Recent arguments for events
Performance
Better live state management
Inexpensive synchronization
Better scheduling and locality
All true but…
No inherent problem with threads!
Thread implementations can be improved
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
Runtime Overhead
Criticism: Threads don’t
perform well for high
concurrency
Avoid O(n) operations
Minimize context switch
overhead
Simple scalability test
100000
Requests / Second
Response
110000
90000
80000
70000
Threaded Server
60000
Event-Based Server
50000
40000
30000
20000
1
10
100
1000
10000
Concurrent Tasks
100000
Slightly modified GNU Pth
Thread-per-task vs.
single thread
Same performance!
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
1e+06
Control Flow
Criticism: Threads have
restricted control flow
Response
Programmers use simple patterns
Call / return
Parallel calls
Pipelines
Complicated patterns are
unnatural
Hard to understand
Likely to cause bugs
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
Synchronization
Criticism: Thread synchronization is
heavyweight
Response
Basically non-preemption helps in events
Other factors
Starvation & fairness
Multiprocessors
Compiler support helps
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
Live State Management
Criticism: Stacks are bad for live state
Response
Stack overflow vs. wasted space
Compiler Support
Dynamically link stack frames
Events based system requires effort by
programmer
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
Scheduling
Criticism: Thread schedulers are too
generic
Can’t use application-specific information
Response
Use task & program location for
scheduling
Threads can do that too!
Lauder Needham duality paper
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
The Proof
User-level threads package
Simple web server: Knot
700 lines of C code
Similar performance
Linear increase, then steady
Drop-off due to poll()
overhead
900
800
KnotC (Favor Connections)
KnotA (Favor Accept)
Haboob
700
Mbits / second
Intercept blocking system
calls
No O(n) operations
Support > 100K threads
600
500
400
300
200
100
0
1
4
16
64
256
1024
4096
16384
Concurrent Clients
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
Threads Also Have...
More natural programming model
Control flow is more apparent
Exception handling is easier
State management is automatic
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
The Future:
Compiler-Runtime Integration
Specific targets
Dynamic stack growth
Compiler analysis for amount of stack space needed for a
function call
Live state management
Pop temporary variables
Reorder overlapping variables
Synchronization
Compile time analysis for data races
Challenging!!
Implemented by TinyOS
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt
Conclusion
Performance
Ease of use
Compiler-runtime integration is key
reference:http://capriccio.cs.berkeley.edu/pubs/threads_hotos_2003_slides.ppt