Experiences with Client-based
Speculative Remote Display
John R. Lange & Peter A. Dinda
Department of Electrical Engineering and Computer Science
Northwestern University
Sam Rossoff
Department of Computer Science
University of Victoria
Prescience Lab
http://presciencelab.org
http://empathicsystems.org
Speculative Remote Display
• Remote display interactivity fundamentally
constrained by network latency
– Critical for WANs
• Speculation can improve interactivity
– Can be done without server support
• Significant predictability exists in remote display
protocol streams
• Speculation can be introduced into remote
display clients
– User-controlled tradeoff between latency and display
errors
– User studies currently give mixed results for tradeoff
2
Outline
• Speculative Remote Display
• Predictability study of Remote Display
protocol streams
– RDP and VNC
• Design and implementation of a clientbased speculative remote display system
– VNC-SRD
• User study of VNC-SRD
– Examination of user-controlled speculation
• Conclusions
3
Remote Display Systems
• Historically popular in many forms
– VNC, RDP, THINC, Citrix, X
– Thin clients, remote administration, mobile users, etc
• Implemented at all levels of the stack
– Virtual hardware, device drivers, window system
extensions
– Large variance in semantic properties of the protocols
• Lots of work has gone into optimizing Remote
Displays
– Predominantly to decrease bandwidth requirements
4
Latency vs. Bandwidth
Optimization
• Large amount of work in optimizing bandwidth
usage of Remote Displays
– Compression/encoding, caching, protocol design
• Bandwidth is important…
• But so is interactivity
• User interaction implies waiting for server
updates
– Network latency is a critical component
5
Prospects for Speculative Remote Display
User Events
Screen Events
(Windows RDP)
• Can we predict screen events well?
• If so, can we speculatively execute screen
events before server responds?
6
Optimizing Latency Through
Speculation
• Speculation can improve
interactivity
– Display screen events before
they arrive
• RTT is no longer limiting factor
• UI Interaction intuitively
appears to be predictable
– Menus, dialog boxes, etc
– Often tied to specific user action
7
Questions
• Is there predictability?
• How do we design and implement such a tool
–
–
–
–
Client
Client/Server
Predictor
…
• Can users tolerate display errors?
• Can users control tradeoff between display
errors and interactivity?
8
Analysis of Predictability
• Effectiveness of speculation requires
predictable event sequences
• Examined predictability with state-limited
kth order Markov models
– Simple
– General prediction symbols
• Unique event signatures as symbols
– Can provide multi step ahead predictions
9
Trace Collection
• Instrumented versions of rdesktop and VNC
• User studies
– Northwestern students & faculty participants
• 5 users for each protocol
– Single study model
• Full screen display of remote Windows machine
• Private local network
• Standardized task sequence
• Event traces from typical Windows workloads
–
–
–
–
Word, PowerPoint, Web browsing
15 minute periods
VNC: 12K-24K user events, 8.5K-17K screen events
RDP: 47K-77K user events, 712K-1M screen events
10
Analysis of Predictability
• Generated symbols from event sequences
• Ran symbol streams through Markov model
– Model is continuously being updated, just as in
system we later built
– Screen->Screen, Screen+User->Screen
– Different model orders considered
– Different constraints on state space size considered
• Analyzed one-step-ahead predictability
– Percentage of predictions that were correct
• Predictor Accuracy
– Percentage of total events correctly predicted
• Protocol Predictability
11
RDP Prediction Accuracy
Screen+User -> Screen
12
RDP Predictability
Screen+User -> Screen
13
VNC Prediction Accuracy
Screen+User->Screen
14
VNC Predictability
Screen+User->Screen
15
Implications
• RDP displays a high degree of predictability
– Screen events are highly correlated
• RDP results are much better than VNC
– Most likely due to number of messages in an update
– Differences in protocol semantics
– RDP screen event rate much higher than VNC
• Errors are possible
– Users will experience inconsistent screen states
– Implementation must correct mis-predictions
– Can users tolerate this?
16
User Control
• Errors will occur, but…
• Sensitivity to errors varies…
– Between applications
– Between users
• Let user decide tradeoff between aggressive use
of predictions to reduce latency, and increasing
display errors
• Can users make that decision?
– What is the best way to present the control to users?
17
Why not RDP/SRD?
• Implementation simplicity
– VNC has much less client state
– VNC has fewer event types
– VNC execution is straightforward
• If it works with VNC then it surely works
with RDP
18
VNC-SRD
• Implementation of SRD inside VNC client
– ~2300 LOC (C++) added to VNC client
– ~850 LOC (Perl) Markov predictor
implementation
• User-driven prediction application policy
– Allows user to adjust amount of speculation
• Generic predictor interface
– Event signature-based predictions
• Error correction and rollback
19
20
Prediction Application Policy
• System predicts event signatures
– Constructs events from signatures
• Events directed to one of two queues
– Applied Queue
• Applied events currently displayed to the user
• Held until validated by arrival of server events
– Pending Queue
• Predictions waiting to be displayed
• Events applied to screen at a given rate
– Rate: predicted server event rate
– Rate * (desired latency reduction) = # of steps ahead21
User Control
• Users/workloads have varying
error tolerance
– Allow direct user control
• VNC-SRD adds a user adjustable
scale to the interface
– User specifies desired network
latency reduction
22
Error Handling
• Speculation can result in wrong display
– System must determine and correct errors
• Correction
– VNC-SRD fetches entire modified region
• All applied predictions are rolled back
• Local rollback is straightforward
• Validation
– VNC-SRD validates prediction against next event
• Error on mismatch
– Conservative approach
• Out of order predictions may be displayed correctly
• All errors have equal weight
23
User Study
• Students from Northwestern and University of
Victoria
– 16 total users
• 4 network scenarios
– Varied latency and drop rate
• 300ms, 100ms, 50ms, 10ms + 10% drop rate
– 3 randomly presented to each user
• 3 Application workloads
– Word, PowerPoint, and Web Browsing
– Performed in each of the network scenarios
– 5 minute runs
• Predictor state cleared for each run
24
Purpose of Study
• Determine whether users can tolerate
screen artifacts due to speculation
• Determine whether users can trade off
using our interface
25
Study Questionnaire
• Users rated the system after each
scenario
– Ability to find a comfortable predictor setting
– Responsiveness at the most comfortable
setting
– Acceptability of display errors at the most
comfortable setting
– Scale: 1 (very bad) to 10 (very good)
26
User Study Caveats
• Few predictions were made
– Study runs not long enough to sufficiently
seed predictor
– VNC has low predictability
• Ratings should thus be taken with a large
grain of salt
• Future work planned to improve this
27
Able to find a comfortable setting?
28
Are Display Errors Acceptable?
29
Is the System Responsive?
30
Results
• We still do not know for sure whether
users…
– Can tolerate the screen artifacts
– Can trade off between screen artifacts and
latency
• More work needed
31
Alternative Design Approaches
• Speculative support on the server
– Synchronized predictors
• Predictions used for other purposes
– Cache hints
– Update regions
– Update ordering
• Other prediction algorithms/symbols
– Time series predictions of event coordinates
32
Future Work
• Exploration of predictability in Remote
Display protocols
– How far ahead can predictions be made?
– What prediction algorithm should be used?
– To what degree can SRD decrease latency?
– Comparison of more protocols
• Exploration of different SRD designs
33
Conclusions
• Remote display interactivity fundamentally
constrained by network latency
– Critical for WANs
• Speculation can improve interactivity
– Can be done without server support
• Significant predictability exists in remote display
protocol streams
• Speculation can be introduced into remote
display clients
– User-controlled tradeoff between latency and display
errors
– User studies currently give mixed results for tradeoff 34
• Prescience Lab
– http://presciencelab.org
• Empathic Systems Project
– http://empathicsystems.org
• John Lange
– http://www.artifex.org/~jarusl
35