Interactive Programs
• Games are interactive systems - they must respond
to the user
• Today is all about how interactive programs are
designed
• Also, some issues to think about in interactive
systems
CS 638, Fall 2001
Interactive Program Structure
• Event driven programming
Initialize
– Everything happens in
response to an event
User Does Something
or
Timer Goes Off
System Updates
• Events come from two
sources:
– The user
– The system
• Events are also called
messages
CS 638, Fall 2001
– An event causes a message
to be sent…
User Events
• The OS manages user input
– Interrupts at the hardware level …
– Get converted into events in queues at the windowing
level …
– Are made available to your program
• It is generally up to the application to make use of
the event stream
• Windowing systems may abstract the events for
you
CS 638, Fall 2001
System Events
• Windowing systems provide timer events
– The application requests an event at a future time
– The system will provide an event sometime around the
requested time. Semantics vary:
• Guaranteed to come before the requested time
• As soon as possible after
• Almost never right on (real-time OS?)
• Application is not interrupted - it has to look for the
event
– Exception: Alarm signals in UNIX
CS 638, Fall 2001
Polling for Events
while ( true )
if ( e = checkEvent() )
switch ( e.type )
…
do more work
• Most windowing systems provide a non-blocking event
function
– Does not wait for an event, just returns NULL if one is not ready
• What type of games might use this structure?
• Why wouldn’t you always use it?
CS 638, Fall 2001
Waiting for Events
e = nextEvent();
switch ( e.type )
…
• Most windowing systems provide a blocking event function
– Waits (blocks) until an event is available
– Usually used with timer events. Why?
• On what systems is this better than the previous method?
• What types of games is it useful for?
CS 638, Fall 2001
The Callback Abstraction
• A common event abstraction is the callback mechanism
• Applications register functions they wish to have called in
response to particular events
– Translation table says which callbacks go with which events
• Generally found in GUI (graphical user interface) toolkits
– “When the button is pressed, invoke the callback”
– FLTK works this way
– Many systems mix methods, or have a catch-all callback for
unclaimed events
– In X windows, Xlib defines events, Xtoolkit defines callbacks
• Why are callbacks good? Why are they bad?
CS 638, Fall 2001
LithTech Messaging
• LithTech allows any of the methods we have
discussed
– Callbacks for the system timer
• the update function called once per frame
– Callbacks bound to multiple events
• Same callback gets many events and internally decides what to
do with each one
• Bindings done with a configuration file
– Polling for keyboard input. What sort of input?
• Read the Programming Guide chapter on Input to
get an overview
CS 638, Fall 2001
Upon Receiving an Event …
• Event responses fall into two classes:
– Task events: The event sparks a specific task or results in some
change of state within the current mode
• eg Load, Save, Pick up a weapon, turn on the lights, …
• Call a function to do the job
– Mode switches: The event causes the game to shift to some other
mode of operation
• eg Start game, quit, go to menu, …
• Switch event loops, because events now have different meanings
• Software structure reflects this - menu system is separate
from run-time game system, for example
CS 638, Fall 2001
Real-Time Loop
• At the core of games with animation is a real-time loop:
while ( true )
process events
update animation
render
• What else might you need to do?
• The number of times this loop executes per second is the
frame rate
– # frames per second (fps)
CS 638, Fall 2001
Lag
• Lag is the time between when a user does something and
when they see the result - also called latency
– Too much lag and causality is distorted
– With tight visual/motion coupling, too much lag makes people
motion sick
• Big problem with head-mounted displays for virtual reality
– Too much lag makes it hard to target objects (and track them, and do
all sorts of other perceptual tasks)
• High variance in lag also makes interaction difficult
– Users can adjust to constant lag, but not variable lag
• From a psychological perspective, lag is the important
variable
CS 638, Fall 2001
Computing Lag
Process input
Frame
time
Update state
Render
Process input
Update state
Render
Process input
• Lag is NOT the time it
Event
takes to compute 1 frame!
time
• What is the formula for
maximum lag as a
Lag
function of frame rate, fr?
• What is the formula for
average lag?
2
lag max 
fr
1 .5
lag average 
fr
CS 638, Fall 2001
Frame Rate Questions
• What is an acceptable frame rate for twitch games?
Why?
• What is the maximum useful frame rate? Why?
• What is the frame rate for NTSC television?
• What is the minimum frame rate required for a
sense of presence? How do we know?
• How can we manipulate the frame rate?
CS 638, Fall 2001
Frame Rate Answers (I)
• Twitch games demand at least 30fs, but the higher
the better (lower lag)
– Users see enemy’s motions sooner
– Higher frame rates make targeting easier
• The maximum useful frame rate is the monitor
refresh rate
– Time taken for the monitor to draw one screen
– Synchronization issues
• Buffer swap in graphics is timed with vertical sweep, so ideal
frame rate is monitor refresh rate
• Can turn of synchronization, but get nasty artifacts on screen
CS 638, Fall 2001
Frame Rate Answers (II)
• NTSC television draws all the odd lines of the screen, then
all the even ones (interlace format)
– Full screen takes 1/30th of a second
– Use 60fps to improve visuals, but only half of each frame actually
gets drawn by the screen
– Do consoles only render 1/2 screen each time?
• It was once argued that 10fps was required for a sense of
presence (being there)
– Head mounted displays require 20fps or higher to avoid illness
– Many factors influence the sense of presence
– Perceptual studies indicate what frame rates are acceptable
CS 638, Fall 2001
Reducing Lag
• Faster algorithms and hardware is the obvious answer
• Designers choose a frame rate and put as much into the game
as they can without going below the threshold
– Part of design documents presented to the publisher
– Threshold assumes fastest hardware and all game features turned on
– Options given to players to reduce game features and improve their
frame rate
• There is a resource budget: How much of the loop is dedicated
to each aspect of the game (graphics, AI, sound, …)
• Some other techniques allow for more features and less lag
CS 638, Fall 2001
Decoupling Computation
• It is most important to minimize lag between the user
actions and their direct consequences
– So the input/rendering loop must have low latency
• Lag between actions and other consequences may be less
severe
– Time between input and the reaction of enemy can be greater
– Time to switch animations can be greater
• Technique: Update different parts of the game at different
rates, which requires decoupling them
– For example, run graphics at 60fps, AI at 10fps
– Done in Unreal engine, for instance
CS 638, Fall 2001
Animation and Sound
• Animation and sound need not be changed at high
frequency, but they must be updated at high frequency
– For example, switching from walk to run can happen at low
frequency, but joint angles for walking must be updated at every
frame
• Solution is to package multiple frames of animation and
submit them all at once to the renderer
– Good idea anyway, makes animation independent of frame rate
• Sound is offloaded to the sound card
CS 638, Fall 2001
Graphics Review
• Recall the standard graphics pipeline:
CS 638, Fall 2001
Normal Vectors
• The intensity of a surface depends on its orientation
with respect to the light and the viewer
– CDs are an extreme example
• The surface normal vector describes the orientation
of the surface at a point
– Mathematically: Vector that is perpendicular to the
tangent plane of the surface
• What’s the problem with this definition?
– Just “the normal vector” or “the normal”
– Will use N to denote
CS 638, Fall 2001
Local Shading Models
• Local shading models provide a way to determine
the intensity and color of a point on a surface
– The models are local because they don’t consider other
objects at all
– We use them because they are fast and simple to
compute
– They do not require knowledge of the entire scene, only
the current piece of surface
CS 638, Fall 2001
Local Shading Models (Watt 6.2)
• What they capture:
– Direct illumination from light sources
– Diffuse and Specular components
– (Very) Approximate effects of global
lighting
• What they don’t do:
–
–
–
–
Shadows
Mirrors
Refraction
Lots of other stuff …
CS 638, Fall 2001
“Standard” Lighting Model
• Consists of three terms linearly
combined:
– Diffuse component for the amount
of incoming light reflected equally
in all directions
– Specular component for the amount
of light reflected in a mirror-like
fashion
– Ambient term to approximate light
arriving via other surfaces
CS 638, Fall 2001
Diffuse Illumination
kd I i ( L  N )
• Incoming light, Ii, from direction L, is reflected equally in
all directions
– No dependence on viewing direction
• Amount of light reflected depends on:
– Angle of surface with respect to light source
• Actually, determines how much light is collected by the surface, to
then be reflected
– Diffuse reflectance coefficient of the surface, kd
• Don’t want to illuminate back side. Use kd I i max( L  N ,0)
CS 638, Fall 2001
Diffuse Example
Where is the light?
CS 638, Fall 2001
Specular Reflection (Phong
Model)
L
V
ks I i ( R  V )
n
• Incoming light is reflected primarily in the mirror
direction, R
– Perceived intensity depends on the relationship between the
viewing direction, V, and the mirror direction
– Bright spot is called a specularity
• Intensity controlled by:
– The specular reflectance coefficient, ks
– The parameter, n, controls the apparent size of the specularity
• Higher n, smaller highlight
CS 638, Fall 2001
R
Specular Example
CS 638, Fall 2001
Putting It Together
I  ka I a  I i kd ( L  N )  k s ( H  N )
n

• Global ambient intensity, Ia:
– Gross approximation to light bouncing around of all other surfaces
– Modulated by ambient reflectance ka
• Just sum all the terms
• If there are multiple lights, sum contributions from each
light
• Several variations, and approximations …
CS 638, Fall 2001
Flat shading
• Compute shading at a
representative point and apply to
whole polygon
– OpenGL uses one of the vertices
• Advantages:
– Fast - one shading value per
polygon
• Disadvantages:
– Inaccurate
– Discontinuities at polygon
boundaries
CS 638, Fall 2001
Gourand Shading
• Shade each vertex with it’s own
location and normal
• Linearly interpolate across the
face
• Advantages:
– Fast - incremental calculations
when rasterizing
– Much smoother - use one normal
per shared vertex to get continuity
between faces
• Disadvantages:
– Specularities get lost
CS 638, Fall 2001
Phong Interpolation
• Interpolate normals across faces
• Shade each pixel
• Advantages:
– High quality, narrow specularities
• Disadvantages:
– Expensive
– Still an approximation for most
surfaces
• Not to be confused with Phong’s
specularity model
CS 638, Fall 2001
CS 638, Fall 2001