Cognitive Issues in Virtual
Reality
Wickens, C.D., and Baker, P., Cognitive issues in
virtual environments, in Virtual Environments and
Advanced Interface Design, Barfield and Furness,
pp. 514-541.
Summarized by Geb Thomas
The Main (Cognitive ) Features
of VR
•
•
•
•
•
3D viewing vs. 2D viewing
Dynamic vs. static displays
Closed-loop vs. open-loop interaction
Inside-out vs. outside in frame of reference
Multimodal interaction
A Comparison
Less Real
More Real
Dimensionality 2D
3D
Motion
Static
Dynamic
Interaction
Open loop
Closed loop
Frame of Ref.
Outside-in
Inside-out
Multi-modal
Limited
Multi-modal
Uses of Virtual Reality
• What cognitive issues lie behind each
application
• How do these play into the user’s perceptual
strengths and weaknesses
On-Line Performance
• Direct manipulation capabilities in a remote
environment
• Principle issues:
– Closed-loop perceptual motor performance
– Situation awareness
– Low workload and cognitive effort
Off-line Training and Rehearsal
•
•
•
•
Lumbar injection
Maneuvering a space craft
Rehearsing a dangerous mission
Key consideration:
– Effective transfer of training
On-line comprehension
• Reaching understanding, comprehension or
insight
• Scientist interacting with a database
• Key:
– perceiving relations
– perceiving constraints
– perceiving constancies
Off-line Knowledge Acquisition
• Useful for education
• Key issue:
– knowledge transfer
How to Cognitively Engineer VR
• Select features that aid and do not disrupt
cognitive needs
– closed-loop performance may not help on-line
performance but is key to understanding
• Different parts of the visual system are
involved in perceptual-motor coordination
and navigation than are involved with
perceptual understanding of spatial location
– Ambient vs. focal vision
The Operator Brings
•
•
•
•
•
Wide sensory bandwidth
Limited perceptual bandwidth
Constraints on attention
Constraints on working memory
High level of natural perceptual-motor
coordination
• Large repertoire of facts and knowledge
Search
• Find an object of interest
• Object of the search may be concrete
instance or an abstraction of the rendering
in VE
• A map often facilitates searching
– Minimize map clutter
– Flexible frame of reference
– Tie or link map and VE
Navigation
• Challenging because of the removal of
constraints
• Speed and flexibility can cause loss of
situational awareness
• May help to partially automate movement,
such as logarithmically control speed
• Metaphors matter
– flight in mazes
– “in-hand” for objects
Perceptual Biases
• Gibson and ecological perception, regularly
spaced texture, level surfaces for gradients,
slant and optical flow.
• Ellis, McGreevy et al.
– Virtual space effect because of minification or
magnification from FOV
– 2D-3D effect perceived rotation of vectors
towards viewing plane
– Display enhancement
Visual Motor Coupling and
Manipulation
•
•
•
•
•
Gain
Time delay
Control order
Target view decoupling
Field of view
Field of View
• Wider provides greater situational
awareness
• Wider distorts perceived position
• Wider provide better sense of motion
• Wider can promote motion sickness
Perception and Inspection
• For navigation, spatial relations
predominate
• For inspection, light, shadow, motion
parallax
• Schematic figures for learning
Learning
• Procedural learning
– Lower realism
• Perceptual motor skill learning
– Active participation in control loop is important
– Do dynamic simulation help?
• Spatial Learning and Navigation Rehearsal
– rotating frame can inhibit map building
– Active control loop is not always a benefit
– Head-mounted displays may be a hindrance
• Conceptual learning
– Multi-modal, active
HF Guidelines in Learning
• Consistency
• Redundancy
• Visual Momentum
– Consistent representations
– Graceful transitions
– Highlight anchors