ENERGETIC VS COGNITIVE RESOURCES IN SEARCH
1
Balancing Energetic and Cognitive
Resources: Memory Use During
Search Depends on the Orienting
Effector
Grayden J. F. Solman & Alan Kingstone
University of British Columbia, Vancouver, BC, Canada
Correspondence to: Grayden Solman, Department of Psychology, University of British Columbia, Vancouver,
BC, V6T 1Z4, Canada. E-mail: [email protected]
This is an author generated postprint of the article:
Solman, G. J. F., & Kingstone, A. (2014). Balancing energetic and cognitive resources: Memory use during
search depends on the orienting effector. Cognition, 132, 443-454.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International License.
This article may not exactly replicate the final published version. It is not the copy of record.
The publisher’s version is available at: http://dx.doi.org/10.1016/j.cognition.2014.05.005
Abstract
Search outside the laboratory involves tradeoffs among a variety of internal and external
exploratory processes. Here we examine the conditions under which item specific memory from
prior exposures to a search array is used to guide attention during search. We extend the
hypothesis that memory use increases as perceptual search becomes more difficult by turning to
an ecologically important type of search difficulty – energetic cost. Using optical motion tracking,
we introduce a novel head-contingent display system, which enables the direct comparison of
search using head movements and search using eye movements. Consistent with the increased
energetic cost of turning the head to orient attention, we discover greater use of memory in headcontingent versus eye-contingent search, as reflected in both timing and orienting metrics. Our
results extend theories of memory use in search to encompass embodied factors, and highlight the
importance of accounting for the costs and constraints of the specific motor groups used in a
given task when evaluating cognitive effects.
Keywords: search, memory, embodied cognition
POST-PRINT
doi:10.1016/j.cognition.2014.05.005
2
SOLMAN & KINGSTONE
Introduction
Increasingly often, the distinction is made between
Visual Search as a particular cognitive paradigm
(Treisman & Gelade, 1980), and human search
behavior more generally (Hollingworth, 2012). Visual
search has been the subject of considerable research,
and is in many ways quite well understood and
successfully modeled (e.g., Itti & Koch, 2000; Wolfe,
1994; 2007). In contrast, the field of research
examining general search behavior remains nebulous,
faced with the familiar challenges of embodied
cognition – to understand the ongoing interplay
between internal cognitive processing and the
constraints and affordances of the material environment
(Clark, 1999; Glenberg, 2010; Wilson, 2002). As the
scope of search is extended beyond the initial moments
of sensory processing, there is reason to believe that the
relative contributions of and tradeoffs between internal
processing (e.g., memory and prediction) and external
exploration (e.g., attentional shifting, eye-movements,
body movements) may shift from what is observed in
more ‘dis-embodied’ laboratory tasks. For instance,
Gilchrist, North, and Hood (2001) had participants
perform an embodied foraging task, where search items
were embedded in film canisters distributed throughout
a room – so that searchers had to walk through the
space to inspect items. Compared to purely oculomotor
search, they found a reduced rate of item revisits,
suggesting an increased role for memory of the
locations inspected – though it could not be determined
whether this reflected memory for specific items, or a
consequence of more systematic search paths.
Changes across tasks in the relative weighting of
internal and external processing has been proposed to
reflect an optimizing principle, so that the relative costs
of cognition as compared to orienting and sensation
will determine the proportionate reliance on these
modes for a given task. For instance, when participants
are copying patterns of blocks from a model, they
routinely check the model twice for each block – once
to determine which colour to acquire, then again after a
suitable block has been found to determine where it
should be placed in the copy (Ballard, Hayhoe, & Pelz,
1995; Ballard, Hayhoe, Pook, & Rao, 1997). This
pattern indicates a preference to use knowledge ‘in-theworld’ (i.e., the model) over knowledge ‘in-the-head’
(i.e., working memory), as the latter case predicts at
most one model-check per block, to determine both the
necessary colour and position. Studies of change
POST-PRINT
detection during a block-sorting task in virtual reality
have also been used to determine the extent and
timecourse of working memory use during sorting
(Droll, Hayhoe, Triesch, & Sullivan, 2005). In this
context, participants were found to store a block’s
characteristics in working memory when it was picked
up, but routinely failed to notice changes to these
features prior to the sorting decision – typically sorting
instead on the basis of the pre-change feature.
Subsequent work indicated that task predictability and
memory load played a crucial role in determining
whether participants relied on working memory or a
‘just-in-time’ strategy whereby item features were
queried from the environment only just before they
were needed (Droll & Hayhoe, 2007). The authors
concluded that the data reflect “some kind of
optimization or trade-off with respect to a set of
constraints on the part of the observer,” though what
specifically was being optimized remained unclear.
Several findings implicate a role for the effort
associated with using external information stores in
determining this trade-off. In the block-copying task
described above, when the model was placed farther
away from the participant, necessitating larger
orienting movements, the amount of model checking
was reduced (Ballard, Hayhoe, & Pelz, 1995).
Similarly, when solving arithmetic problems, the
amount of note taking during intermediate steps is
influenced by the availability of the note taking
apparatus (Cary & Carlson, 2001). In a study where
participants prepared to write a report from text-based
sources, the likelihood of printing out a page of
information was reduced when the printing process was
made more complicated and time-consuming
(Schönpflug, 1986). Finally, in a computer based
puzzle-solving task, participants relied more on
planning and less on exploratory manipulations of the
puzzle when those manipulations required the input of
lengthy commands (O’Hara & Payne, 1998). All of
these results are consistent with the notion that
increasing the effort needed to acquire information ‘inthe-world’ promotes an increased reliance on internal
cognitive processes.
One concrete proposal, the ‘soft constraints
hypothesis,’ holds that it is solely temporal costs that
are being minimized, via behavioral selections
occurring every 500 – 1,000ms (Gray & Fu, 2004;
Gray, Sims, Fu, & Schoelles, 2006), adopting the
commendably explicit position that “milliseconds
matter and they matter the same regardless of the type
doi:10.1016/j.cognition.2014.05.005
ENERGETIC VS COGNITIVE RESOURCES IN SEARCH
of activity with which they are filled”. There is reason,
however, to doubt the exclusivity of temporal costs. In
particular, a time-only model fails to account for the
kinematics of the perceptuo-motor actions involved in
using knowledge ‘in-the-world’. In contrast, a large
body of work in the domain of motor control has
evaluated the optimization criteria for movement, and
has routinely implicated the importance of minimizing
such factors as ‘jerk’ (acceleration transients; Hogan,
1984), torque change (Uno, Kawato, & Suzuki, 1989),
and metabolic energy expenditure (Sparrow & Newell,
1998). Composite utility models including kinematic
factors as well as time generally show a ‘knee’ region,
such that a reasonable tradeoff between time and
kinematic cost exists up to a point (the ‘knee’), but as
time is further reduced to its minimum, kinematic costs
rapidly increase (Nelson, 1983). Indeed, even in the
context of traditional orienting behaviors, recent work
successfully reproducing eye movement dynamics with
a model using a weighted tradeoff between timeoptimal and minimum energy criteria, found that as
movement size increased, the contribution of timeoptimality became negligible (Wang & Hsiang, 2011).
Since naturalistic tasks typically involve a full suite of
effectors with varying kinematic properties, the results
summarized above suggest that energetic cost may
often have an equal or larger influence than time
expenditure in the control of naturalistic behavior.
Search behavior provides a valuable context for
investigating internal-external resource tradeoffs, as the
information being sought in a search task is the
location of a target, a qualitative distinction from task
contexts that involve referencing information from a
constant known position (c.f. ‘perceptuo-motor search’
vs. ‘perceptuo-motor access’; Gray & Fu, 2004).
Conceptually, it is akin to the difference between
checking the time by looking at your watch and
checking the time by finding a clock in an unfamiliar
room. In the former case, relying on the external
environment has an effectively constant orienting cost
(i.e., aligning your eyes to your wrist). In the case of
search, however, the energetic effort associated with
relying on knowledge ‘in-the-world’ (i.e., searching
anew each time) will increase with both the physical
scale of the search space and the number of objects it
contains.
To date, existing studies of repeated search through
identical or nearly-identical displays have examined
only relatively low-cost forms of orienting (i.e., gazefixed, or eye-movements only; Kunar, Flusberg, &
POST-PRINT
3
Wolfe, 2008; Oliva, Wolfe, & Arsenio, 2004; Solman
& Smilek, 2010; Võ & Wolfe, 2012; Wolfe, Klempen,
& Dahlen, 2000). These studies have found that
memory for specific item locations provides a
relatively weak source of guidance in typical search
contexts, although increased memory benefits have
been found both for more eccentric targets and for
search
requiring
more
difficult
perceptual
discrimination (Solman & Smilek, 2012). In contrast,
orienting during naturalistic search involves not just
eye-movements, but also head- and trunk-movements,
manipulation of the environment, and movement of the
body through space (Foulsham, Walker, & Kingstone,
2011). If internal processing and external exploration
trade off on the basis of relative energetic cost, the
differential recruitment of effectors in different search
contexts may have important consequences for the use
of memory, so that search necessitating the use of a
costly effector like the head should increase the
propensity to use memory when compared to search
needing only much cheaper eye-movements.
The present research explores the degree to which
minimization of energetic cost might explain the
selection of cognitive as compared to exploratory
strategies during search. In particular, we examine the
degree to which searchers use memory for past target
locations as opposed to searching anew on each trial,
and compare this balance for search relying on either
the eyes or the head, using a gaze-contingent
methodology which tracks either the eye position or the
head position. Given the infrequency of considering
head movements during search, however, we first
provide a brief summary of head movements in the
context of orienting, and of the typical coordination
between eye and head movements.
In cognitive science, orienting behavior is most
often thought of in terms of covert attentional shifts
and eye movements. Naturalistic orienting, however,
routinely involves the recruitment of multiple nested
systems of effectors, for instance with trunk position
constraining head position, which then constrains eye
position (Land, 2004). Behaviorally, the classic portrait
of coordination between head and eye movements
when shifting gaze to a visual target consists of a
roughly synchronous movement onset (though the eye
generally leads by a few tens of milliseconds), with the
initial eye saccade terminating well in advance of the
slower head movement – owing to longer contraction
times for neck muscles, as well as greater inertial
forces acting on the head as compared to the eye (Bizzi,
doi:10.1016/j.cognition.2014.05.005
4
SOLMAN & KINGSTONE
Kalil, Tagliasco, 1971; Winters & Stark, 1985).
Consequently, as the neck movement completes, the
eye must compensate by counter-rotating in order to
maintain fixation on the target. This compensatory
movement was believed to emerge relatively
automatically in response to vestibular signals resulting
from the head movement, via the vestibulo-ocular
reflex (Barnes, 1979; Morasso, Bizzi, & Dichgans,
1973). However, both early and later studies have
noted departures from this stereotypical pattern of
coordination in a range of contexts (e.g., Bizzi, Kalil,
Tagliasco, 1971; Goossens, & Van Opstal, 1997; Lee,
1999; Sparks, 1991; see Freedman, 2008 for a review).
In addition, for a given gaze shift amplitude, there
appears to be considerable individual variability in the
extent to which head movements are recruited (Fuller,
1992). It is clear then, that what we might consider the
‘default’ mode of coordination emerging from
brainstem circuits is subject to considerable modulation
from higher order inputs ranging from cortex to the
basal ganglia (Isa & Sasaki, 2002), a point
foreshadowed by the observation that superior
colliculus (SC) stimulation in decerebrated cats lead to
increased rates of coordinated motor activity as
compared to stimulation of SC in intact animals,
implicating a prominent role for cortical afferents in
contextually regulating the extent to which eye and
head movements are coordinated (Faulkner & Hyde,
1958).
The present research tests the specific hypothesis
that, controlling for perceptual factors, searchers using
head movements will show a greater propensity for
memory use during repeated search than will searchers
using only eye movements. Using a motion tracking
system, we devised a head-contingent display method
that enabled us to directly compare performance in
head-contingent search and eye-contingent search (see
Figure 1). We predict that the need to recruit the large
costly muscle groups of the neck to orient the head
should lead to greater use of memory to offset this cost,
compared to the much smaller and relatively cheap
oculomotor movements needed for eye-contingent
search. Notably, we highlight the distinction between
memory use (i.e., making an attempt to recall the
location of an item) and memory success (i.e., correctly
recalling the location of an item), and although overall
response time effects are likely to be dominated by the
rate of success, the present hypothesis concerns
memory use. Importantly, memory success is bounded
both by memory use and by memory capacity, so that
POST-PRINT
set size effects in response times may reflect an
interaction between use and capacity. Fortunately, the
distinction between use and success can be better
resolved by decomposing response times to isolate preand post-search effects, and by evaluating measures of
early orienting accuracy, as described below.
We evaluate memory use in terms of basic response
measures (response times and search slopes), as well as
additional performance metrics derived from head and
eye position trajectories. First, we localize performance
differences in RTs by subdividing total response time
into three non-overlapping components: 1) time to
initiate search, 2) time to respond after reaching the
target, and 3) the intervening ‘actual search’ time
(Solman, Cheyne, & Smilek, 2011). We suggest that
memory use should be reflected primarily in initiation
times, indexing the time spent querying memory prior
to the first orienting movement. Memory success
(though necessarily bounded by use) may emerge in
decision times, reflecting expectancy or readiness to
respond to the target in its proper location, or in the
intervening search times if memory influences the
process of search itself. In addition to response time
measures, we also examine the angular accuracy of
initial trajectories, using maximum likelihood
estimation (MLE) to estimate the relative contributions
of orienting based on successful memory and random
orienting during repeated search (adapting Zhang &
Luck, 2008). The expected dependency of memory
success on memory use is evaluated by testing for a
positive correlation between initiation time effects and
memory success estimates.
Method
Participants. Sixty undergraduate students (36
female, 24 male) from the University of British
Columbia participated for course credit or
remuneration. All participants reported normal or
corrected-to-normal visual acuity. Informed consent
was obtained from all participants, and all experimental
procedures and protocols were reviewed and approved
by the University of British Columbia Behavioral
Research Ethics Board.
Displays. Display parameters were matched for both
Eye- and Head-contingent conditions, so that stimuli
were visually identical across effectors. Each trial
included viewer-contingent fixation, target, and search
displays. Fixation displays consisted of a single black
dot centered on a dark grey background. Target
displays showed the target item for the trial, subtending
doi:10.1016/j.cognition.2014.05.005
ENERGETIC VS COGNITIVE RESOURCES IN SEARCH
5
Figure 1. Illustration of the apparatus used to extract gaze-position and update the gaze-contingent window during search (A,
B). For eye-contingent search (A), participants’ eyes were tracked using a desktop mounted eye-tracker with head position
fixed. For head-contingent search (B), participants’ heads were tracked using an optical motion capture system. (C) Example
display sequence for a search at set size 12 (colors adjusted for clarity of representation).
1.8 degrees of visual angle (d.v.a.), in a bright green
box subtending 2.6 d.v.a., centered on the screen.
Search displays consisted of either 12 or 24 items
drawn from the set of 24 capital letters, excluding ‘M’
and ‘W’ (for both width and similarity concerns, and to
reduce the set to 24 items total), displayed in black on a
light grey background, with each item subtending
approximately 0.6 d.v.a. This display was occluded by
a dark grey screen with black gaussian patches at each
of the item locations. A circular gaze-contingent
window with a radius of 3.0 d.v.a. tracked either the
participant’s eye or head (depending on condition),
displaying the identifiable search items within the
window while the remainder of the screen displayed
only the gaussian place-holders. The central target
template remained visible at all times. Items were
placed in a jittered rectangular grid, excluding the
POST-PRINT
central 10.5 d.v.a., and target positions were evenly
distributed in the display.
Procedure. Each trial began with a fixation display.
Once participants brought the gaze-contingent window
within 1.5 d.v.a (one half of the window’s radius) of
the central dot, the target display was triggered. The
target alone was presented centrally for 500 ms,
followed by the search display (with the target still
present). Participants were instructed to locate the
target item, and to press the SPACE bar on a keyboard
while the item was visible within the gaze contingent
window. A new trial was initiated immediately
following each response (see Figure 1C).
Participants were randomly assigned to either the
head-contingent or the eye-contingent condition.
During head-contingent search, no specific instructions
doi:10.1016/j.cognition.2014.05.005
6
SOLMAN & KINGSTONE
were given regarding eye movements. Each participant
completed four blocks of 60 trials, one block for each
Set Size for both Repeated and Random searches.
Blocks were counterbalanced first by the Repeated vs.
Random factor, and then by Set Size, with the order of
Set Size preserved for both Repeated and Random
searches for a given individual. Within each block,
each of the twelve possible targets was used an equal
number of times (i.e., five repetitions of each target).
For Set Size 12, all items were used as possible targets,
and for Set Size 24 a random subset of twelve of the
items was used. For Repeated search, the configuration
of items in all search displays at a given Set Size were
identical. For Random search, the configuration of
items was independent from trial to trial.
Tracking methods. For both effector conditions, we
(1) performed temporal smoothing of displayed
positions to reduce jitter in the window position, and
(2) item positions were ‘magnetized’ such that when
the raw gaze coordinate fell 4.5 d.v.a. or closer to an
item, the displayed position was corrected toward the
item’s position by an amount inversely proportional to
the distance (i.e., the farther from the item, the smaller
the correction). Since the window size and spacing of
items made it impossible to view two items
simultaneously, this ‘magnetization’ had no influence
on transitioning between items, but it, and the temporal
smoothing, greatly improved the perceived quality and
stability of individual inspections. Both raw and
displayed (i.e., transformed as above) coordinates were
recorded at a rate of 60 Hz.
Apparatus – Eye contingent. The experiment was
created in MATLAB, using version 3 of the
Psychophysics Toolbox (Brainard, 1997; Pelli, 1997)
and the Eyelink Toolbox (Cornelissen, Peters, &
Palmer, 2002), and run on an SR Research supplied
host PC with a 2.67GHz Intel Core2 Quad CPU. The
stimulus displays were presented on a 24” Dell
P2411Hb monitor at a resolution of 1920 by 1080 and a
distance of 55cm. Eye movements were recorded
throughout the task using a desktop-mounted Eyelink
1000 system (SR Research), with participants’ heads
stabilized by a chin and forehead rest (Figure 1A).
Calibration was performed prior to each block.
Apparatus – Head contingent. The experiment was
created in MATLAB, using version 3 of the
Psychophysics Toolbox (Brainard, 1997; Pelli, 1997),
and run on a Dell Precision T3500 computer with a
3.07GHz Intel Xeon Processor. Stimulus displays were
POST-PRINT
rear-projected with a Canon LV8235-UST projector
onto a Da-Lite screen with a diagonal span of 132” at a
resolution of 1920 by 1080 and a distance of 300cm.
Head position was recorded using an OptiTrack optical
motion tracking system (Natural Point, Inc.) with six
V100:R2 cameras. The participant’s head was tracked
using a rigid body (providing 6DOF position and
orientation) defined by five passive reflection markers,
affixed to the front of a baseball cap (Figure 1B). The
position and orientation of the head was then used to
determine the window position on the screen.
Results
Accuracy. A trial was accurate if the participant
made a response while the target was within the gaze
contingent window (i.e., visible). Overall accuracy was
high across conditions (ranging from 93.5% - 97.1%).
Mean accuracy is reported in Table 1, and was
analyzed with an Effector (Eye, Head) by Repetition
(Random, Repeated) by Set Size (12, 24) mixed-factors
repeated measures ANOVA. Only the Effector by
Repetition interaction was significant, F(1, 58) = 7.98,
MSE = .001, p < .01 (all other effects: largest F = 2.19,
p = .144). This interaction was resolved by evaluating
the main effect of Repetition in a separate ANOVA for
each Effector. For Head-contingent search, there was a
significant main effect of Repetition, F(1, 29) = 5.61,
MSE = .002, p < .05, such that searchers were more
accurate in Repeated (96.8%) as compared to Random
(95.0%) conditions. For Eye-contingent search, the
main effect of Repetition was not significant (F = 2.37,
p = .135).
Table 1. Accuracy for Eye-contingent and Head-contingent
search, split by Random and Repeated conditions, and by Set
Size. Bracketed values are one standard error of the mean.
Random
Repeated
12
24
12
24
Eye
94.7%
(0.7)
95.3%
(0.8)
94.9%
(0.8)
93.5%
(1.0)
Head
95.1%
(1.3)
94.9%
(1.1)
97.1%
(0.6)
96.5%
(0.7)
Response Times. Response time (RT) distributions
across
conditions
were
positively
skewed.
Consequently, we plot and analyze median RTs (Figure
2A). The data were analyzed with an Effector (Eye,
Head) by Repetition (Random, Repeated) by Set Size
(12, 24) mixed-factors repeated measures ANOVA. All
doi:10.1016/j.cognition.2014.05.005
ENERGETIC VS COGNITIVE RESOURCES IN SEARCH
7
Figure 2. Median Response Times in seconds (A) and mean Slopes in milliseconds per item (B), plotted for Head-contingent
and Eye-contingent search, across Random and Repeated conditions. Error bars depict one standard error of the mean.
three main effects were significant, with faster RTs for
the smaller Set Size, F(1, 58) = 211.4, MSE = .809, p
< .001, faster RTs for Eye-contingent as compared to
Head-contingent search, F(1, 58) = 34.4, MSE = 2.669,
p < .001, and faster RTs for Repeated as compared to
Random search, F(1, 58) = 83.5, MSE = 1.271, p
< .001. These effects were qualified by several
interactions. A significant Effector by Repetition
interaction showed that the Repetition effect was larger
for Head-contingent than for Eye-contingent search,
F(1, 58) = 11.2, MSE = 1.271, p < .005. In addition, a
Repetition by Set Size interaction indicated that search
was not only faster under Repeated conditions, but also
more efficient (i.e., shallower RT by Set Size slopes),
F(1, 58) = 12.4, MSE = .304, p < .001. Both of these
two-way interactions were further implicated in a
marginal three-way Effector by Repetition by Set Size
interaction, F(1, 58) = 3.45, MSE = .304, p = .068. The
Effector by Set Size interaction was not significant (F <
1, p = .419), indicating comparable overall efficiency
across Effector types. These results were followed up
by conducting a separate Repetition by Set Size
ANOVA for each Effector type. For Eye-contingent
search, all effects were significant, with faster RTs at
the smaller Set Size, F(1, 29) = 184.8, MSE = .516, p
< .001, faster RTs for Repeated as compared to
POST-PRINT
Random search, F(1, 29) = 21.3, MSE = .999, p < .001,
and more efficient search for Repeated as compared to
Random conditions, F(1, 29) = 17.7, MSE = .248, p
< .001. In contrast, for Head-contingent search, RTs
were again faster for the smaller Set Size, F(1, 29) =
69.2, MSE = 1.102, p < .001, and for Repeated as
compared to Random search, F(1, 29) = 64.2, MSE =
1.543, p < .001, but there was no difference in
efficiency (F = 1.163, p = .290).
In sum, overall response time differences
demonstrate the classic repeated search benefit in main
effects. While this main effect was significantly larger
for head-contingent than for eye-contingent search, we
also report a selective efficiency improvement for eyecontingent but not for head-contingent search. This
somewhat ambiguous pattern in overall RTs can be
further resolved with more specific measures.
Response Time Decomposition. To gain further
insight, we leveraged the continuously sampled gaze
position data to partition overall RTs on each trial into
three components: 1) Initiation Time, the time elapsed
between onset of the search display and gaze-position
leaving the central target template, 2) Search Time, the
time elapsed between Initiation Time and the first
occasion that the target item fell within the gazecontingent window, and 3) Decision Time, the time
doi:10.1016/j.cognition.2014.05.005
8
SOLMAN & KINGSTONE
elapsed between the first occasion that the target fell
within the gaze-contingent window and the manual
response that terminated the trial. Average times were
determined for each of these components, and analyzed
separately (Figure 3).
Initiation Time. Initiation Times reflect the delay
between onset of the search display, and onset of overt
search behaviors. This time is likely to be occupied
primarily with encoding of the target and planning of
the initial movement and subsequent search path. Since
target encoding demands are constant across conditions,
differences in Initiation Time for a given effector will
reflect differences in the planning stage. In particular,
on the assumption that planning an arbitrary or ‘default’
search movement will require less time than planning a
movement on the basis of memory-driven target
location expectancies, memory use will be reflected in
increased Initiation Times for Repeated as compared to
Random search.
Initiation Time was computed by finding the first
sample in each trial such that the distance between the
center of the gaze-contingent window and the center of
the screen (where the target template was displayed)
was greater than or equal to the radius of the window
(Figure 3A). Average Initiation Times were analyzed
with an Effector (Eye, Head) by Repetition (Random,
Repeated) by Set Size (12, 24) ANOVA. Initiation
Times were longer for Head-contingent search, F(1,
58) = 11.5, MSE = .051, p < .005, and longer for
Repeated as compared to Random search, F(1, 58) =
18.6, MSE = .020, p < .001. The Effector by Repetition
interaction was also significant, F(1, 58) = 10.3, MSE
= .020, p < .005, with a smaller Repetition effect for
Eye-contingent search (20ms; CI95: -31 to 72) than for
Head-contingent search (137ms; CI95: 85 to 189). No
other effects were significant (largest F = 2.55, p
= .115). Consistent with expectations, Initiation Times
were greater overall for Repeated search than for
Random search, reflecting the pre-search temporal
costs of memory use. Importantly, these increases were
markedly larger (~7 times) for Head-contingent than
for Eye-contingent search, supporting a greater use of
memory when orienting with a larger and more costly
effector.
Search Time. Search Time was computed as the
time elapsed between Initiation Time and the time of
the first sample where the target was visible (Figure
3B). These times were analyzed with an Effector (Eye,
Head) by Repetition (Random, Repeated) by Set Size
POST-PRINT
Figure 3. Response Time decomposition. Initiation Time
(A), Search Time (B), and Decision Time (C), in seconds,
plotted for Head-contingent and Eye-contingent search,
across Random and Repeated conditions, and across Set Size.
Error bars depict one standard error of the mean.
(12, 24) ANOVA. The pattern of results was identical
to the overall RTs, with the exception that the
Repetition by Set Size interaction only approached
significance, F(1, 58) = 2.90, MSE = .183, p = .094.
Critically, however, the qualifying three-way Effector
by Repetition by Set Size interaction was significant,
F(1, 58) = 4.66, MSE = .183, p < .05, reflecting again
that Repeated search was more efficient than Random
doi:10.1016/j.cognition.2014.05.005
ENERGETIC VS COGNITIVE RESOURCES IN SEARCH
search only for the Eye-contingent condition, and not
for the Head-contingent condition.
Decision Time. Decision Time was measured as the
time between the first sample where the target item was
visible (i.e., fell within the gaze-contingent window)
and the termination of trial via manual response. This
measure captures the time required to register and
confirm the target’s identity, and to initiate a button
response. We anticipated that confirming a target’s
identity at an expected location would be faster than
confirming a target’s identity when it had been merely
happened upon by chance. In other words, when a
target’s location is successfully recalled, searchers
should be faster to confirm its identity and respond,
leading to shorter Decision Times.
Average Decision Times (Figure 3C) were analyzed
with an Effector (Eye, Head) by Repetition (Random,
Repeated) by Set Size (12, 24) ANOVA. Decision
Times were faster for the smaller Set Size, F(1, 58) =
61.0, MSE = .044, p < .001, faster for Eye-contingent
than Head-contingent search, F(1, 58) = 15.9, MSE
= .267, p < .001, and faster for Repeated than Random
search, F(1, 58) = 28.0, MSE = .086, p < .001.
Additionally, an Effector by Set Size interaction
showed that the influence of Set Size was greater for
Head-contingent search, F(1, 58) = 17.6, MSE = .044,
p < .001. Finally, the reduction in Decision Times
under Repeated conditions was more pronounced for
Head-contingent search (Effector by Repetition
interaction), F(1, 58) = 8.48, MSE = .086, p < .01, with
an average reduction of 90ms (CI95: -197 to 17) for
Eye-contingent search, and an average reduction of
310ms (CI95: -417 to -203) for Head-contingent search.
In sum, searchers were faster to identify and respond to
the target under Repeated search conditions, consistent
with searchers being able to predict the target location
and thereby reducing the necessary decision process to
one of verification. Further, the magnitude of this
savings was greater (~3 times) for Head-contingent
than for Eye-contingent searchers, suggesting more
successful target location recollection in this condition.
Early Orienting - Departure Angle. Early orienting
accuracy was assessed by examining the first
movement during search, and determining to what
extent these initial movements were systematically
biased toward the target – which could only be
explained by successful memory for the target location.
Departure Angles were computed based on the position
of the gaze-contingent window at Initiation Time. A
POST-PRINT
9
vector was constructed from the center of the screen to
this position (‘observed’ vector), and from the center of
the screen to the position of the target (‘ideal’ vector).
The signed angle between these vectors measures the
directional accuracy of the initial search trajectory,
with angles closest to 0° indicating trajectories directly
towards the target, and angles closest to ±180°
indicating trajectories directly away from the target.
The observed distributions of target-relative trajectories
are plotted in Figure 4.
To estimate the contribution of memory success
during search, we adapt the methods used by Zhang &
Luck (2008), using maximum likelihood estimation
(MLE) to fit the observed distributions during repeated
search trials to a mixture of random and memorydriven trials. Memory-driven trials were assumed to
follow a von Mises distribution (the circular analogue
to the normal distribution) with parameters μ and κ, for
the mean and dispersion, respectively (see Fisher,
1995). For random trials, we use each participant’s own
observed distribution from trials in the random
condition. Specifically, we computed the screenrelative distribution of initial trajectories during
random trials and used this distribution to compute the
likelihood of a given target-relative trajectory for each
trial in the repeated conditions. This approach was
favored over use of the uniform circular distribution in
order to compensate for both the non-uniform
distribution of target positions in the rectangular screen,
as well as systematic idiosyncrasies in participants’
scanpaths (e.g., Gilchrist & Harvey, 2006; Noton &
Stark, 1971). Indeed, initial departure trajectories in the
random conditions differed both from a circular
uniform distribution (Eye: χ2 = 38.0, p < .001; Head: χ2
= 18.7, p = .068), and from an ‘item-neutral’
distribution constructed using the configuration of
items on the screen and assuming uniform selection
across items (Eye: χ2 = 69.9, p < .001; Head: χ2 = 48.1,
p < .001). Overall, participants appeared to exhibit a
strong bias to initiate their first movement toward the
lower left during random trials (Figure 5). Although
this bias appeared somewhat more pronounced for Eye
movements as compared to Head movements, the
effector conditions were not found to differ
significantly from each other, χ2 = 10.1, p = .523.
The critical parameter to be estimated, Pm, is the
probability of memory success, which determines the
relative weighting of memory trials and random trials,
with (1-Pm) giving the probability of a random trial.
Because of the peaked nature of the random
doi:10.1016/j.cognition.2014.05.005
10
SOLMAN & KINGSTONE
Figure 4. Observed distributions of departure angles relative to the target position in 30° bins, plotted as polar probability
density functions, with the target position directly upwards at 0°. Distributions for Random (dotted lines) and Repeated (solid
lines) conditions are plotted across Effect condition (Eye, Head) and across Set Sizes (12, 24).
distributions (see Figure 4, Figure 5), the mean of the
von Mises distribution, μ, was fixed at 0.0 to prevent
fitting background peaks that were unrelated to the
target and thereby erroneously inflating memory
estimates. The two remaining parameters κ, and Pm
were estimated using MLE for each subject and each
set size in the Repeated condition. Since the random
prior was based on the random trials, this analysis was
restricted to the Repeated conditions.
Ensemble predictions based on these parameter
estimates were tested against observed distributions by
comparing histograms with twelve 30° bins with a
linear regression predicting the observed data with the
modeled data. More than 50% of the variance was
POST-PRINT
predicted in every condition, and as much as 83% in
the best condition (Eye, Set Size 12: Adjusted R2
= .725, p < .001; Eye, Set Size 24: Adjusted R2 =.514,
p < .01; Head, Set Size 12: Adjusted R2 =.832, p
< .001; Head, Set Size 24: Adjusted R2 =.822, p < .001).
As further validation for the Pm estimates, we
attempted to reconstruct the observed RT patterns in
repeated search conditions, based on the observed
Initiation Time ti and Decision Time td on Repeated
trials, the observed Search Time on Random trials tr,
the estimated values of Pm, and a constant tm reflecting
the time required for successful memory searches
(excluding initiation and decision times):
RT* = ti + td + (1-Pm) tr + Pm tm
doi:10.1016/j.cognition.2014.05.005
ENERGETIC VS COGNITIVE RESOURCES IN SEARCH
11
tests were consistent with this observation (Eye: t < 1,
p = .585; Head: t(29) = 2.115, p < .05). There was no
main effect of Set Size (F = 1.19, p = .280). Consistent
with previously reported measures, estimated Pm values
indicated a higher probability of memory success for
Head-contingent as compared to Eye-contingent search,
offering further evidence that the relative costs of using
different orienting effectors can be offset by
differential memory use.
Figure 5. Distribution of screen-relative absolute departure
angles during Random search, plotted as polar probability
density functions. Distributions for Eye-contingent (dotted
black) and Head-contingent (solid black) are compared with
a circular Uniform distribution (dotted grey) and an itemNeutral distribution (solid grey; see text).
Memory Use and Memory Success. Next, we tested
the relation between individual differences in Initiation
Time and in Pm estimates. Recall that Pm reflects the
proportion of trials where memory was successful,
whereas Initiation Time differences (subtracting
Random ti from Repeated ti) ought to reflect the extent
to which memory search was attempted, regardless of
success. Consequently, these measures should be
positively related – as the proportion of trials where
memory is successful is necessarily bounded by the
extent to which memory is queried in the first place.
We make the simplifying assumption that
successful memory searches take zero time (having
already accounted for initiation and decision times) –
i.e., that tm = 0. Consequently we can expect that our
estimates will be faster than the actual RTs, but this
should not unduly influence estimated slopes.
Reconstructed Repeated RTs* were entered with
observed Random RTs into a new Effector (Eye, Head)
by Repetition (Random, Repeated) by Set Size (12, 24)
ANOVA. All observed effects from the original
response time analysis were reproduced. Critically, this
included the observed reduction in repeated compared
to random search slopes for Eye-contingent search:
Repetition X Set Size interaction, F(1, 29) = 10.5, MSE
= .507, p < .005, but not for Head-contingent search: F
< 1, p = .922. These results provide good support for
the validity of the estimated Pm values, and likewise
suggest that the unusual slope patterns in RTs may be
traced to the interaction between set size and the
likelihood of successful target memory. We expand
upon these observations in the discussion.
Estimated Pm. The estimated values for Pm (Figure
6) were tested with an Effector (Eye, Head) by Set Size
(12, 24) ANOVA. Pm values were larger for Headcontingent than for Eye-contingent search, F(1,58) =
5.22, MSE = .154, p < .05, with the suggestion of a
qualifying Effector by Set Size interaction, F(1,58) =
3.52, MSE = .080, p = .066, whereby Pm values appear
to be reduced at the larger Set Size for Head-contingent
but not Eye-contingent search. Unplanned post-hoc tPOST-PRINT
Figure 6. Average parameter estimates for the probability of
successful memory use (Pm) during repeated search, plotted
for Eye-contingent (dark bars) and Head-contingent (light
bars) conditions, and across Set Sizes (12, 24). Error bars
depict one standard error of the mean.
doi:10.1016/j.cognition.2014.05.005
12
SOLMAN & KINGSTONE
Consistent with this prediction, small but
significant positive correlations were observed for
Head-contingent search at both set sizes (Set Size 12: r
= .378, p < .05; Set Size 24: r = .556, p < .005) and for
Eye-contingent search at Set Size 24, r = .393, p < .05,
though not at Set Size 12 (r = .196, p = .298). These
relations provide further support for the underlying
assumptions of our measures.
Estimated κ. The parameter κ, provides a metric of
dispersion, corresponding to the accuracy of memorydriven trials, with larger values of κ yielding tighter
distributions (i.e., higher accuracy). Unfortunately, as
Pm tends to zero, the value of κ becomes increasingly
unconstrained – as the von Mises distribution
contributes increasingly little to the overall prediction.
Computing weighted averages of estimated κ, weighted
by Pm, suggests greater accuracy for eye movements
than for head movements, but these observations are
necessarily only suggestive.
Discussion
When searchers use head-movement rather than eye
movement for perceptual exploration, there is evidence
for both an increased use of memory (longer initiation
times), and a commensurate increase in successful
recall of target locations (more probable early orienting
to the target). Participants in both effector conditions
showed improvements in search speed and orienting
metrics; however, these advantages were consistently
more pronounced for head-contingent searches.
Response time decomposition showed that headcontingent searchers waited substantially longer to
initiate search when displays were repeated than did
eye-contingent searchers, consistent with more
resources being devoted to recall efforts prior to
initiating a perceptual search. Commensurately, headcontingent searchers were also more likely to orient
directly to the target item, as reflected in larger Pm
estimates, and were faster to respond once the target
was acquired, measured by reduced Decision Times in
repeated search.
An apparent exception to this pattern of increased
benefits for head-contingent search emerged in search
efficiency (response time by set size slopes). Although
the main effect of repetition was significantly larger for
head-contingent search, efficiency improved only for
eye-contingent search. An explanation for this result is
readily found by examining how the likelihood of
POST-PRINT
memory success varied across effectors and set size –
noting that reconstructed repeated RTs, based on
estimated Pm values and observed random RTs,
successfully reproduced the pattern of slopes.
Examining the Pm values (Figure 6) we see a constant
likelihood of success for eye-contingent search, but a
decreasing likelihood for head-contingent search with
increasing set size. Coupled with the observation that
memory use (as indexed by delayed Initiation Times)
was lower in eye-contingent search, the strong
implication is that head-contingent searchers exploited
memory close to or at capacity, whereas eye-contingent
searchers did not. Consequently, whereas headcontingent searchers were much more likely to
successfully recall and acquire the target location at set
size 12, when the set size doubled this advantage over
eye-contingent searchers disappeared. The ordinal
trend in dispersion (κ) estimates suggests this could be
because the targeting accuracy of head movement
versus eye movement searches ranges from relatively
coarse to fine, respectively. Further studies, including
explicit measures of capacity limits and their relation to
movement accuracy, will be necessary to confirm and
fully resolve these observations.
Another remaining question is to what degree the
present results can be confidently attributed to the
energetic costs of orienting with different effectors as
opposed to the temporal costs (c.f. the ‘soft constraints
hypothesis’; Gray, Sims, Fu, & Schoelles, 2006). In
other words, do head-contingent searchers rely more
heavily on memory in order to save energy, or to save
time? Although the motor kinematics literature offers
indirect support for a role for energy (e.g., Nelson,
1983; Sparrow & Newell, 1998; Wang & Hsiang,
2011), using the present data, we can attempt to answer
this question more directly. First, we note that for the
range of gaze shift eccentricities in the present study,
head movements have previously been reported to be
between 4 and 10 times slower than corresponding eye
movements (Zangemeister & Stark, 1981). However,
from our results, during random trials head-contingent
search was only about 1.5 times slower, and actually
exhibited a shallower slope than eye-contingent search,
suggesting that – like oculomotor paralysis patients
who cannot use eye movements (Land, Furneaux, &
Gilchrist, 2002) – head-contingent searchers already
compensate for their time delays by searching more
efficiently.
We can further ask: if eye-contingent searchers used
memory at the rate of head-contingent searchers, then
doi:10.1016/j.cognition.2014.05.005
ENERGETIC VS COGNITIVE RESOURCES IN SEARCH
would the temporal savings be greater or less than
those observed? If greater temporal savings during eyecontingent search could have been achieved by relying
more heavily on memory, then the observed failure to
do so cannot be accounted for on the basis of time. In
fact, temporal savings should have been, conservatively,
about 2.3 times (600ms - 800ms) greater than was
actually observed1. (We say conservatively because this
large gap between predicted and observed savings
arose despite the fact that we controlled for the
increased initiation times and discounted potential
savings in decision time that would have resulted in a
much larger gap.) In short, if greater use of memory
would have offered additional savings, then it is
difficult to explain the observed results from a timeonly perspective. Although temporal savings are
certainly likely to contribute to behavioral decisions,
time alone does not offer a compelling explanation for
the present results.
Concluding Comments
Naturalistic tasks, and extended behaviors in general,
involve both cognitive processes and perceptuo-motor
processes, and these tasks can often be accomplished
by various combinations of those processes. To
adequately characterize behavior in the real world, it is
important to determine what principles govern this
tradeoff. Our results add to the extensive collection of
studies examining these questions (e.g., Ballard,
Hayhoe, & Pelz, 1995; Ballard, Hayhoe, Pook, & Rao,
1997; Cary & Carlson, 2001; Droll & Hayhoe, 2007;
Droll, Hayhoe, Triesch, & Sullivan, 2005; Gray & Fu,
2004; Gray, Sims, Fu, & Schoelles, 2006; O’Hara &
Payne, 1998; Schönpflug, 1986), showing how the
energetic costs of the particular effectors used to
accomplish a given task can play an important role in
determining the balance between cognitive and
perceptuo-motor recruitment. As tasks become
physically extended, orienting will increasingly often
involve more – and more diverse – sets of effectors.
The present results indicate that this recruitment of
1
Adapting the RT reconstruction method used to validate Pm
estimates in the results section, we estimated the expected
eye-contingent repeated search RTs if memory use matched
that observed for head-contingent search. In particular, we
used head-contingent initiation times (hi) and Pm values, eyecontingent random search times (er) and decision times (ed),
and a non-zero memory search time chosen as the duration
of a typical saccade (tm = 50ms): RT* = hi + ed + (1-Pm) er +
P m tm
POST-PRINT
13
larger and more costly muscle groups is likely to
increase the reliance on internal processing. As the
study of search extends beyond traditional static
computer tasks and embraces the full suite of embodied
behaviors involved in search (e.g., Gilchrist, North, &
Hood, 2001; Robinson, Koth, & Ringenbach, 1976;
Ruddle, & Lessels, 2006; Smith, Hood, & Gilchrist,
2008; Solman, Cheyne, & Smilek, 2012; Solman, Wu,
Cheyne, & Smilek, 2013; Summala, Pasanen, Räsänen,
& Sievänen, 1996; Thomas et al., 2006), consideration
of these kinds of motoric costs and constraints may
become increasingly important in the interpretation of
results.
Acknowledgements
This work was supported by an NSERC Discovery
Grant to AK, and by an NSERC Postdoctoral
Fellowship and a Killam Trust Postdoctoral Research
Fellowship to GJFS.
References
Ballard, D. H., Hayhoe, M. M., & Pelz, J. B. (1995).
Memory representations in natural tasks. Journal of
Cognitive Neuroscience, 7, 66-80.
Ballard, D. H., Hayhoe, M. M., Pook, P. K., & Rao, R.
P. N. (1997). Deictic codes for the embodiment of
cognition. Behavioral and Brain Sciences, 20, 723767.
Barnes, G. R. (1979). Vestibulo-ocular function during
co-ordinated head and eye movements to acquire
visual targets. Journal of Physiology, 287, 127-147.
Bizzi, E., Kalil, R. E., & Tagliasco, V. (1971). Eyehead coordination in monkeys: Evidence for
centrally patterned organization. Science, 173, 452454.
Brainard, D. H. (1997). The Psychophysics Toolbox.
Spatial Vision, 10, 443-446.
Cary, M., & Carlson, R. A. (2001). Distributing
working memory resources during problem solving.
Journal of Experimental Psychology: Learning,
Memory, and Cognition, 27, 836-848.
Clark, A. (1999). An embodied cognitive science?
Trends in Cognitive Sciences, 3, 345-351.
Cornelissen, F. W., Peters, E., & Palmer, J. (2002). The
Eyelink Toolbox: Eye tracking with MATLAB and
the Psychophysics Toolbox. Behavior Research
Methods, Instruments, & Computers, 34, 613-617.
Droll, J. A., & Hayhoe, M. M. (2007). Trade-offs
between gaze and working memory use. Journal of
doi:10.1016/j.cognition.2014.05.005
14
SOLMAN & KINGSTONE
Experimental Psychology: Human Perception and
Performance, 33, 1352-1365.
Droll, J. A., Hayhoe, M. M., Triesch, J., & Sullivan, B.
T. (2005). Task demands control acquisition and
storage of visual information. Journal of
Experimental Psychology: Human Perception and
Performance, 31, 1416-1438.
Faulkner, R. F., & Hyde, J. E. (1958). Coordinated eye
and body movements evoked by brainstem
stimulation in decerebrated cats. Journal of
Neurophysiology, 21, 171-182.
Fisher, N. I. (1995). Statistical Analysis of Circular
Data. New York, Cambridge University Press.
Foulsham, T., Walker, E., & Kingstone, A. (2011). The
where, what and when of gaze allocation in the lab
and the natural environment. Vision Research, 51,
1920-1931.
Freedman, E. G. (2008). Coordination of the eyes and
head during visual orienting. Experimental Brain
Research, 190, 369-387.
Fuller, J. H. (1992). Head movement propensity.
Experimental Brain Research, 92, 152-164.
Gilchrist, I. D., North, A., & Hood, B. (2001). Is visual
search really like foraging? Perception, 30, 14591464.
Gilchrist, I. D., & Harvey, M. (2006). Evidence for a
systematic component within scan paths in visual
search. Visual Cognition, 14, 704-715.
Glenberg, A. M. (2010). Embodiment as a unifying
perspective for psychology. Wiley Interdisciplinary
Reviews: Cognitive Science, 1, 586-596.
Goossens, H. H. L. M., & Van Opstal, A. J. (1997).
Human eye-head coordination in two dimensions
under
different
sensorimotor
conditions.
Experimental Brain Research, 114, 542-560.
Gray, W. D., & Fu, W-T. (2004). Soft constraints in
interactive behavior: the case of ignoring perfect
knowledge in-the-world for imperfect knowledge
in-the-head. Cognitive Science, 28, 359-382.
Gray, W. D., Sims, C. R., Fu, W-T., & Schoelles, M. J.
(2006). The soft constraints hypothesis: A rational
analysis approach to resource allocation for
interactive behavior. Psychological Review, 113,
461-482.
Hogan, N. (1984). An organizing principle for a class
of voluntary movements. The Journal of
Neuroscience, 4, 2745-2754.
POST-PRINT
Hollingworth, A. (2012) Guidance of visual search by
memory and knowledge. In M. D. Dodd, J. H.
Fowlers (Eds.) The Influence of Attention, Learning,
and Motivation on Visual Search, Nebraska
Symposium on Motivation (pp. 63-89). New York:
Spring Science.
Isa, T., & Sasaki, S. (2002). Brainstem control of head
movements during orienting; organization of the
premotor circuits. Progress in Neurobiology, 66,
205-241.
Itti, L., & Koch, C. (2000). A saliency-based search
mechanism for overt and covert shifts of visual
attention. Vision Research, 40, 1489-1506.
Kunar, M. A., Flusberg, S., & Wolfe, J. M. (2008). The
role of memory are restricted context in repeated
search. Perception & Psychophysics, 70, 314-328.
Land, M. F. (2004). The coordination of rotations of
the eyes, head and trunk in saccadic turns produced
in natural situations. Experimental Brain Research,
159, 151-160.
Land, M. F., Furneaux, S. M., & Gilchrist, I. D. (2002).
The organization of visually mediated actions in a
subject without eye movements. Neurocase, 8, 8087.
Lee, C. (1999). Eye and head coordination in reading:
roles of head movement and cognitive control.
Vision Research, 39, 3761-3768.
Morasso, R., Bizzi, E., & Dichgans, J. (1973).
Adjustment of saccade characteristics during head
movements. Experimental Brain Research, 16, 492500.
Nelson, W. L. (1983). Physical principles for
economies of skilled movements. Biological
Cybernetics, 46, 135-147.
Noton, D., & Stark, L. (1971). Scanpaths in saccadic
eye movements while viewing and recognizing
patterns. Vision Research, 11, 929-942.
O’Hara, K. P., & Payne, S. J. (1998). The effects of
operator implementation cost on planfulness of
problem solving and learning. Cognitive Psychology,
35, 34-70.
Oliva, A., Wolfe, J. M., & Arsenio, H. C. (2004).
Panoramic search: The interaction of memory and
vision in search through a familiar scene. Journal of
Experimental Psychology: Human Perception and
Performance, 30, 1132-1146.
Pelli, D. G. (1997) The VideoToolbox software for
visual psychophysics: Transforming numbers into
movies. Spatial Vision, 10, 437-442.
doi:10.1016/j.cognition.2014.05.005
ENERGETIC VS COGNITIVE RESOURCES IN SEARCH
Robinson, G. H., Koth, B. W., & Ringenbach, J. P.
(1976). Dynamics of the eye and head during an
element of visual search. Ergonomics, 19, 691-709.
Ruddle, R. A., & Lessels, S. (2006). For efficient
navigational search, humans require full physical
movement, but not a rich visual scene.
Psychological Science, 17, 460--465.
Schönpflug, W. (1986). The trade-off between internal
and external information storage. Journal of
Memory and Language, 25, 657-675.
Smith, A. D., Hood, B. M., & Gilchrist, I. D. (2008).
Visual search and foraging compared in a largescale search task. Cognitive Processing, 9, 121-126.
Solman, G. J. F., Cheyne, J. A., & Smilek, D. (2011).
Memory load affects visual search processes
without influencing search efficiency. Vision
Research, 51, 1185-1191.
Solman, G. J. F., Cheyne, J. A., & Smilek, D. (2012).
Found and missed: Failing to recognize a search
target despite moving it. Cognition, 123, 100-118.
Solman, G. J. F., & Smilek, D. (2010). Item-specific
memory in visual search. Vision Research, 50,
2430-2438.
Solman, G. J. F., & Smilek, D. (2012). Memory
benefits during visual search depend on difficulty.
Journal of Cognitive Psychology, 24, 689-702.
Solman, G. J. F., Wu, N., Cheyne, J. A., & Smilek, D.
(2013). In manually-assisted search, perception
supervises rather than directs action. Experimental
Psychology, 60, 243-254.
Sparks, D. L. (1991). The neural control of orienting
eye and head movements. In: Humphrey, D. R.,
Freund, H.-J. (Eds.) Motor Control: Concepts and
Issues. Wiley, New York, pp. 263-275.
Sparrow, W. A., & Newell, K. M. (1998). Metabolic
energy expenditure and the regulation of movement
economy. Psychonomic Bulletin & Review, 5, 173196.
Summala, H., Pasanen, E., Räsänen, M., & Sievänen, J.
(1996). Bicycle accidents and drivers’ visual search
at left and right turns. Accidental Analysis &
Prevention, 28, 147-153.
Thomas, L. E., Ambinder, M. S., Hsieh, B., Levinthal,
B., Crowell, J. A., Irwin, D. E., Kramer, A. F.,
Lleras, A., Simons, D. J., & Wang, R. F. (2006).
Fruitful visual search: Inhibition of return in a
virtual foraging task. Psychonomic Bulletin &
Review, 13, 891-895.
POST-PRINT
15
Treisman, A., & Gelade, G. (1980). A featureintegration theory of attention. Cognitive
Psychology, 12, 97-136.
Uno, Y., Kawato, M., & Suzuki, R. (1989). Formation
and control of optimal trajectory in human
multijoint arm movement. Biological Cybernetics,
61, 89-101.
Võ, M., & Wolfe, J. M. (2012). When does repeated
search in scenes involve memory? Looking at
versus looking for objects in scenes. Journal of
Experimental Psychology: Human Perception and
Performance, 38, 23-41.
Wang, X., & Hsiang, S. M. (2011). Modeling trade-off
between time-optimal and minimum energy in
saccade main sequence. Biological Cybernetics, 104,
65-73.
Wilson, M. (2002). Six views of embodied cognition.
Psychonomic Bulletin & Review, 9, 625-636.
Winters, J. M., & Stark, L. (1985). Analysis of
fundamental human movement patterns through the
use of in-depth antagonistic muscle models. IEEE
Transactions in Biomedical Engineering, BME-32,
826-839.
Wolfe, J. M. (1994). Guided search 2.0: A revised
model of visual search. Psychonomic Bulletin &
Review, 1, 202-238.
Wolfe, J. M. (2007). Guided search 4.0: Current
progress with a model of visual search. In W. Gray
(Ed.) Integrated Models of Cognitive Systems (pp.
99-119). New York: Oxford.
Wolfe, J. M., Klempen, N., & Dahlen, K. (2000).
Postattentive vision. Journal of Experimental
Psychology: Human Perception and Performance,
26, 693-716.
Zangemeister, W. H., & Stark, L. (1981). Active head
rotations and eye-head coordination. Annals of the
New York Academy of Sciences, 374, 540-559.
Zhang, W., & Luck, S. J. (2008). Discrete fixedresolution representations in visual working
memory. Nature, 453, 233-235.
doi:10.1016/j.cognition.2014.05.005