PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 53rd ANNUAL MEETING—2009
16
Do Velocity Vectors Support Multiple Object Tracking?
Krista Oinonen1, Lauri Oksama1, Esa Rantanen2 and Jukka Hyönä1
1
University of Turku, Finland
Rochester Institute of Technology, Rochester, NY, USA
2
We examined the effect of velocity vectors on human ability to extrapolate movement of multiple moving
objects. This was achieved using an identity tracking task, in which objects (line-drawings of familiar objects) first move for 5 s, after which they temporarily disappear from view. In order to examine different
aspects of vector information, we compared 4 velocity vector conditions: (1) no vectors were displayed; (2)
only a history trail was displayed; (3) a condition where the end of the velocity vector pointed to exactly
where the objects would reappear after the masking; (4) a constant heading length was displayed. Based on
recent studies on Keane and Pylyshyn (2006) and Oinonen et al. (2009), we hypothesized that the condition, where exact location information is presented, would improve performance compared to baseline. The
results supported this hypothesis by indicating that history trail or directional vector do not improve performance but only complete visual information about the future location will help to anticipate movement
in multiple object tracking.
Copyright 2009 by Human Factors and Ergonomics Society, Inc. All rights reserved. 10.1518/107118109X12524440832665
INTRODUCTION
In many real-life dynamic visual environments, such as
traffic, sports, or air traffic control (ATC), it is important to
anticipate in what direction critical objects are about to move
and where they would be positioned in the near future. In order to aid to predict the targets’ trajectories into the future, velocity vectors (or, track velocity and direction vector lines) are
common in ATC plan view displays (PVD). The velocity vectors achieve a very simple thing of making the dynamic characteristics of targets (i.e., aircraft) on controllers’ display visually available. Without such visualization the controller
would be under a heavy cognitive burden having to rely on
past experience on the target dynamics (memory) and integration of a large number of circumstantial factors to predict the
targets’ trajectories into the future.
It appears that the putative benefits of visual representation of target dynamics were almost coincidental to the old radar technology and cathode ray tube (CRT) displays. Because
the radar antenna continuously rotates and thus ‘sweeps’ the
atmosphere for returns from solid objects, a target is only ‘illuminated’ for the moment the radar antenna is pointed at it;
phosphorus coating of the CRT displays provided an afterglow
of the radar return (Cole, 1985), forming a history trail for a
moving target for past 3-5 positions (depending on the phosphorus decay rate). From such history trails controllers can
glean targets’ direction, velocity (the distance between successive past positions), and rate of turn information, and extrapolate from these the future trajectories of the targets. In subsequent digital radar PVDs the history of the targets continued to
be provided, together with the computed velocity vector projected ahead of the target (Nolan, 1998). However, remarkably
little research has been published on the theoretical underpinnings of predictor displays in ATC.
But are vectors really helpful? Do they really help in
movement extrapolation? We raise these questions as recent
studies on tracking of multiple moving objects that temporarily disappear from the observer’s field of view have suggested
that observers are not able to update or anticipate objects’ lo-
cations (Keane & Pylyshyn, 2006; Oinonen, Oksama &
Hyönä, 2009). Our ability to extrapolate movement of multiple moving objects during interruption or occlusion hence
appears to be quite poor.
According to Keane and Pylyshyn (2006) observers can
only remember where the critical objects were located at the
time of their disappearance or slightly before that. Even knowing the objects’ movement directions does not help observers
to extrapolate and predict the future locations of multiple objects. According to the displacement hypothesis of Keane and
Pylyshyn, predicting object’s location is the poorer the farther
the objects move while invisible, causing bigger displacement
from their last visible locations.
These results seem to cast a shadow on the intuitive idea
that observers can easily anticipate multiple objects’ movement and benefit from velocity vector information. In particular, it follows from these studies that vectors including only directional information (heading or history) may not be beneficial at all, as they do not give any exact visual information on
targets’ future locations. Cognitively demanding movement
extrapolation process is still required. Instead, only vectors
that provide exact information on targets’ future location may
be truly helpful.
Because these experimental tasks are clearly analogous to
those of air traffic controllers, where the occlusion of targets
corresponds to the controller taking his or her eyes off the
PVD to perform other tasks, an ATC task offers an attractive
approach to deeper understanding of the contribution of velocity vectors to human multiple object tracking performance.
Past Research on Predictor Displays
Much of predictor display literature pertains to control
engineering (e.g., Kelley, 1962; 1968) and active vehicular
control with applications in nuclear submarines (Berbert &
Kelley, 1962) and lunar landing (Fargel & Ulbrich, 1963). For
a flavor of this body of literature and further examples of driving an automobile in traffic, a blind pedestrian using a cane or
electronic obstacle detector, and remote manipulation of solid
PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 53rd ANNUAL MEETING—2009
objects using artificial sensors and effectors, see Sheridan
(1966). Textbook descriptions of predictor displays are quite
short and lack any treatment of theoretical background. Kantowitz and Sorkin (1983) only cite Palmer, Jago, Baty, and
O’Connor (1980) about the benefits of predictor displays, and
Sanders and McCormick (1993) cite Kelley (1962; 1968) as
well as Roscoe, Corl, and Jensen (1981) and Jensen (1981)
about quickened displays.
As far as empirical human-in-the-loop (HITL) studies are
concerned, these come from either direct vehicular control or
airborne collision detection settings. Kelley (1968) cites several studies that have demonstrated the benefits of predictor
displays in vehicular control, for example controlling a spacecraft and correct rapidly changing thrust disturbances (Besco,
1964) and submarine displays in collision avoidance task
(McLane & Wolf, 1966). In the aviation domain most of the
predictor display research has been on advanced cockpit displays, either perspective flight displays (e.g., Jensen, 1981;
Grunwald, 1981, Merwin, & Wickens, 1996; Wickens,
Haskell, & Harte, 1989) or cockpit displays of traffic information (CDTIs; Hart & Loomis, 1980; Kelly, 1983; Kelly &
Abbot, 1984; Kelly & William, 1983; Palmer, Baty, & O'Connor, 1979; Palmer et al., 1980; Williams; 1983). These studies
also include HITL simulations with strong evidence about the
benefits of predictor displays to human performance in estimating future events (e.g., conflicts with other aircraft).
Palmer et al. (1980) examined straight and curved predictor lines in an airborne conflict detection task where subjects
estimated whether the intruder would pass in front of or behind the ownship. Predictors clearly aided the participating pilots’ performance when both aircraft were moving along
straight trajectories as well as when either aircraft was turning
if the predictors displayed turn-rate information. Interestingly,
the presence or absence of flight path history or the information update rate did not influence performance (although the
pilots preferred having history trails displayed and continuous
ownship position update).
In a similar experiment Hart and Loomis (1980) explored
the effect of CDTI velocity vectors on pilots' ability to judge
lateral relationships between their own and other aircraft. The
participating pilots judged as quickly as possible whether the
other aircraft would pass ahead of or behind the ownship at the
closest point of approach. Straight predictor lines reduced response time from about 38 s to 31 s and errors from about
15% to 0%in straight flight paths scenarios, and from 40 s to
34 s and from 28% to 5%, respectively, in curved flight path
scenarios. Straight predictors in curved flight path scenarios
did not help much.
Palmer (1983) also manipulated the quality of information
provided by the predictor vector in simulated conflict avoidance maneuvering tasks. The participating pilots were able to
avoid triggering conflict avoidance system (CAS) alarm 90%
of the time when the predictive information was free of noise,
but their performance suffered substantially (avoiding triggering the CAS alarm only 78% of the time) when predictor information was degraded.
In an eye-movement study on a CDTI Ellis and Stark
(1986) showed that the most common points of interest in that
17
display were the leader line predictors, with the aircraft symbol and history trail receiving respectively fewer visual fixations. These results attest to the value of the velocity vectors to
the subjects (8 experienced airline pilots) in their task (intruder
passing in front of or behind the ownship).
Although there seems to be ample empirical evidence for
predictor lines supporting task performance, it should be noted
that all the experimental tasks reviewed above were typically
very short in duration (quick judgments) and involved only a
few targets (typically only two, ownship and an intruder).
They were hence quite different from the ATC-like tasks of
interest to us in our study.
Velocity Vectors in ATC
There is little empirical research on different predictor
displays in ATC, despite their ubiquitous presence on most
modern ATC systems. Perhaps this is due to the long evolution of predictor information on ATC displays and the recent
research focus on the development of cockpit displays (i.e.,
CDTIs). Unfortunately, the results from CDTI research generalize poorly to ATC because the air traffic controller’s task is
fundamentally different from the pilot’s. In all of the experiments reviewed in the previous section the task was to estimate at the most two targets’ relative position in the immediate future, or compare a vehicle’s state to some criterion
value in direct vehicular control. However, while controllers
need to estimate aircraft trajectories and judge potential conflicts between aircraft in a pair, they also must maintain situation awareness on a much larger scale, involving tens of aircraft in busy sectors. The research paradigm of tracking multiple moving objects that are temporarily masked appears uniquely suitable for examination of the effects of velocity vectors on tracking performance in a task that is closely analogous to that of air traffic controllers.
The present study
In the present study we were interested in the influence of
velocity vectors on movement extrapolation. A version of
multiple identity tracking task (MIT, see Oksama & Hyönä,
2004, 2008) was employed, where at some point all moving
targets temporarily disappear from the screen (Oinonen, Oksama, & Hyönä, 2009). Participants were instructed to continue tracking the invisibly moving targets and extrapolate their
movement. When the targets reappeared, the participants’ task
was to indicate as fast as possible the current location of one
auditorily probed target. In MIT, the moving elements all have
distinct identities analogously to real-life tracking tasks; thus,
the task required continuous updating of what–where–
bindings. The task was designed to be a sensitive measure of
immediate situation awareness of the observers.
In order to examine different features of vector information, we compared four velocity vector conditions (no vectors,
only a history trail, velocity vectors corresponding to the
masking duration, and heading vectors with constant length)
over three target masking durations. Based on Keane and Pylyshyn (2006) and Oinonen et al. (2009) we expected that the
PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 53rd ANNUAL MEETING—2009
vectors corresponding to masking duration, and thereby visually presenting exact target location into the future, would
improve performance the most compared to the baseline.
METHOD
Participants
Twenty undergraduate psychology students (median age
24 years) from University of Turku, Finland, volunteered for
the study.
Apparatus and Experimental Task
The stimuli were presented on a 19-inch Samsung SyncMaster monitor with a resolution of 1280 by 1024 pixels controlled by NVIDIA GeForce FX5200 card, AMD Sempron
2400+, 1.66 GHz, 1.0 GB of RAM computer. The experiment
was programmed with the E-prime software (Schneider,
Eschman, & Zuccolotto, 2002a, 2002b). The program that
generated the motion sequences was written in Visual Basic.
The stimuli included 8 black-and-white line drawings
(familiar objects and animals), chosen from the picture corpus
of Snowgrass and Vanderwart (1980). The pictures were chosen so that their names in Finnish began with a consonant and
comprised 5 letters and 2 syllables. Their pronunciation time
was about 700 ms. The names were also matched with respect
to their frequency in a written language corpus (Laine & Virtanen, 1999). Pictures were 75 x 39 pixels in size, and extended a visual angle of 1.9 x 1.0 degrees.
The objects’ movement directions were randomly chosen
from among the four intermediate compass directions (northeast, south-east, south-west, and north-west). Object speed
was 2.42 degrees/s. Objects were allowed to move with intersecting trajectories. The objects moved for 5 s. after which
they disappeared from the display but continued to move. Participants were instructed to continue to track the invisibly
moving targets and extrapolate their movement during masking (disappearance). After a given time the objects reappeared
and the participants’ task was to immediately locate and indicate the auditorily designated target on the screen.
Design
Independent variables. We manipulated the masking duration (1680, 2520 and 3360 ms, resulting in the objects’ displacement during masking of approximately 4, 6, and 8 degrees of visual angle, respectively) and the number of targets
to be tracked (2 and 3 out of the total of 8 objects). We also
employed four velocity vector conditions: (1) in the baseline
(B) condition, no vectors were displayed; (2) in the track (T)
condition, only a history trail was displayed, indicating the objects’ movement direction but not their future positions; (3) in
the congruent (C) condition the velocity vectors corresponded
to the masking duration, that is, the end of the velocity vector
pointed to exactly where the objects would reappear after the
masking; (4) finally, in the heading (H) condition the objects’
velocity vector lengths were constant predicting the objects’
18
positions 2520 ms into the future. Note that this matched the
congruent condition when the masking duration was 2520 ms,
but with masking duration of 1680 ms the vector pointed
farther and with 3360 ms masking duration closer than where
the objects would reappear. This condition tested whether the
length of the vector aided in anticipation of the targets’
movement. In all visual vector conditions (track, congruent
and heading) the vectors were presented in all eight objects.
Dependent variable. The movement extrapolaton performance was measured by response time (RT) to locate a requested target at a moment of targets’ reappearance.
Design. The experimental design included three withinparticipant variables: set-size (2-3), masking duration (840 ms,
1680 ms, 2520 ms) and vector condition (B, T, C, or H). Each
of the resulting 24 conditions was replicated 10 times, resulting in a total of 240 trials for each participant.
Procedure
A total of 60 different object trajectories were created, for
10 different trajectories per set-size and masking duration. The
same trajectories were used in all vector conditions. The target
pictures to be identified and their mean distance from the
screen center at a moment they reappeared were matched
across all experimental conditions.
At the beginning of each trial, the objects were displayed
stationary for 1s. After that, a black frame flashed on and off
for ten times (3s) around the target objects. All objects then
began to move in a random and continuous fashion on the
screen. The tracking period lasted for 5 s, after which the objects disappeared for one of the three masking durations.
When the objects reappeared on the screen (they no longer
moved), participants heard simultaneously a name of one target picture via the earphones. They were instructed to locate
and point the specific target with the computer mouse as
quickly as possible. The mouse cursor was initially positioned
in the center of the screen. Participants’ eye-to-screen distance
(57 cm) was controlled using a chin rest.
The baseline, track, congruent and heading conditions
were presented in separate blocks. The other variables were
randomized within these blocks. The order of blocks was
counterbalanced across participants. There was a short rest period between the blocks. The entire session took about 100
min.
RESULTS
The results supported our hypothesis. The congruent condition where the vector length corresponded to the masking
duration and thus predicted the target position upon its reappearance, resulted in the fastest identification (see Figure 1).
Before analysis, erroneous responses (1.3%) were removed from the data. A few response time outliers (0.6%)
were also removed (z < -3.29 or z > 3.29). The remaining response time data were analyzed with a 2 (Set-Size) x 3 (Masking Duration) x 4 (Vector Condition) repeated measures analysis of variance (ANOVA). A Greenhouse-Geisser correction
was applied to the p-values whenever needed.
PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 53rd ANNUAL MEETING—2009
1300
1250
1200
Baseline
Track
ms 1150
Congruent
Heading
1100
1050
1000
1680
2520
3360
Masking duration
Figure 1. Response times (ms) as a function of masking duration (ms) and vector condition (baseline, track, congruent and
heading). Vertical lines depict standard error of means (±1
SEM).
The main effect of set-size was significant, F(1, 19) =
233.53, p < .001; RTs were lengthened as the set-size increased. The main effect of masking duration was significant
as well, F(2, 38) = 10.77, p < .001; RTs were lengthened as as
a function of masking duration. The main effect of vector condition was also significant, F(3, 57) = 4.49, p < .01. Planned
comparisons were conducted where the baseline condition was
compared to other vector conditions. Comparisons indicated
that the congruent vector condition differed from the baseline,
F(1, 19) = 6.51, p < .05); RTs were faster in the congruent
vector than in the baseline condition. The other vector conditions did not differ from baseline; track, F(1, 19) = 1.68, p >.
20; heading, F < 1.
The interaction between set-size and masking duration
was significant, F(2, 28) = 5.84, p < .05); with the effect of
masking duration was more robust for set-size three than setsize two. The other interactions were not significant (Set-size
x Vector Condition, F(3, 57) = 1.87, p > .10; Masking Duration x Vector Condition, F < 1; 3-way, F < 1).
DISCUSSION
Our study examined the influence of velocity vectors on
movement extrapolation when tracking multiple invisible objects. We tested this by using the MIT task where at some
point all moving targets temporarily disappeared from view.
The participants task was to immediately locate and indicate a
requested target after it reappeared on the screen. We compared four velocity vector conditions (no vector, only a history
trail, velocity vector corresponding to the masking duration,
and constant vector length) over three target masking durations.
Our results indicate that only the vector providing complete information about the target’s future location improved
19
movement extrapolation when compared to the condition
where no visual vectors were present. Surprisingly, neither the
history trail nor the forward heading vector with constant
length (not pointing to the future location in all trials) did improve performance. If anything, the history trail condition was
slightly worse than the baseline condition (no velocity information at all).
Our results suggest that only explicit visual information
about the future locations of invisibly moving targets will help
to anticipate their locations after occlusion. On the other hand,
movement history or directional information of targets’ trajectory does little to help on movement extrapolation with multiple targets. Our results are consistent with Keane and Pylyshyn’s (2006) displacement hypothesis. According to this hypothesis, when objects disappear from view, observers maintain the last visible locations of the target objects in the visual
short term memory (VSTM). When the objects reappear, this
VSTM representation is used as a reference in relocating the
targets. This view is also supported by Oksama and Hyönä
(2008), whose model of multiple identity tracking (MOMIT)
suggests that in order to maintain an up-to-date representation
of multiple moving targets focal attention is serially switched
between the targets. When focal attention is directed to a target element its identity-location binding is refreshed and updated. And when a target becomes unattented, or in this case
invisible, its last visible and attended location is stored in the
VSTM. Therefore, relocating a recently unattended target relies on stored information on that target’s last position. Congruent vectors provide explicit location information of the targets’ position after occlusion, which information can then be
stored in VSTM for future use. This is why congruent vectors
are superior to other vector conditions in multiple identity
tracking. This may also explain why the trail condition produced the worst performance. The trail may draw observers’
visual attention toward it and thus bias movement extrapolation in the wrong direction.
REFERENCES
Berbert, A. G., & Kelley, C. R. (1962). Piloting nuclear submarines with controls that look into future. Electronics,
XXXV (June 8, 1962).
Besco, R. O. (1964). Manual attitude control systems: Vol II.
Display format considerations (Technical report to the
NASA). Culver City, CA: Hughes Aircraft Company.
Cole, H. W. (1985). Understanding radar. London: Collins.
Ellis, S.R., & Stark, L. (1986). Statistical dependency in visual
scanning. Human Factors, 28(4), 421-438.
Fargel, L. C., & Ulbrich, E. A. (1963). Predictive controllers
applied to lunar landing. Instrumentation and Control
Systems, 36, 130-131.
Grunwald, A. J. (1981). Predictor symbology in computergenerated perspective displays. In J. Lyman & A Bejczy
(Eds.), Proceedings of the 17th Annual Conference in Manual Control (NASA-JPL Pub 81-95).
Hart, S. G., & Loomis, L. L. (1980). Evaluation of the potential format and content of a cockpit display of traffic information. Human Factors, 22(5), 591-604.
PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 53rd ANNUAL MEETING—2009
Jensen, R. S. (1981). Prediction and quickening in perspective
flight displays for curved landing approaches. Human
Factors, 23, 355-363.
Kantowitz, B. H. & Sorkin, R. S. (1983). Human factors: Understanding people-systems relationships. New York:
John Wiley & Sons.
Keane, B. P., & Pylyshyn, Z. W. (2006). Is motion extrapolation employed in multiple object tracking? Tracking as a
low-level, non-predictive function. Cognitive Psychology,
52, 346-368.
Kelley, C. R. (1962). Predictor instruments look to the future.
Control Engineering, 86, 86-90
Kelley, C. R. (1968). Manual and automatic control. New
York: Wiley
Kelly, J. R. (1983). Effect of lead-aircraft ground-speed quantization on self-spacing performance using a cockpit display of traffic information. NASA TP-2194.
Kelly, J. R., & Abbott T. S. (1984). In-trail spacing dynamics
of multiple CDTI-equipped aircraft queues. NASA TM85699. Hampton, VA: NASA Langley.
Kelly, J. R. & Williams, D. H. (1983). Factors affecting intrail following using CDTI. Proceedings of the Eighteenth
Annual Conference on Manual Control, Frank L. George,
ed., (AFWAL-TR-83-3021), pp. 485-513. U.S. Air Force.
Laine, M. & Virtanen, P. (1999). WordMill Lexical Search
Program. Turku, Finland: Centre for Cognitive Neuroscience, University of Turku.
McLane, R. C., & Wolf, J. D. (1966). Symbolic and pictorial
displays for submarine control. Paper presented at the
MIT–NASA Working Conference on Manual Control,
Cambridge, MA, February 28–March 2, 1966.
Merwin, D. H., & Wickens, C. D. (1996). Evaluation of perspective and coplanar cockpit displays of traffic information to support hazard awareness in free flight (Technical
Report ARL-96-5). Savoy, IL: Aviation Research Laboratory.
Nolan, M. S. (1998). Fundamentals of air traffic control (3rd
ed.). Pacific Grove, CA: Brooks/Cole—Wadsworth.
Oinonen, K., Oksama, L., & Hyönä, J. (2009). Movement
extrapolation and identity-location binding for invisible
objects. In submission.
Oksama, L., & Hyönä, J. (2004). Is multiple object tracking
carried out automatically by an early vision mechanism
20
independent of higher-order cognition? An individual difference approach. Visual Cognition, 11, 631-671.
Oksama, L., & Hyönä, J. (2008). Dynamic binding of identity
and location information: A serial model of multiple identity tracking. Cognitive Psychology, 56, 237-283.
Palmer, E. A. (1983). Conflict resolution maneuvers during
near miss encounters with cockpit traffic displays. Proceedings of the 27th Annual Meeting of the Human Factors Society. Santa Monica, CA: Human Factors Society.
Palmer, E. A., Jago, S. J., Baty, D. L., & O’Connor, S. L.
(1980). Perception of horizontal aircraft separation on a
cockpit display of traffic information. Human Factors,
22(5), 605-620.
Palmer, E., Baty, D., & O'Connor, S. (1979). Perception of
aircraft separation with various svmbols on a cockpit display of traffic information. Proceedings of the 15th Annual Conference on Manual Control. Wright-Patterson
Air Force Base, OH.
Schneider, W., Eschman, A., & Zuccolotto, A. (2002a). Eprime user’s guide. Pittsburgh, PA: Psychology Software
Tools, Inc.
Schneider, W., Eschman, A., & Zuccolotto, A. (2002b). Eprime reference guide. Pittsburgh, PA: Psychology Software Tools, Inc.
Sanders, M. A. McCormick, E. J. (1993). Human factors in
engineering and design. New York: McGraw-Hill.
Sheridan, T. B. (1966). Three models of preview control.
IEEE Transactions on Human Factors in Electronics,
HFE-7(2), 91-102.
Snodgrass, J. G., & Vanderwart, M. (1980). A standardized set
of 260 pictures: Norms for name agreement, image
agreement, familiarity and visual complexity. Journal of
Experimental Psychology: Human Learning and Memory, 6, 174–215.
Roscoe, S. N., Corl, L., & Jensen, R. S. (1981). Flight display
dynamics revisited. Human Factors, 23, 341-353.
Wickens, C. D., Haskell, I., & Harte, K. (1989). Ergonomic
design for perspective flight path displays. IEEE Control
Systems Magazine, 9(4), 3-8.
Williams, D. H. (1983). Time-based self-spacing techniques
using cockpit display of traffic information during approach to landing in a terminal area vectoring environment. NASA-TM-84601.