J Neurophysiol 93: 2279 –2293, 2005;
doi:10.1152/jn.01042.2004.
Object Motion Computation for the Initiation of Smooth Pursuit Eye
Movements in Humans
Julian M. Wallace,1 Leland S. Stone,2 and Guillaume S. Masson1
1
Institut de Neurosciences Cognitives de la Méditerranée, Centre National de la Recherche Scientifique, Unité Mixte de Recherche 6193,
Marseille France; 2Human Information Processing Research Branch, Human Factors Research and Technology Division, National
Aeronautics and Space Administration, Ames Research Center, Moffett Field, California
Submitted 4 October 2004; accepted in final form 8 November 2004
INTRODUCTION
When an object moves across the visual field, smooth
pursuit eye movements act to stabilize its image on the retina
by smoothly rotating the line of sight at the same speed and
direction as the object. The signal driving the initiation of
smooth pursuit eye movements is dominated by the visual
motion of the target (Rashbass 1961). Reconstructing the
target’s motion is a complex task because of the aperture
problem (Wallach 1996): early stages of visual motion processing sense local changes in the image through detectors with
a limited field of view and spatiotemporal properties such that
they respond best to motion orthogonal to the local onedimensional (1D) edge (Albright 1984). As individual 1D
motion measurements are ambiguous and can be different from
Address for reprint requests and other correspondence: G. S. Masson, Team
DyVA, INCM–CNRS UMR-6193, 31 Chemin Joseph Aiguier, 13402 Marseille, France (E-mail: [email protected]).
www.jn.org
the actual two-dimensional (2D) object motion, the latter is
reconstructed by combining these piecewise local signals.
Several simple “two-stage” computational solutions have been
proposed to solve the aperture problem and compute the 2D
object motion. Vector averaging (VA) of the various 1D
motions associated with each 1D edge of the object is one
simple way of combining the 1D information. However, this
approach does not always yield the correct target motion
(Ferrera and Wilson 1990). The Intersection of Constraints
(IOC) rule (Adelson and Movshon 1982; Fennema and Thompson 1970) computes the true translation vector in velocity
space as the intersection of the different constraint lines formed
by the 1D edge motions. Finally, feature tracking (FT) strategies can also yield the correct translation vector by considering
the velocity of unchanging features of the object (such as
corners, line-ends, texture elements. . .) the luminance profile
of which is intrinsically 2D and therefore the 2D motion of
which is unambiguous (e.g., Gorea and Lorenceau 1991;
Lorenceau et al. 1993; Mingolla et al. 1992). It remains
unresolved whether any of these three rules best describes the
neural 2D motion computation underlying human perception or
behavioral performance. Indeed, more recently probabilistic
computational theories have been put forward to account for
motion integration (e.g., Koechlin et al. 1999; Rao 2004; Weiss
et al. 2002).
We have recently demonstrated that the initiation of pursuit
depends on the shape of simple, line-drawing diamonds moving along horizontal or vertical trajectories (Masson and Stone
2002). In humans, pursuit was always initiated in the direction
predicted by the vector average of the four motion signals
orthogonal to the four diamond edges. More specifically, when
this VA prediction and the actual object motion differed by
44°, tracking direction slowly evolved from the oblique VA
prediction to the cardinal veridical object-motion direction
(e.g., IOC or FT prediction) over a period of 200 ms. Furthermore, once steady-state tracking direction matched the actual
object-motion direction, transiently (100 ms) blanking the
object had no impact on the eye-movement direction, indicating that the aperture problem was solved visually once at target
motion onset and that sensorimotor feedback was not required
for pursuit to converge onto the correct target motion. Previous
studies documented the same basic finding for human motion
perception and monkey pursuit and neural responses. Using
The costs of publication of this article were defrayed in part by the
payment of page charges. The article must therefore be hereby marked
“advertisement” in accordance with 18 U.S.C. Section 1734 solely to
indicate this fact.
0022-3077/05 $8.00 Copyright © 2005 The American Physiological Society
2279
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
Wallace, Julian M., Leland S. Stone, and Guillaume S. Masson.
Object motion computation for the initiation of smooth pursuit eye
movements in humans. J Neurophysiol 93: 2279 –2293, 2005; doi:
10.1152/jn.01042.2004. Pursuing an object with smooth eye movements requires an accurate estimate of its two-dimensional (2D)
trajectory. This 2D motion computation requires that different local
motion measurements are extracted and combined to recover the
global object-motion direction and speed. Several combination rules
have been proposed such as vector averaging (VA), intersection of
constraints (IOC), or 2D feature tracking (2DFT). To examine this
computation, we investigated the time course of smooth pursuit eye
movements driven by simple objects of different shapes. For type II
diamond (where the direction of true object motion is dramatically
different from the vector average of the 1-dimensional edge motions,
i.e., VA ⫽ IOC ⫽ 2DFT), the ocular tracking is initiated in the vector
average direction. Over a period of less than 300 ms, the eye-tracking
direction converges on the true object motion. The reduction of the
tracking error starts before the closing of the oculomotor loop. For
type I diamonds (where the direction of true object motion is identical
to the vector average direction, i.e., VA ⫽ IOC ⫽ 2DFT), there is no
such bias. We quantified this effect by calculating the direction error
between responses to types I and II and measuring its maximum value
and time constant. At low contrast and high speeds, the initial bias in
tracking direction is larger and takes longer to converge onto the
actual object-motion direction. This effect is attenuated with the
introduction of more 2D information to the extent that it was totally
obliterated with a texture-filled type II diamond. These results suggest
a flexible 2D computation for motion integration, which combines all
available one-dimensional (edge) and 2D (feature) motion information to refine the estimate of object-motion direction over time.
2280
J. M. WALLACE, L. S. STONE, AND G. S. MASSON
METHODS
We used the same behavioral paradigm and the same basic stimuli
as Masson and Stone (2002), but we have now investigated a full set
of parameters such as speed, contrast, and relative weight of 1D and
2D features to better understand the 2D object motion computation
underlying smooth pursuit initiation.
Visual stimuli and behavioral paradigm
The stimuli were line-figures (line luminance: 60cd/m2) of equal
area (71deg2) but of different shapes and tilts (Beutter and Stone
2000), moving either in horizontal (rightward or leftward) or vertical
(upward or downward) directions at different speeds over a black
(⬍0.1cd/m2) untextured background. The two shapes were a square
diamond (main axes: 11.9°) and a counter-clockwise (CCW) elonJ Neurophysiol • VOL
gated and tilted diamond (28.3 ⫻ 5° parallelogram, tilted ⫾45° with
respect to vertical with internal angles of 14.5 and 165.5°). These
square and elongated diamonds have properties equivalent to those of
type I and II moving plaids, respectively (Wilson et al. 1992). For the
square diamond, the vector average of the edge motions (i.e., the
motions orthogonal to the edges) is co-linear with the object-motion
direction. For the elongated diamonds, it is biased ⫾44° away from
the object’s direction (see Beutter and Stone 2000).
Several parameters of the visual motion stimuli were changed for
both square and tilted diamonds. For each parameter, corresponding
type I and II visual stimuli were interleaved. In the first experiment,
the speed of object motion was varied from 2.5 to 20°/s. In the second
experiment, the luminance of the edges was changed. For this sole
experiment, objects were projected over a gray background of constant luminance so that object contrast was varied from 10 to 90%.
The next two experiments were designed to probe the role of 1D
(edges) and 2D (corners) motion cues. In the third experiment, the
relative salience of edges, corners, and line endings was manipulated
applying a circular Gaussian filter (␴ ⫽ 5 pixels, for square edges;
␴ ⫽ 12.5 pixels, for shorter diamond edge; ␴ ⫽ 16 pixels, for longer
diamond edge) centered either on the center of the edges or on the
corners themselves. In the fourth experiment, the line figures were
filled with a texture made of a random dot pattern with 50% dot
density. In a control experiment, we checked for a possible role of
expectation on motion computation by statically presenting either a
square or a tilted diamond for 0, 50, 100, 200, or 400 ms before
motion onset.
Observers had their head stabilized by chin and forehead rests. Each
trial started with the presentation of a stationary fixation point.
Observers were required to fixate for 300 ⫾ 150 ms within a 1 ⫻ 1°
window. The fixation point was then extinguished and a moving
object presented. In the control experiment, the object was presented
as static for a given duration together with the fixation point. The
fixation point was then extinguished as the object was set into motion.
In all cases, the object moved for 450 ⫾ 50 ms. Observers were
instructed to track the object center and trials were aborted if eye
position did not stay within 2° of the object center (⬃2% of trials,
independent of condition). All conditions were randomly interleaved
to minimize cognitive expectations and anticipatory pursuit. We
collected ⬃60 trials per condition and observer over several days.
Eye-movement recordings and analysis
Eye movements were recorded from three observers (1 naı̈ve) using
methods described in detail elsewhere (Masson et al. 2000). Briefly, a
PC running REX controlled stimulus presentation and data acquisition. Stimuli were generated with an SGI Octane workstation and
back-projected along with the red fixation point onto a large translucent screen (80 ⫻ 60°) using a video-projector (1,280 ⫻ 1,024 pixels
at 76 Hz). The position of the right eye was sampled at 1 kHz using
the scleral search-coil technique (Collewijn et al. 1975). Data were
then stored on disk for off-line analysis.
Eye-position data were linearized, smoothed with a spline interpolation (Busettini et al. 1991) and then differentiated to obtain
vertical and horizontal eye-velocity profiles. After visual inspection
using an interactive graphic software, we used a conjoint velocity and
acceleration threshold to detect and remove saccades (Krauzlis and
Miles 1996). The latency of each trial was computed for both
horizontal and vertical eye-velocity profiles using an objective method
(Krauzlis and Miles 1996; Masson and Castet 2002). Trials were then
realigned so that time 0 was now response onset (Fig. 1A). To
compute tracking direction, mean vertical and horizontal eye positions
were computed over 40-ms bins, regularly spaced from 0 (response
onset) to 320 ms (steady-state tracking). The change in eye position
between successive bins was then computed for each trial, for both
type I and II stimuli. Thus we generated new time bins representing
the mean direction over an 80-ms period. For each object motion
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
oblique moving lines, Pack and Born (2001) found that monkeys initiate ocular tracking in the direction orthogonal to the
lines. Lorenceau and colleagues (Castet et al. 1993; Lorenceau
et al. 1993) found that humans manifest similar biases in
motion direction and speed estimation for short presentation of
a single moving tilted line. Moreover, the alignment of the
initial voluntary pursuit with the VA prediction is consistent
with our previous work on ocular-following responses in humans. We showed that reflexive tracking is always initiated in
the VA direction of all 1D motion signals, albeit with a latency
of ⬃85 ms (shorter than the usual 100-ms latency of voluntary
pursuit) but that the later portion of the ocular responses is
influenced by a number of factors including the presence of
unambiguous 2D motion signals moving in the actual patternmotion direction (Masson and Castet 2002; Masson et al.
2000).
Our preliminary brief report (Masson and Stone 2002) about
pursuit of line-drawing objects did not fully resolve a number
of important questions. First, our earlier estimate of the time
course of pursuit direction was ⬃90 ms, which is close to the
internal latency of the visuomotor feedback loop (Goldreich et
al. 1992), suggesting that negative feedback loop might still be
playing a role in reducing the initial tracking error. Herein, our
goal was to address this question more closely by offering a
better quantification of the time course of pursuit direction and
by examining it over a large range of stimulus parameters.
Second, we previously observed the initial VA bias in pursuit
initiation only using very slow speeds and high contrast. Biases
in perceived direction were observed under similar conditions
but not with high speed and strong contrast (Lorenceau et al.
1993). By varying target speed and contrast, we could determine whether or not the initiation of pursuit follows the same
pattern and therefore shares the same solution to the aperture
problem as motion perception, as has been shown for steadystate tracking (Beutter and Stone 2000; Stone et al. 2000).
Third, we examined the role of local 1D edges and 2D features
(corners) by manipulating their relative salience in the images
and measuring the effect on the initial pursuit direction error
and its time course of recovery. An additional goal of this last
experiment was to examine to what extent our findings could
be extended to more complex objects, such as textured surfaces
where multiple motion cues coexist that could improve the
accuracy of visual motion signal driving the initiation of
pursuit.
2D OBJECT MOTION FOR PURSUIT EYE MOVEMENTS
2281
direction, the coordinate system was rotated so that object-motion
direction was equal to zero and the vector average prediction was of
⫹44°. Thus tracking directions could be averaged across different
object motion directions.
Figure 2A plots these directions, in bins spaced 40 ms apart
centered from 40 to 280 ms (i.e., the 1st bin centered on 40 ms
represents direction from 0 to 80 ms, the final bin centered in 280 ms
represents the direction from 240 to 320 ms). For each bin, tracking
direction errors (⌰e) were computed by subtracting the mean tracking
direction obtained for type I diamonds from that obtained for type II
diamonds (Fig. 2B). This estimate reflects the biased tracking direction of responses to tilted diamonds and the relationship between time
and tracking direction errors reflect the temporal dynamics of smooth
eye movements to 2D object motion. This relationship was then fit
with the sum of two exponential functions using the following
formula
⌰ e ⫽ A关exp共t/␴i兲 ⫹ exp共 ⫺ t/␴d兲兴 ⫹ B
where ␴i and ␴d are time constants for the early (growing) and late
(decaying) exponential functions, respectively (Fig. 2C). The
goodness of fit was estimated by the r2 values, which ranged
between 0.61 and 0.99 [0.9562 ⫾ 0.0607 (SD)] and were always
significant (P ⬍ 0.05). The estimated peak of the function indicates
the maximal deviation of the initial ocular response. The ␴d and B
parameters give a precise estimate of the time course over which
this initial tracking direction error converges toward the true
object-motion direction and of the residual error once steady-state
pursuit is reached. These parameters were then compared across
conditions to investigate the dependence of the temporal dynamics
of ocular responses on visual stimulus parameters such as speed,
contrast, and so on. Of particular interest is the tracking direction
error at the point in time equal to the mean estimate of the vertical
and horizontal latencies (dashed vertical line). This value indicates
by how much the initial tracking bias has been reduced before the
closing of the oculomotor control loop.
J Neurophysiol • VOL
FIG. 2. Computation of tracking direction errors. A: relationship between
mean tracking directions for a rightward motion of either type I (F) or type II
(E) diamonds. Tracking directions are computed over 40-ms bins. - - -, the true
object-motion direction (0°) and the vector average prediction (44°) for type II
diamonds. B: tracking direction errors are computed by subtracting the tracking
directions observed for the type I to the type II diamonds. C: the average
tracking direction errors across 4 motion directions are fitted with an doubleexponential function. This descriptive function captures the sharp initial
increase in the tracking direction error, followed by a regular decay toward the
asymptotic value corresponding to the object motion direction. Parameters of
interest are the peak direction error and its time of occurrence as well as the
decay time constant (␶).
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
FIG. 1. Temporal dynamics of pursuit eye movements. A: horizontal (ėh) and vertical (ėv) velocity profiles of smooth pursuit responses to type I and II moving
diamonds. Eye movements have been realigned relative to pursuit onset. Rightward (leftward) type II moving diamond elicited large transient vertical components
in the upward (downward) direction. B: distribution of tracking direction errors across trials for 3 different time bins and each type of diamond. Vertical dotted
lines indicate the object motion direction (defined as 0°) and the vector average direction (⫹44° with respect to the true object-motion direction).
2282
J. M. WALLACE, L. S. STONE, AND G. S. MASSON
RESULTS
Figure 1A illustrates the horizontal and vertical eye velocity
profiles of smooth pursuit responses to either type I or II
moving diamonds, realigned to eye-movement onset. As previously reported by Masson and Stone (2002), initial tracking
of type II diamonds is along an oblique direction, as evidenced
by the presence of an initial eye acceleration in both horizontal
and vertical directions (Fig. 1A, right). A transient vertical
component reached its peak ⬃150 ms after pursuit onset and
then progressively declined to 0 in ⬃150 ms. Thus ⬃300 ms
after pursuit onset (⬃420 ms after target motion onset), the
tracking direction was now aligned with the pure object-motion
direction. This transient vertical component was absent in
responses to type I diamonds (Fig. 1A, left). To examine this
phenomenon more thoroughly than in our preliminary report,
the tracking error analysis was done on a trial-by-trial basis to
determine how consistent the initial bias was across trials.
Figure 1B plots the distributions of tracking direction for three
different time bins spanning the time period of the phenomeJ Neurophysiol • VOL
non (see METHODS). It is evident that for type I diamonds (dark
bars), initial tracking is aligned with object-motion direction.
Over the initial eye acceleration period, the distribution narrowed, but the average tracking direction did not change. For
the type II diamond (white bars), the whole initial distribution
is shifted toward the VA prediction. As tracking becomes more
reliable across trials, the whole distribution progressively narrows and shifts toward the true object-motion direction (0°).
Overall, it is evident that the initial responses were consistent
across trials. In particular, bimodal distributions were never
observed for any of our three observers. Interestingly, the
variance of the initial distributions for type I and II were
similarly broad, although the type II variance appears somewhat larger for the early bins.
Figure 2A plots the time course of the tracking direction
averaged across trials (see METHODS). The mean tracking
direction stayed close to the true object-motion direction
(0°) for the type I diamond (closed symbols). However, for
the Type II diamond the initial tracking direction grew to
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
FIG. 3. Effects of object speed. A: for each subject, mean horizontal (ėh) and vertical (ėv) velocity profiles of ocular tracking to type II diamond drifting
rightward at 5 different speeds indicated by the numbers at the right end of the curves. B, top: mean tracking direction errors, across 4 motion directions, for
a 20°/s motion stimulus, as a function of time. —, best-fitting double-exponential functions. Bottom: the same data obtained with different target speeds. - - -,
the mean pursuit latency, estimating the time at which the oculomotor loop closes. Thus the yellow shaded area correspond to the open-loop period of the responses.
2D OBJECT MOTION FOR PURSUIT EYE MOVEMENTS
TABLE
2283
1. Dependence of pursuit direction on object-motion speed
Speed, °/s
GM
2.5
5
10
15
20
Error max, °
T peak
34
32
40
41
41
Latency, ms
62
70
70
67
68
126
128
124
121
119
␶, ms
62
71
70
68
69
38 ⫾ 4
26
25
37
38
32
67 ⫾ 3
46
51
56
55
50
123 ⫾ 3
122
123
119
116
115
68 ⫾ 3
46
51
56
55
50
2.5
5
10
15
20
32.6 ⫾ 6
29
33
38
38
39
51.6 ⫾ 4
104
98
78
78
76
119 ⫾ 3.5
127
118
118
117
114
52 ⫾ 4
104
41
78
77
33
Mean
35.4 ⫾ 4.3
86.8 ⫾ 13.2
118.8 ⫾ 4.4
66.6 ⫾ 29.2
Average
34.9 ⫾ 5.2
68.6 ⫾ 16.7
120.6 ⫾ 4.5
62.1 ⫾ 17.6
29.73 ⫾ 3.76
30.26 ⫾ 4.22
38.06 ⫾ 1.87
38.92 ⫾ 1.94
37.48 ⫾ 4.47
70.67 ⫾ 29.96
73.00 ⫾ 23.64
68.00 ⫾ 11.14
66.67 ⫾ 11.50
64.67 ⫾ 13.32
124.82 ⫾ 2.69
122.97 ⫾ 4.66
120.37 ⫾ 3.00
118.06 ⫾ 2.74
115.94 ⫾ 2.56
70.96 ⫾ 29.72
54.17 ⫾ 14.97
68.18 ⫾ 11.01
66.92 ⫾ 11.16
50.68 ⫾ 18.06
Mean
IV
Mean
2.5
5
10
15
20
⬃30° before slowly decaying (open symbols). Tracking
directions for type I and II stimuli were not distinguishable
⬃250 ms after pursuit onset. Figure 2B plots the tracking
direction errors as computed by subtracting results of type I
from type II for a given object direction. In Fig. 2C, the data
points are now the tracking error averaged across the four
possible object directions, and — shows the best-fitting
double exponential function (see METHODS). This plot shows
that the tracking direction error reached a maximum ⬃80 ms
after pursuit onset and then decayed over ⬃200 ms to reach
a value where eye and object trajectories appropriately
coincide.
In the three subjects, we ran a control experiment to check if
expectation or anticipation could explain these results. We ran
an experiment where the same stimuli were first presented as
stationary objects (for a number of different durations), and
then set into motion for 450 ms. We found that static presentation had no impact whatsoever on pursuit initiation. With the
same three subjects, we then conducted a set of experiments to
investigate the effect of varying a number of visual parameters
on the initial pursuit response.
Dependence on object-motion speed
Figure 3A shows, for each subject, the vertical and horizontal velocity profiles obtained with type II objects moving
rightward with speeds ranging from 2.5 to 20°/s. Increasing the
speed of the object motion resulted in an increase of the initial,
horizontal eye velocity. The initial transient in the vertical
direction was similarly scaled as target speed increased, and,
for all target speeds, this vertical component was reduced to 0
after ⬃300 ms of pursuit. Figure 3B, top, illustrates for each
J Neurophysiol • VOL
subject the relationship between the mean tracking error
(across trials and object-motion directions) and time after
pursuit onset, for 20°/s object speed. The - - - indicates the
mean latency values of the responses, and therefore the yellow
shaded areas plot observer-specific estimates of the open-loop
period of pursuit. One can see that the peak of the tracking
error and associated beginning of the exponential decay occurred before the closing of the oculomotor feedback loop.
Indeed, the mean peak error was reached at 68.6 ⫾ 16.7 ms
(⫾SD across subjects), while the mean latency (a measure of
the open-loop duration) (see Lisberger and Westbrook 1985)
was 120 ⫾ 4.3 ms. Figure 3B, bottom, shows, for all three
subjects, the time course of their mean tracking direction error
for all object speeds tested. It is evident that the maxima of the
best-fitting functions occur at the same point in time and that
the decay rates of the error were similar across speeds. However, the two slowest target speeds (2.5 and 5°/s) did yield
smaller maximum direction errors.
Table 1 summarizes for each subject the best-fitting parameters of the double exponential functions illustrated in Fig. 3 by
the continuous lines. The mean peak direction errors ranged
from 29.7 ⫾ 3.8° (at 2.5°/s) to 38.9 ⫾ 1.9° (at 15°/s) with a
tendency toward larger errors for faster object motions (⬎10°/s).
A one-way repeated-measures ANOVA showed a significant
effect of target speed on peak direction error [F(2,4) ⫽ 15.22,
P ⬍ 0.008]. Figure 4A shows this relationship for each subject,
where increasing the target speed between 2.5 and 10°/s
yielded larger initial bias, whereas no clear increase was
observed for speeds ⬎10°/s. At these higher speeds, the peak
bias was, on average, ⬃39°, which is only 5° away from the
vector average prediction (44°). However, the time to peak for
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
2.5
5
10
15
20
Mean
JW
2284
J. M. WALLACE, L. S. STONE, AND G. S. MASSON
the direction error was found to be essentially constant across
all target speeds, ranging from 65 ⫾ 13 to 73 ⫾ 24 ms.
Therefore the correction of the direction error is already well
underway before the end of the open-loop period with the
magnitude of the correction during this period ranging from
⬃30% at the lowest speed tested to ⬃17% at the highest speed.
The decay of the tracking error was not affected by the target
speed. Figure 4B plots the time constants of the tracking error
decay for each speed and each subject. No systematic trend
was found over the investigated range of target speeds [1-way
repeated-measures ANOVA, F(2,4) ⫽ 0.75; P ⫽ 0.59]. Thus
the time course of the decaying initial tracking error appears to
be rather independent of target speed, averaging from 54 ⫾ 15
(at 5°/s) to 68 ⫾ 11 ms (at 10°/s). In brief, our results show that
target speed has a limited, but significant, impact on the
magnitude of the initial tracking error but no effect on its
temporal dynamics. While object speed did not modulate the
dynamics of the effect, the fact that the reduction of the initial
error began before the closing of the visuo-motor control loop
was evident across all speeds tested.
Effects of stimulus contrast
Next, we investigated the effect of target contrast on pursuit
initiation. Figure 5 shows the results obtained in the same three
J Neurophysiol • VOL
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
FIG. 4. Effects of object speed on pursuit dynamics. A: best-fitting estimates of peak direction errors, as a function of object-motion speed, for each
subject. - - -, the vector average prediction, 44° away from the actual objectmotion direction. B: for the same 3 subjects, the best-fitting decay time
constants are plotted against object motion speed.
observers in the same format as Fig. 3. For all target contrast
values, the initial transient onset of the vertical component can
be seen, albeit with different amplitudes across subjects. As we
used a relatively slow target (10°/s), the amplitude of this
transient component was particularly small in subject JW but
consistent with the results plotted in Fig. 3A, middle. For all
subjects, at the lowest contrast, initial horizontal eye acceleration was reduced, the transient vertical eye onset was larger
(producing a larger deviation of the initial tracking direction
toward the oblique axis), and the initial tracking error took
longer to subside. Furthermore, horizontal eye velocity during
steady-state tracking was also slightly reduced. Figure 5A, top,
illustrates the mean tracking direction errors as a function of
time for the highest contrast target (90%). Results were identical to those found in the previous experiments (compare with
Fig. 3B). The bottom panels plot the same relationship, but
now for all five contrasts tested. The light blue curves illustrate
the time course of the tracking error for the lowest contrast
(10%). It is evident that this lower contrast object produced a
larger peak tracking direction error and a slower decay toward
the true 2D object trajectory for all three observers. On average, across target-motion directions, this initial tracking error
was not fully resolved after 300 ms of pursuit, indicating a
large residual mis-estimation of the actual 2D target trajectory.
For all other contrasts, the initial tracking error decays with a
faster time constant, and it was resolved ⬃200 ms after pursuit
onset. The - - - indicates the mean pursuit latency across
contrast values, as no significant effect of target contrast was
found on the response onset (see Table 2). The yellow shaded
area thus indicates the mean open-loop period (106 ⫾ 5 ms,
averaged across subjects and conditions). The latencies are
somewhat lower than the previous experiment, suggesting an
effect of the mean luminance (gray background) on pursuit.
For all contrasts tested, the direction error begins to decay
before the end of the open-loop period.
Figure 6A shows the raw direction error computed in three
time bins as a function of contrast for each of the three
observers. The three time bins span a range from before and
after the maximum error and before and after the end of the
open-loop period. The error decreases as contrast is increased
for all three time bins. This effect is similar in the two earlier
bins and somewhat larger in the latest bin. Figure 6, B and C,
summarizes the relationship between the two main parameters
of interest (the peak direction error and the time constant of
error decay) of the double-exponential functions and target
contrast for each subject. Corresponding values (with the other
estimated parameters) are given in Table 2, together with mean
values across subjects. Figure 6B shows that for each subject,
increasing target contrast resulted in a smaller peak direction
error. At very low contrast (10%), the initial tracking direction
perfectly matched the vector average prediction (44°, - - -) for
each subject (mean across subjects: 43 ⫾ 3°). The initial
tracking bias was reduced to ⬃32 ⫾ 2° when increasing the
target contrast from 10 to 40%. Further increase in contrast did
not lead to obvious changes in the initial tracking direction,
which reached an asymptote around 30°. A one-way repeatedmeasures ANOVA indicated that the relationship between peak
direction error and contrast was highly significant [F(2,4) ⫽
37.94; P ⬍ 0.0001].
2D OBJECT MOTION FOR PURSUIT EYE MOVEMENTS
2285
Unlike target speed, target contrast had some impact on the
temporal dynamics of the direction error correction, as shown
in Fig. 6C. By the end of the open-loop period, the initial error
was reduced by only 8.5% for the lowest contrast condition but
by 32 ⫾ 1% for the higher contrast conditions, suggesting an
increase in the time to correction for the lowest contrast
needed. For all three subjects, increasing target contrast elicited
a shortening of the decay time constant from 168 ⫾ 84 ms (at
10% contrast) to an asymptotic value ⬃60 ms for contrasts
⬎40%. This relationship was highly significant [1-way repeatedmeasures ANOVA, F(2,4) ⫽ 9.141; P ⬍ 0.0044]. Overall, this
experiment indicated that at 10% target contrast, the initial tracking direction is aligned with the VA prediction and a longer time
is required (compared with higher contrast conditions) to reduce
this larger initial error and to align the eye and object-motion
trajectories. In comparison, at higher contrast values both the
initial error and time to correction are reduced, increasing the
contrast from 10 to 53% yielded a 26% decrease in the peak
direction error and a 280% decrease in the decay time constant.
J Neurophysiol • VOL
Relative contribution of edges and corners
Changing either target speed or contrast has a profound
effect on the reliability of local motion, either 1D (edges) or 2D
(corners). To further probe the relative contribution of these
different types of local motion cues, we filtered the object
image with circular Gaussian filters centered either on the
corners or on the center of each of the four edges. Centering the
filter onto the edges decreases the contribution of 1D edge cues
and also adds eight new smoothed, line-ending features. Therefore in this “filter on edge” condition there are in total 12
line-ending features (the 4 corners plus the 8 newly introduced
endings). Centering the filters on the corners removes the
corners and introduces eight new smoothed, line-endings features at each end of the four disconnected edges. Therefore in
this “filter on corner ” condition there are in total eight
line-ending features. Introducing new 2D “intrinsic” features
moving in the true object motion direction by filtering corners
is somewhat similar to the effect of masking corners with
invisible occluders parallel to the object trajectory (Shiffrar et
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
FIG. 5. Smooth pursuit responses to a 10°/s object motion: effects of object contrast. Layout is the same as for Fig. 2. A: mean velocity profiles of pursuit
eye movements driven by a type II diamond drifting rightward for a range of object contrasts. For this experiment, the background was set to a gray level
corresponding to the mean luminance level. B: mean tracking direction errors as a function of time for a single contrast condition (90%, top) and for a range
of object contrasts (bottom). The same color code is used for A and B.
2286
TABLE
GM
JW
IV
2. Dependence of pursuit direction on object contrast
C, %
Error max, °
T peak, ms
Latency, ms
␶, ms
10
30
53
72
90
10
30
53
72
90
10
30
53
72
90
10
30
53
72
90
46
39
34
33
32
43
34
32
29
33
41
31
30
29
31
43.0 ⫾ 2.6
34.6 ⫾ 4.3
32.0 ⫾ 2.3
30.5 ⫾ 2.4
32.0 ⫾ 1.0
76
55
52
50
51
63
45
41
40
39
106
83
79
72
72
81.7 ⫾ 22.1
61.0 ⫾ 19.7
57.3 ⫾ 19.6
54.0 ⫾ 16.4
54.0 ⫾ 16.7
114
109
107
105
104
101
101
100
99
99
112
109
108
107
106
109.23 ⫾ 7.03
106.56 ⫾ 4.66
104.98 ⫾ 4.15
103.83 ⫾ 3.79
103.00 ⫾ 3.61
156
57
52
50
52
91
45
41
41
40
257
148
85
92
94
168.0 ⫾ 83.8
83.7 ⫾ 56.3
59.6 ⫾ 22.8
61.0 ⫾ 26.9
62.0 ⫾ 28.3
al., 1995). Filtered objects were interleaved with unfiltered
objects, for both type I and type II stimuli. In the “no filter”
condition as in the previous experiments, there are only four
line-ending features (corresponding to the corners of the object). Figure 7A illustrates the horizontal and vertical velocity
profiles obtained for each subject with each type of stimulus,
always moving rightward at 10°/s. With the unfiltered type II
diamond, a large initial, transient, vertical eye acceleration was
again found, replicating the results from experiments 1 and 2.
When filters were applied to the centers of the edges (filter on
edge, green curve), the initial vertical transient was greatly
reduced in at least two subjects, for this particular object
motion direction. When filters were applied to the corners
(filter on corner condition, blue curves), the initial transient
vertical response was reduced relative to the no filter condition.
Figure 7B plots the mean tracking direction error for the
unfiltered condition only (top) or for all three conditions
together (bottom). It can be seen that peak direction error was
largely reduced in the filter on edge condition and arrived
earlier in the temporal course. The initial tracking error was
then reduced to 0 within ⬃200 ms of pursuit, that is ⬃100 ms
earlier than in the fully visible condition. The results from the
filter on corner conditions were intermediate between the two
other condition, both in terms of peak direction error and time
course.
Table 3 summarizes the best-fitting parameters obtained for
each subject and each image filtering condition. It is worth
noting that smooth pursuit eye movements were initiated in
response to each image type with the usual latencies (121.8 ⫾
7.4 ms on average), and again, the reduction of the initial error
was initiated well before the closing of the loop (58.6 ⫾16.1
ms on average). Pairwise comparison revealed no significant
difference between image types. Mean values of the parameters of interest are plotted in Fig. 8. For the unfiltered condition, mean peak direction error was of 33.3 ⫾ 2.5°, a value
similar to those found in the first two experiments. Removing
edge information (and thus adding unambiguous 2D features)
significantly reduced this peak direction error to 12.7 ⫾ 2.3°
[t-test, t(4) ⫽ 10.47; P ⬍ 0.0004]. By comparison, adding 2D
J Neurophysiol • VOL
features alone induced a smaller, albeit significant reduction in
the peak direction error [mean : 22.7 ⫾ 3.1°, t-test, t(4) ⫽ 4.7,
P ⫽ 0.01]) as compared with the no filter condition. By the
closing of the loop, these peak errors were already reduced by
20.17% (no filter), 65.37% (filter on edge), and 47.67% (filter
on corners).
The mean time constants of the exponential decay of the
tracking direction error showed similar differences to those of
the peak direction error. With the unfiltered stimulus, the mean
␶ was of 72 ⫾ 17 ms, similar to those reported above. The
shortest ␶ values were found in the filter on edge condition
[mean: 49 ⫾ 13 ms, t-test, t(4) ⫽ 12.4; P ⬍ 0.002] as
compared with no filter. The filter on corner condition elicited
an intermediate time course [mean: 56 ⫾ 12 ms, t-test, t(4) ⫽
5.98; P ⬍ 0.003], as compared with the unfiltered condition.
Pursuing texture-filled objects
In this experiment, we manipulated the proportion of 1D
(edges) and 2D (features) motion signals by filling the object with
a high-density random pattern of small (0.2 ⫻ 0.2°) dots. For
comparison, these textured diamonds were interleaved with the
line-drawing figures and with a solid gray rhombus of same
geometry. This latter filled stimulus controls for the effect of
FIG. 6. Effects of object contrast on pursuit dynamics. A: relationships
between mean (across direction) tracking direction errors and target contrast,
for 3 different time bins and each subject. B: best-fitting peak direction errors,
as a function of object contrast, for each subject. - - -, the vector average
prediction, 44° away from the actual object-motion direction. C: for the same
3 subjects, the best-fitting decay time constants are plotted against contrast.
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
Mean
J. M. WALLACE, L. S. STONE, AND G. S. MASSON
2D OBJECT MOTION FOR PURSUIT EYE MOVEMENTS
2287
having luminance information distributed within the object. Figure 9A illustrates, for the three subjects, the vertical and horizontal
mean velocity profiles obtained with each stimulus, for 10°/s
rightward motion. Empty and uniformly filled objects yielded
identical ocular responses both for the regular horizontal component and for the transient vertical component. However, when
filled with a random pattern texture, the same object gave no
transient component along the vertical direction and brisker initial
eye acceleration along the horizontal direction. Figure 9B, top,
TABLE
3. Effects of Gaussian filtering on pursuit dynamics
Filters
GM
No filter
Filter on
Filter on
JW
No filter
Filter on
Filter on
IV
No filter
Filter on
Filter on
Mean No filter
Filter on
Filter on
corner
edge
corner
edge
corner
edge
corner
edge
Error max,° T peak, ms Latency, ms
␶, ms
36
66
115
65
26
64
121
65
14
52
122
52
33
57
129
58
22
42
131
42
14
34
132
35
31
91
111
91
20
61
117
62
10
60
118
61
33.3 ⫾ 2.5 71.3 ⫾ 17.6 118.6 ⫾ 7.6 71.7 ⫾ 17.2
22.7 ⫾ 3.1 55.7 ⫾ 11.9 122.9 ⫾ 5.9 56.0 ⫾ 12.2
12.7 ⫾ 2.3 48.7 ⫾ 13.3 124.3 ⫾ 5.8 49.0 ⫾ 12.8
J Neurophysiol • VOL
shows the mean tracking direction error as a function of time, for
a line-drawing figure moving at 10°/s. Yellow shaded areas
indicate the open-loop period from the latency measurements
obtained in this experiments (average latency: 115 ⫾ 7 ms), and
again the peak direction error was reached (63 ⫾ 13 ms) before
the oculomotor control loop was closed. Results are similar to
those observed with similar visual conditions in the previous
experiments. Bottom plots show the same data but now together
with the mean tracking direction errors observed with a texturefilled and a uniformly filled rhombus. First, there is no difference
between the transiently biased responses to empty and filled
objects. Second, for all three observers, the transient bias toward
the VA prediction is absent (or at least dramatically reduced) with
the texture-filled objects. The decay time constant could not be
estimated as the time course of the tracking direction error was
almost flat, remaining close to the veridical object-motion direction across time.
DISCUSSION
This study extends our previous finding that the initiation of
smooth pursuit eye movements reflects the dynamics of early
motion integration. By manipulating the contrast, speed, texture filling, and the shape of simple, line-drawings targets, we
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
FIG. 7. Smooth pursuit responses to a 10°/s object motion: effects of edges, corners, and line endings. Layout is the same as for Figs. 3 and 5. A: mean velocity
profiles of pursuit eye movements driven by 3 different types of tilted diamonds drifting rightward. Insets: the 3 different types of images used with the color
code for the velocity profiles. B: mean tracking direction errors as a function of time for a single control condition (unfiltered diamonds, top) and for the 3 different
filtered image types (no filter, filter on edge, filter on corner, bottom). The same color code is used for A and B.
2288
J. M. WALLACE, L. S. STONE, AND G. S. MASSON
Pursuit as a probe of the temporal dynamics of
motion integration
FIG. 8. Effects of edges and corners upon pursuit dynamics. Top and
bottom: the mean (⫾SD, across subjects) Peak direction errors and time
constants, respectively, for each image type.* and ** indicate 0.05 and 0.001
significance level for pair-wise comparisons.
were able to measure the influence of these parameters on the
tracking direction accuracy, and in particular to examine the
time course over which the neural direction signal driving
pursuit develops to match the veridical motion direction of the
pursued object. Our main findings are the following. First, the
initiation of pursuit to empty line-drawing objects is launched
in the direction of the VA solution computed by averaging the
local motions orthogonal to the edges, despite the fact that for,
type II diamonds, this estimate is ⬃44° away from the actual
2D trajectory of the object. This initial mis-estimation gradually disappears within ⬃300 ms such that steady-state tracking
is then aligned with the true object-motion direction. However,
we found that the reduction of the initial tracking error starts
before the closing of the oculomotor control loop, indicating
that the initial refinement of the direction estimate is purely
visually based, as opposed to being a consequence of a visuomotor correction driven by negative visual feedback. Second,
the phenomenon is clearly manifested and has similar dynamics over a large range of target speeds and contrasts although
the peak error and decay time constant do depend on object
J Neurophysiol • VOL
Because pursuit is a closed-loop negative feedback system,
when interpreting the time course of the initial smooth eyemovement responses to visual motion, one must be vigilant to
dissociate the dynamics of the underlying visual motion signals
driving pursuit (the open-loop sensory drive) from that caused
by the dynamic properties of negative feedback (the sensorimotor feedback signal). In our earlier study (Masson and Stone
2002), we addressed this issue indirectly by transiently blanking the visual stimulus during steady-state pursuit. The postblanking re-appearance of the stimulus did not engender a
re-initiation of the tracking error. Instead, the steady-state
response to the same type II diamond motion immediately
drove pursuit back in the correct object-motion direction.
While this result did show that the steady-state pursuit response
is latched by an initial visual decision process that does not
need to be re-performed, this finding did not resolve the more
pointed question of whether or not the initial pursuit dynamics
were dominated by feedforward visual or feedback sensorimotor signals. Herein, we resolved this question by demonstrating that the initial tracking error begins to converge toward
the veridical direction before the closing of the oculomotor
control loop, that is, before feedback signals could possibility
affect the response. The estimated direction error typically
peaked after only ⬃60 ms, which is ⬃30 – 60 ms before
feedback is thought to affect pursuit (Lisberger and Westbrook
1985). At the time of closing the feedback loop, the initial
direction was already reduced by ⬃20 –30%. Moreover, under
high-contrast and high-speed conditions, the time constant of
the decaying tracking error was as fast as ⬃60 ms, which is
about twice as fast as the internal time constant of pursuit
pathways (Goldreich et al. 1992). Finally, at low target contrast, initial eye speed was reduced by ⬃20%, but the peak
direction error was increased by ⬃15%, and the mean time
constant increased by ⬃200%. This indicates that, at low
contrast, the much longer time it takes to converge to the true
object-motion direction cannot be accounted for by the more
sluggish pursuit responses. We therefore can conclude with
confidence that the decay of the tracking error is largely
reflective of the time course of the visual refinement of the
neural object-motion signal and is largely independent of any
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
contrast and speed. In particular, lower contrast and higher
speed produce larger initial deviations toward the VA solution.
Third, varying the relative strength of 1D (edges) and 2D
(corners) motion cues has a profound impact on the initial
tracking behavior. The main finding herein is that when enough
nonambiguous 2D cues (such as surface texture elements) are
provided, the initial tracking responses is no longer aligned
with the VA prediction, but instead follows immediately—and
at the same latency—the true object motion trajectory. In
summary, this study shows that the time course of pursuit onset
is a powerful tool that can be used to probe the dynamics of
human cortical motion processing. More specifically, our results shed light on the relationship between the visual signals
driving perception and pursuit, provide insight into the dynamics of the neural representation of object motion direction, and
challenge current models of primate cortical motion processing
as well as of pursuit control.
2D OBJECT MOTION FOR PURSUIT EYE MOVEMENTS
2289
pursuit-driven feedback signals. As an extension of the work of
Lisberger and Westbrook (1985) with small spots, our results
show that examination of the initial open-loop pursuit response
to judiciously constructed stimuli is a powerful probe to reveal
the temporal dynamics of the neural processing of 2D visual
motion signals.
Comparison with perceptual studies
Previous attempts to measure the temporal dynamics of
motion integration have examined the effects of stimulus
duration on perceived direction. Apart from the agreement that
for very short durations (⬍100 ms), the perceived direction
matches the VA solution (Lorenceau et al. 1993; Yo and
Wilson 1992), there is no clear consensus on the time course
over which the percept shifts to the actual 2D direction of the
pattern/object. For high-contrast type II plaids, human observers report seeing the correct pattern direction for durations
⬎150 ms (Yo and Wilson 1992). For multiple moving tilted
lines, for which the direction of motion orthogonal to the line
differs from the true motion direction of the line endings,
asymptotic performance was reached with stimulus durations
J Neurophysiol • VOL
⬎400 ms (Lorenceau et al. 1993). Our pursuit findings are
broadly consistent with these perceptual findings. For a better
comparison, we fit a single-exponential function to the psychophysical data from Lorenceau et al. (1993) and Yo and Wilson
(1992) and found a large variability in estimated decay time
constants: ⬃200 and ⬃30 ms, respectively. The latter estimate
reflects the change in the actual perceived direction over time
and is therefore more directly comparable to our behavioral
results. Thus the psychophysical temporal dynamics are in
reasonable agreement with the present results, consistent with
a shared visual direction signal for both perception and pursuit
(Beutter and Stone 2000; Krukowski and Stone 2004; Stone and
Krauzlis 2003; Stone et al. 2000; Watamaniuk and Heinen 1999).
For measuring response dynamics, the approach of examining pursuit responses provides however a number of advantages over forced-choice psychophysical techniques. First, it is
much more time efficient in that a single pursuit trial provides
data for the entire range of time points, whereas forced-choice
or adjustments psychophysical measures must be repeated for
each time point studied. Second, even with brief stimulus
presentations, the psychophysical decision may not be limited
to the stimulus duration alone: observers have until they
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
FIG. 9. Smooth pursuit responses to a 10°/s object motion: effects of surface 2D texture elements. Layout is the same as for Figs. 3, 5, and 7. A: mean velocity
profiles of pursuit eye movements driven by 3 different types of tilted diamonds drifting rightward. Insets: the 3 different types of images used with the color
code for the velocity profiles. B: mean tracking direction errors as a function of time for a single control condition (line-figure diamonds, top) and for the 3
different fill patterns (none, uniform, and textured, bottom). The fits obtained with the textured diamond were not significant. The same color code is used for
A and B.
2290
J. M. WALLACE, L. S. STONE, AND G. S. MASSON
provide a response to continue processing the visual information collected during the stimulus presentation. Third, while
forced-choice psychophysics provide a measure of the signalto-noise value inferred from the percentage of trials where
object-motion direction is reported, pursuit data simultaneously provide separate measures of signal (the mean pursuit
direction) and noise (the variance in that direction) that allow
for clearer interpretation. Indeed, in this study, we found a
smooth transition over time for the ensemble of trials as shown
by the gradual shift of a monotonic distribution of tracking
directions (Fig. 1B). This is easily distinguishable from a
stochastic shift between discrete VA and IOC decisions, which
would have yielded bimodal distributions of pursuit directions.
Our quantitative analysis revealed three main characteristics
of motion integration. First, at low contrast (10%) and high
speed (20°/s), the peak direction error is almost identical to the
VA solution (⬃43 and 38°, respectively). Second, when the
surface object is filled with high-spatial frequency 2D local
information, the peak direction error is always reduced to
small, residual biases (⬍5°), meaning that the earliest part of
pursuit onset is already aligned with the true object-motion
direction. Third, intermediate peak direction errors were found
for a variety of conditions, including high contrast, slow
speeds, filtered objects with edge motion, and/or increased
line-ending motion. Across all these conditions, mean peak
direction errors ranged between 20 and 30°, that is, in between
VA and IOC directions. In particular, when circular Gaussian
filters were centered onto the object corners (filter on corner
condition), the peak direction error was lowered down to
⬃20°, likely due to the introduction of new 2D features at the
end of the four edges, which move in the object-motion
direction (Shiffrar et al. 1995). When circular Gaussian filters
were centered onto the edges (filter on edge condition), thereby
removing much of the edge motion information while also
introducing new line endings, this reduced the peak direction
error even more, to ⬃10° (Fig. 7B) as now pursuit is driven
predominantly by the motion of 2D features (the corners and
the line endings). Interestingly, the latency of pursuit responses
remained remarkably constant across all these conditions.
At first glance, the fact that under some conditions the initial
pursuit direction can be aligned with the true object/pattern
motion direction might appear to be contradictory with our
earlier studies on the temporal dynamics of short-latency ocular following responses in humans (Masson 2004). We have
shown previously that reflexive tracking eye movements always start first in the direction of 1D (grating or edge) motion
such as with the barber-pole stimulus (Masson et al. 2000) or
in the VA direction of multiple 1D motion signals such as with
type I plaid patterns (Masson and Castet 2002). However, the
latency of these 1D-driven reflexive responses is only ⬃85 ms,
which is shorter than the latencies found in the present series of
experiment (⬃100 –120 ms). Furthermore, we found that 2D
motion cues such as line endings in barber-pole stimuli or
blobs in plaid patterns had an influence on tracking direction
⬃25 ms later, i.e., with a latency of ⬃110 –120 ms after visual
motion onset. Such temporal dynamics were found over a large
range of pattern speeds and contrasts (Masson and Castet 2002;
Masson et al. 2000). These results on reflexive, short-latency
J Neurophysiol • VOL
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
Motion-integration computation for pursuit initiation
tracking responses provide two insights into the present results
on voluntary pursuit. First, 1D edge motions are extracted first
and vector averaged to determine the potentially biased initial
estimate of global motion. Second, at high contrast, ⬃110 ms
after stimulus onset, that is at the time of smooth pursuit onset,
2D motion features have been extracted and are now being
integrated into the computation of the 2D object motion. Thus
at the time of voluntary pursuit onset, (i.e., ⬃100 –110 ms after
visual motion onset), the relative weighting between 1D and
2D motion can explain the continuum we observed for the peak
direction errors, lying in between the VA and true objectmotion directions.
Classical views of motion integration postulate that motion
signals orthogonal to an elongated edge are the most salient
information present in the image, particularly at low contrast
(Simoncelli et al. 1991; Wilson et al. 1992) and therefore
dominate early perceptual responses. Our results show that this
assumption applies to voluntary smooth eye movements. At
low contrast, the initial estimate of global motion relies mostly
on the simple VA solution and 2D feature motion has very
little, if any, impact. Increasing the target speed had similar
effects as sweeping an edge or a feature more rapidly over a
restricted receptive field would increase the level of measurement uncertainty at the level of the edge-motion detectors.
Moreover, at low contrast, the latency of visual processing of
2D features is delayed (Masson and Castet 2002), and therefore
the earliest phase of tracking is driven nearly exclusively by 1D
edge motion signals. Thus the initial pursuit direction nearly
perfectly matches the VA prediction. At high contrast, 2D
features are integrated more rapidly and therefore can start
influencing the tracking direction earlier (Wilson et al. 1992).
Weiss et al. (2002) proposed a probabilistic model of motion
integration where a most probable velocity field is computed
from the local velocity likelihood at every image location
(Simoncelli 2003; Simoncelli et al. 1991), which are combined
with an a priori distribution favoring slow and smooth motions.
At low contrast and high speed, i.e., high noise levels, the
product of the prior and the local velocity distribution yields an
a posteriori distribution whose maximum occurs at the VA
solution. At high contrast, however, the maximum of the
posterior probability is found at, or near, the IOC prediction.
Such a Bayesian model captures some of the main characteristics of 2D motion perception, such as a bias toward the VA
solution at low contrast and a gradual shift from VA to IOC
solutions as contrast (or, equivalently, duration) increases. For
the present study, it can predict the initial bias toward the VA
solution and more particularly the larger peak direction errors
found at low contrast and high speed.
However, the simple model proposed by Weiss and coworkers (2002) falls short in two critical ways. First, for linedrawing objects, pursuit always starts in a direction with a
minimum bias of ⬃30°, which is much closer to the VA
solution than any other solution. And this 30° deviation is
observed at high contrast target and slow speeds as well. This
finding clearly differs from the predictions of their model that
was tuned to accommodate perceptual results where the IOC
solution prevails at high contrast (e.g., Lorenceau et al. 1993;
Yo and Wilson 1992). Within their theoretical framework, our
results could, however, be accounted for by the temporal
integration window of the early spatiotemporal filtering being
much longer than the classical pursuit latency (⬃100 ms).
2D OBJECT MOTION FOR PURSUIT EYE MOVEMENTS
J Neurophysiol • VOL
ing on contrast, speed, and spatial filtering, such a model would
give different weights to 1D and 2D motion measurements as
a function of their reliabilities. The prior favoring slow speed
would still dominate motion integration at shorter latencies
when the uncertainty on the 1D motion information is high,
i.e., with low contrasts and high speeds. On the contrary, at
high contrast, 2D feature motion would start to dominate and
therefore lower the impact of the prior. Moreover, as the
latencies of 2D motion detection increases with lower contrast,
one would expect more sluggish dynamics in the gradual shift
between VA and true object-motion direction.
Neural solutions of dynamical motion integration
Area MT is a key structure in the complex neural network
underlying smooth pursuit eye movements (see Krauzlis 2004
for a review). Tracking eye movements get their main visual
input from a population of area MT neurons that represent both
target direction and speed (Lisberger and Movshon 1999;
Priebe et al. 2003, Priebe and Lisberger 2004). MT neurons
have many interesting properties relevant to the present results.
When presented with single oriented bars or gratings, the
direction selectivity of many neurons is normal to the bar/
grating orientation present in the receptive field (Albright
1984). When presented with two overlapping gratings, such as
in plaid patterns, these neurons respond to one component
motion direction. They have been called “component-selective” neurons (Movshon et al. 1985). There is a subgroup of
MT neurons that respond to a moving contour independently
from its orientation (Pack and Born 2001) or to a moving plaid
independently to the orientation/direction of its components
(Movshon et al. 1985; Rodman and Albright 1989). It has been
argued that these ”pattern-selective“ cells form the basis for 2D
motion perception (Movshon et al. 1985). There is recent
neurophysiological evidence pattern-selectivity gradually
emerges over ⬃100 ms. Pack and Born (2001) showed that
when presented with tilted moving bars, the earliest responses
of MT neurons are highly dependent on stimulus orientation,
i.e., their direction selectivity tuning curves point toward the
vector normal to the bar orientation. This dependence decreases gradually so that within ⬃150 ms after stimulus motion
onset, MT cells primarily encode the actual true 2D objectmotion direction of the moving bar, independently of their
orientations. To quantify the neuronal temporal dynamics, we
fit their “neuronal direction selectivity error” data, i.e., the
difference between preferred directions for oblique and upright
bars (Pack and Born 2001) (Fig. 2C) with our double-exponential function. We obtained a best-fitting estimate for the
decay time constant of ⬃40 ms. Such neuronal dynamics are
strikingly consistent with our behavioral observation on pursuit.
More recently, Pack et al. (2001) and Smith et al. (2001) showed
similar temporal dynamics for the emergence of true “patternselective” cells in response to moving plaids. Again, it took
⬎100 ms of neuronal response before an accurate estimate of
2D pattern motion emerged.
It is still unclear why macaque MT cells, as well as smooth
pursuit eye movements in both monkeys (Pack and Born 2001)
and humans (Masson and Stone 2002) exhibit such temporal
dynamics. Pack et al. (2004) suggested a specific role for 2D
features such as line endings, which can be extracted through
specific detectors such as V1 end-stopped cells (Pack et al.
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
Under this assumption, a high-contrast target can still generate
large measurement uncertainty and the prior would still dominate the a posteriori distribution. However, this seems unlikely
given that earlier psychophysical studies have shown that the
optimal motion stimulus duration for speed and direction
discrimination is ⬍100 ms (DeBruyn and Orban 1988; Masson
et al. 1999; McKee 1981; Watson and Turano 1995). This
estimate is consistent with a recent neural information theory
analysis by Osborne et al. (2004) showing that information
about motion direction builds up very rapidly, reaching ⱕ80%
of maximum information in the first 100 ms of neuronal
responses from a small population of area MT neurons. Second, increasing the number of 2D features lowered the initial
peak direction error, suggesting that one should consider the
relative weight of 1D and 2D motion cues to explain why—and
when— the initial tracking direction is deviated toward the VA
solutions. This scheme departs from the model of Weiss et al.
(2002) as it stresses the role of 2D features in a parallel
computation of object-motion direction with the output no
longer dictated by the relative probabilities of the VA and the
IOC signals, which are both based only on the 1D motion
signals extracted from the image. Furthermore, previous studies have shown that different motion percepts and steady-state
pursuit responses can be obtained from identical velocity space
information, demonstrating a clear need to consider other
factors (Lorenceau and Alais 2001; Stone et al. 2000). The final
two experiments showed that manipulating the relative amount
of 1D and 2D signals available has a large effect on the
directional error of pursuit. In fact, when a large proportion of
2D features is provided, as with the textured object in Fig. 9,
the initial tracking direction is in the direction of the true object
motion. (Note that this is not true for a uniform gray stimulus,
indicating it is indeed the texture elements driving the object
response, not simply the addition of luminance to the interior
of the stimulus). This result suggests that there is a competition
between 1D edge and 2D feature motion signals and that the
most reliable signals dominate the direction estimate. Interestingly, recent findings of Lindner and Ilg (2000) have demonstrated a similar effect, in which initial pursuit responses are
driven in the average direction of first- and second-order
information before the true object direction wins the direction
estimate (over a similar time course as found in the present
study). Finally, our results suggest that the 2D features become
reliable at high contrast and low speeds, whereas 1D information is more reliable at low contrast and high speeds, consistent
with the views espoused in several psychophysical studies
(Castet et al. 1993; Lorenceau et al. 1993; Mingolla et al. 1992;
Shiffrar and Lorenceau 1996).
To explain all these results, an alternative view is that 1D
and 2D motion signals have different latencies and therefore
influence the global estimate of object motion at different
points in time. Such a scheme of parallel processing between
1D and 2D motion cues can be easily encorporate a Bayesian
framework with two parallel channels having different dynamics in which both contribute independent likelihood fields to
the final distributed representation object motion (Perrinet et al.
2004). By having two different latencies (⬃85 and 110 ms,
respectively), the same model can render some dynamical
aspects of both short-latency ocular following (latencies ⬍110
ms) and smooth pursuit responses (for latencies ⬎110 ms) via
parallel, asynchronous inputs to the integration stage. Depend-
2291
2292
J. M. WALLACE, L. S. STONE, AND G. S. MASSON
Conclusions
We have shown that the initiation of smooth pursuit eye
movements exhibits interesting dynamic properties of 2D motion integration similar to that previously observed for perception and for neural responses in macaque area MT. All of these
results call for the development of models of 2D motion
integration wherein an initial estimate of object motion is first
extracted by combining the different motion signals available
at different points in time within a 2D integration stage and
then refines its output over time.
ACKNOWLEDGMENTS
We are grateful to D. Paleressompoule and A. De Moya for technical
assistance. We thank I. Vanzetta for carrying on the experiments and E. Castet
and L. Perrinet for insightful discussions and comments about the manuscript.
GRANTS
This work was supported by the Centre National de la Recherche Scientifique and by the European Commission through the Perception and Recognition for Action Research and Training Network (HPRN-CT-2002-00226). L.
Stone was supported by NASA’s Biomedical Research and Countermeasures
(111-10-10) and Aviation Operation Systems (727-05-30) programs.
REFERENCES
Adelson EH and Movshon JA. Phenomenal coherence of moving visual
patterns. Nature 300: 523–525, 1982.
Albright TD. Direction and orientation selectivity of neurons in visual area
MT. J Neurophysiol 52: 1106 –1130, 1984.
Beutter BR and Stone LS. Motion coherence affects human perception and
pursuit similarly. Vis Neurosci 17: 139 –153, 2000.
Bowns L. Evidence for a feature tracking explanation for why type II plaids
move in the vector sum directions at short directions. Vision Res 36:
3685–3694, 1996.
Busettini C, Miles FA, and Schwarz U. Ocular responses to translation and
their dependence on viewing distance. II. Motion of the scene. J Neurophysiol 66: 865– 878, 1991.
Castet E, Lorenceau J, Shiffrar M, and Bonnet C. Perceived speed of
moving lines depends on orientation, length, speed and luminance. Vision
Res 33: 1921–1936, 1993.
Collewijn H, van der Mark F, and Jansen TC. Precise recordings of human
eye movements. Vision Res 15: 447– 450, 1975.
De Bruyn B and Orban GA. Human velocity and direction discrimination
measured with random dot patterns. Vision Res 28: 1323–1335, 1988.
Fennema CL and Thompson WB. Velocity determination in scenes containing several moving images. Comp Graph Image Proc 9: 301–315, 1970.
J Neurophysiol • VOL
Ferrera VP and Wilson HR. Perceived direction of moving two-dimensional
patterns. Vision Res 32: 1015–1031, 1990.
Goldreich D, Krauzlis RJ, and Lisberger SG. Effect of changing feedback
delay on spontaneous oscillations in smooth pursuit eye movements of
monkeys. J Neurophysiol 67: 625– 638, 1992.
Goréa A and Lorenceau J. Directional performances with moving plaids:
component-related and plaid-related processing modes coexist. Spatial Vis
5: 231–252, 1991.
Koechlin E, Anton JL, and Burnod Y. Bayesian inference in populations of
cortical neurons: a model of motion integration and segmentation in area
MT. Biol Cybern 80: 25– 44, 1999.
Krauzlis RJ. Recasting the smooth pursuit eye movement system. J Neurophysiol 91: 591– 603, 2004.
Krauzlis RJ and Miles FA. Initiation of saccades during fixation and pursuit:
evidence in humans for a single mechanisms. J Neurophysiol 76: 4175–
4179, 1996.
Krukowski A and Stone LS. Expansion of direction space around the cardinal
axes revealed by smooth pursuit eye movements. Neuron In press
Linder A and Ilg UW. Initiation of smooth-pursuit eye movements to
first-order and second-order motion stimuli. Exp Brain Res 133: 450 – 456,
2000.
Lisberger SG and Movshon JA. Visual motion analysis for pursuit eye
movements in area MT of macaque monkeys. J Neurophysiol 19: 1907–
1916, 1999.
Lisberger SG and Westbrook LE. Properties of visual inputs that initiate
horizontal smooth pursuit eye movements in monkeys. J Neurosci 5:
1662–1673, 1985.
Löffler G and Orbach HS. Computing feature motion without feature
detectors: a model for terminator motion without end-stopped cells. Vision
Res 39: 859 – 871, 1999.
Lorenceau J and Alais D. Form constraints in motion binding. Nat Neurosci
4: 745–751, 2001.
Lorenceau J, Shiffrar M, Wells N, and Castet E. Different motion sensitive
units are involved in recovering the direction of moving lines. Vision Res 33:
1207–1217, 1993.
Masson GS. From 1D to 3D via 2D: dynamics of surface motion segmentation
for gaze stabilization in primates. J Physiol (Paris) 98: 35–52, 2004.
Masson GS and Castet E. Parallel motion processing for the initiation of
short-latency ocular following in humans. J Neurosci 22: 5149 –5163, 2002.
Masson GS, Mestre DR, and Stone LS. Speed tuning of motion segmentation
and discrimination. Vision Res 39: 4297– 4308, 1999.
Masson GS, Rybarczyk Y, Castet E, and Mestre DR. Temporal dynamics of
motion integration for the initiation of tracking eye movements at ultra-short
latencies. Vis Neurosci 17: 753–767, 2000.
Masson GS and Stone LS. From following edges to pursuing objects.
J Neurophysiol 88: 2869 –2873, 2002.
Miles FA. The neural processing of 3D visual information: evidence from eye
movements. Eur J Neurosci 10: 811– 822, 1998.
McKee SP. A local mechanism for differential velocity detection. Vision Res
21: 491–500, 1981.
Mingolla E, Todd J, and Norman J. The perception of globally coherent
motion. Vision Res 32: 1015–1031, 1992.
Movshon JA, Adelson EH, Gizzi MS, and Newsome WT. The analysis of
visual moving patterns. In: Pattern Recognition Mechanisms, edited by
Chagas C, Gattass R, and Gross C. New York: Springer, 1985, p. 117–151.
Movshon JA and Newsome WT. Visual response properties of striate cortical
neurons projecting to area MT in macaque monkeys. J Neurosci 16:
7733–7741, 1996.
Osborne LC, Bialek W, and Lisberger SG. Time course of information
about motion direction in visual area MT of macaque monkeys. J Neurosci
24: 3210 –3222, 2004.
Pack CC, Berezovskii VK, and Born RT. Dynamic properties of neurons in
cortical area MT in alert and anesthetized macaque monkeys. Nature 414:
905–908, 2001.
Pack CC and Born RT. Temporal dynamics of a neural solution to the
aperture problem in visual area MT of macaque brain. Nature 409: 1040 –
1042, 2001.
Pack CC, Gartland AJ, and Born RT. Integration of contour and terminator
signals in visual area MT. J Neurosci 24: 3268 –3280, 2004.
Pack CC, Livingstone MS, Duffy KR, and Born RT. End-stopping and the
aperture problem: two-dimensional motion signals in macaque V1. Neuron
39: 671– 680, 2003.
Perrinet L, Castet E and Masson GS. A model of motion processing underlying
early ocular following. Perception ECVP Abstracts 33: 70b, 2004.
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
2003), although these cell types are not known to feed directly
through to MT (Movshon and Newsome 1996). Nevertheless,
V1 end-stopped cells may play a crucial role in the effects we
found. End-stopping is known to weaken at low contrast
(Sceniak et al. 1999) and, at high contrast, is delayed ⬃20 –30
ms after the initial edge driven response (Pack et al. 2003),
corresponding well with that found in the ocular following
study previously discussed of Masson et al. (2000). The frontend end-stopped response could drive pursuit in the correct
direction when there is an abundance of texture information,
while in situations with more equally available 2D and 1D
information the delayed end-stopped response could underlie
the temporal dynamics of motion integration we have observed. However, several other schemes remain plausible such
as computing a delayed IOC-based 2D signal from different 1D
motion signals (Adelson and Movshon 1982; Movshon et al.
1985) or extracting separately first- and second-order motion
signals before vector averaging them (Löffler and Orbach
1999; Wilson et al. 1992).
2D OBJECT MOTION FOR PURSUIT EYE MOVEMENTS
J Neurophysiol • VOL
Smith MA, Majaj N, Bair W, and Movhson JA. Dynamics of responses to
grating and plaids in macaque area MT. Soc Neurosci Abstr 21: 680.2, 2001.
Stone LS, Beutter BR, and Lorenceau J. Visual motion integration for
perception and pursuit. Perception 29: 771–787, 2000.
Stone LS and Krauzlis RJ. Shared motion signals for human decisions and
oculomotor actions. J Vis 3: 725–736, 2003.
Wallach H. Uber Visuell Wahrgenommene Bewegungrichtung. Psychol
Forsch 20: 325–380, 1935 translated by Wuerger S and Shapley B. Perception 11: 1317–1367, 1996.
Watamaniuk SJN and Heinen SJ. Human smooth pursuit discrimination.
Vision Res 39: 59 –70, 1999.
Watson AB and Turano K. The optimal motion stimulus. Vision Res 35:
325–336, 1995.
Weiss Y and Adelson EH. Slow and smooth: a Bayesian theory for the
combination of local motion signals in human vision. Technical Report
AIM-1624, MIT, Feb. 1998.
Weiss Y, Simoncelli EP, and Adelson EH. Motion illusions as optimal
percepts. Nature Neurosci 5: 598 – 604, 2002.
Wilson HR, Ferrera VP, and Yo C. A psychophysically motivated model for
two-dimensional motion perception. Visual Neurosci 9: 79 –97, 1992.
Yo C and Wilson HR. Perceived direction of moving two-dimensional
patterns that depends on duration, contrast and eccentricity. Vision Res 32:
135–147, 1992.
93 • APRIL 2005 •
www.jn.org
Downloaded from http://jn.physiology.org/ by 10.220.32.247 on June 15, 2017
Priebe NJ, Cassanello CR, and Lisberger SG. The neural representation of
speed in macaque area MT/V5. J Neurosci 23: 5650 –5661, 2003.
Priebe NJ and Lisberger SG. Estimating target speed from the population
response in visual area MT. J Neurosci 24: 1907–1916, 2004.
Rashbass C. The relationship between saccadic and smooth tracking eye
movements. J Physiol 159: 326 –338, 1961.
Rao RPN. Bayesian computation in recurrent neural circuits. Neural Comp 16:
1–38, 2004.
Rodman and Albright TD. Single-unit analysis of pattern-motion selective properties
in the middle temporal visual area (MT). Exp Brain Res 75: 53–64, 1989.
Sceniak MP, Ringach DL, Hawken MJ, and Shapley R. Contrast’s effect on
spatial summation by macaque V1 neurons. Nat Neurosci 2: 733–739, 1999.
Simoncelli EP. Local analysis of visual motion. In: Visual Neuroscience,
edited by Chalupa LM and Werner JS. Cambridge, MA: MIT Press, 2003,
vol. II, ch. 109, p. 1616 –1623.
Simoncelli EP, Adelson EH, and Heeger D. Probability distributions of
optical flow. Proc. IEEE Conf Comput Vis Pattern Recogni 310 –315, 1991.
Shiffrar M, Li X, and Lorenceau J. Motion integration across differing
image features. Vision Res 36: 2137–2146, 1995.
Shiffrar M and Lorenceau J. Increased motion linking across edges with
decreased luminance contrast, edge width and duration. Vision Res 36:
2061–2067, 1996.
2293