Proceedings of the 2005 IEEE
International Conference on Mechatronics
July 10-12, 2005, Taipei, Taiwan
Automatic Speed Measurements of Spherical
Objects Using an Off-the-Shelf Digital Camera
Huei-Yung Lin and Chia-Hong Chang
Department of Electrical Engineering
National Chung Cheng University
160 San-Hsing, Min-Hsiung, Chia-Yi 621, Taiwan
[email protected], [email protected]
algorithms use reference information in the scene, such as
the distance traveled across image frames, to compute the
speed of the moving object.
In this work, we propose an approach for speed measurements of spherical objects based on a single image
taken by a hand-held digital camera. The basic idea of our
approach is as follows. Due to the relative motion between
the camera and the moving object during the finite exposure
time of the camera, motion blur will appear on the dynamic
regions of the image. For any fixed shutter speed, the
moving distance of the object is proportional to the amount
of motion blur caused by the imaging process. Thus, if the
direction and extent of the motion blur can be identified,
it is possible to recover the physical displacement of the
moving object using the intrinsic and extrinsic parameters
of the camera.
Since the projection of a sphere-like object on the image
plane is circular, more accurate motion extent of the object
can be obtained by applying a circle fitting algorithm on a
sequence of deblurred images created with the initial blur
extent estimation. Finally, the speed of the object can be
calculated by means of displacement divided by the camera
exposure time. To make the system fully automatic, a target
object detection algorithm using motion blur information
is proposed. The identified target region is extracted from
the original image and used for speed measurement. The
proposed method also has the advantages of reducing the
computation and increasing the robustness of the proposed
speed measurement approach.
Generally speaking, image degradation caused by motion blur can be classified as either spatially invariant or
spatially variant distortions based on the formation of the
imaging process. Spatially invariant distortion corresponds
to the cases that the image degradation model does not
depend on the position in the image. This type of motion
blurred images is usually a result of camera movement during the imaging process. Restoration of spatially invariant
motion blurred images is a classic problem and several
approaches have been proposed in the past few decades [3],
[4]. The goal is to find the point spread function (PSF) of
the blurring system and then use deconvolution techniques
to restore the ideal image.
The motion blur we have to deal with appears in the
image containing a fast moving object and recorded by
Abstract— A method for automatic speed measurements of
spherical objects using a digital camera is presented. Conventional speed measurement techniques use radar or laser
based devices, which are usually more expensive compared
to a passive camera system. In this work, a single image
captured with object motion is used to estimate the speed of a
spherical object (such as baseball). Due to the relative motion
between the camera and a moving object during the camera
exposure time, motion blur occurs in the dynamic region of
the image. By identifying the motion blur parameters, the
speed of a moving object can be obtained. Automatic target
identification and motion estimation are first done by motion
blur analysis, followed by more accurate blur identification
using circle fitting of the spherical object. Finally, the object
speed is calculated according to the imaging geometry, camera
pose, and blur extent in the image.
Index Terms— speed measurements, motion blur, image
analysis, computer vision
I. I NTRODUCTION
In recent years, because of advances in the development
of various aircraft, baseball exercise, and electronic devices, people are becoming more interested in knowing
the speed of flying objects, the pitching speed of sports
players, and the speeding of motor vehicles detected by law
enforcement agencies. Currently, most speed measurements
of moving objects are achieved by microwave radar or
laser devices. Radar-based devices bounce a radio signal
off to a moving object, and the reflected signal is picked
up by a receiver. The radar receiver then measures the
frequency difference between the original and reflected
signals, and converts it into the speed of the moving
object. A laser-based device times how long it takes a
light pulse to travel from the laser gun to the moving
object and back. Based on this information, it can quickly
find the distance between the gun and the object. By
making several measurements and comparing the distance
the object traveled between measurements, a laser-based
device can accurately determine the speed of the object.
Due to the availability of low-cost imaging hardware
in the past few years, video-based methods have been
proposed for speed estimation of moving vehicles [1], [2].
These methods use reference information in the scene, such
as the distance traveled across image frames, the speed of
the vehicle is then estimated by the distance divided by the
inter-frame time. Most of the video-based speed estimation
0-7803-8998-0/05/$20.00 ©2005 IEEE
66
detector

−1 0
 −1 0
−1 0
is extended to an n × n mask
of the following form

−1 · · · − n−1
2
 −1 · · · − n−1
2

 ..
..
 .
.

 −1 · · · − n−1
2
−1 · · · − n−1
2
Fig. 1.

1
1 
1
(where n is an odd number)
0
0
..
.
n−1
2
n−1
2
0
0
n−1
2
n−1
2
..
.

··· 1
··· 1 

.. 
. 

··· 1 
··· 1
to emphasize the response on the ramp edge region. Typically, a 9 × 9 mask is used and strong response of the ramp
edge spreads a few pixels, as shown in Fig. 2.
Different from sharp edges or regions without texture,
motion blur caused by an object moving in front of a static
background gives a symmetric response on the width of the
ramp edges. To identify the target region, a binary image
is first generated from the gradient image with nonzero
values set as 1, and the run-length code is created for
each image scanline [12]. Some of the run-length codes
are shown below for the marked regions in Fig. 2, where
the first half corresponds to the red region and the second
half corresponds to the yellow region.
7, 6, 5, 44, 41, 6, 1
5, 8, 5, 44, 41, 5, 2
8, 5, 5, 44, 41, 4, 3
0, 1, 8, 4, 4, 45, 41, 4, 3
0, 2, 6, 5, 3, 2, 3, 41, 41, 4, 3
0, 15, 9, 3, 4, 7, 8, 3, 6, 5, 6, 16, 15, 1, 2, 4, 6
0, 16, 15, 7, 8, 2, 7, 5, 5, 17, 19, 2, 7
0, 16, 15, 7, 15, 1, 2, 5, 4, 2, 1, 14, 19, 2, 6, 1
0, 17, 15, 6, 18, 5, 5, 1, 1, 14, 19, 2, 7
0, 6, 1, 10, 14, 8, 14, 14, 2, 12, 20, 2, 7
A unique pair of 1 and 0 runs which are both large
and comparable in size is then identified for each runlength code. The motion blur region is then obtained by
Motion blurred image taken with a fast moving object.
a static camera. The corresponding PSF is a function
of position in the image, and thus it is classified as
spatially variant distortion. Image restoration of this type
of distortion is considered as a more difficult problem
than the spatially invariant case and has been addressed
by only a few researchers [5]–[7]. In this work, we focus
on blur parameter estimation and its application on speed
measurement, rather than developing sophisticated image
restoration algorithms.
II. TARGET O BJECT D ETECTION AND I DENTIFICATION
The first step of a vision-based method for speed measurement of a moving object is to identify the target
automatically. One simple solution for object detection
is using image subtraction with an additional background
image. However, for most situations, it is not possible to
obtain exactly the same scene for two image captures with a
hand-held device, even without the presence of noise. Thus,
registration of the images has to be performed prior to
image subtraction, which complicates the object detection
algorithm [8].
In this work, the target object for speed measurement is
identified using only a single image with motion blur information. As shown in Fig. 1, the moving object generates
unidirectional ramp edges along the motion direction in
its neighborhood, while the static background regions are
either focused with sharp edges or slightly blurred with
isotropic defocus (caused by optical defocus). Thus, the
target object can be identified by detecting the specific
ramp edge regions of the image. We first assume the object
is moving parallel to the horizontal image scanlines, and
then deal with motion blur along any direction.
Commonly used edge detection masks such as Sobel
or Prewitt operators are designed for strong responses on
sharp edges [9]. Although some of the previous work has
been done for ramp edge detection [10], [11], these studies
basically dealt with both the blurred and sharp edges.
To detect the horizontal ramp edges exclusively (while
suppressing the sharp edges), the horizontal Prewitt edge
Fig. 2.
67
Gradient image from horizontal ramp edge detection.
considering the vertical continuity of the above pairs of 1
and 0 runs for all image scanlines and their corresponding
locations in the horizontal direction. Since only the target
is used for speed measurement, a subimage centered at the
detected region is extracted with typically 3 - 5 times of its
height and width. Under the assumption that the object is
moving in front of a uniform background (such as the sky
or ground), the experiments have shown that the proposed
method for target identification using motion blur is very
robust.
If the approximate motion direction of the object is
unknown, three additional images are created by rotating
the original image by 45◦ , 90◦ , 135◦ , respectively. The
above algorithm is then applied on these four images to
identify the target region.
Fig. 3. The original image (left), after image rotation (middle), and after
Sobel detection (right).
It is well known that the response of a sharp edge to an
edge detector is a thin curve, whereas the response of a
blur edge to the same edge detector is spread over a wider
region. This suggests that the result of edge detection can
be used as an initial estimate of the motion blur extent.
To obtain the blur extent along the horizontal direction, a
Sobel edge detector is first applied on the subimage to find
the left and right blur regions. Ideally, there will be two
edges with the same width which correspond to the left
and right partially motion blurred regions for each image
scanline. Thus, the blur extent of a moving object can be
obtained by finding the mode of the edge widths in the
edge image. However, as shown in Fig. 3, the blur extents
along the image scanlines are not all identical due to the
circular shape of the spherical object. A good estimate can
be given by taking the average of the edge widths which
are larger than 75% of the largest edge width.
Since the intensity variations inside the object region
affect the blur extent estimation, they should be suppressed
while searching for the best estimate. The steps of the
algorithm for robust estimation of motion length is thus
given as follows:
1) Calculate the summation of the edge widths for each
row, and find the mode of the summations for the
whole image.
2) Set Edgewidth as half of the edge width corresponding to the mode derived in Step 1). Set the number
of iterations.
3) Compare the summation of the edge widths for each
row with 2 · Edgewidth . Record the summations
which are larger than 75% of 2 · Edgewidth .
4) Find the mode of the above summations for the
whole image and replace Edgewidth with half of
the corresponding summation.
5) Go back to Step 2) and repeat until Edgewidth
converges or pre-defined number of iterations is
reached.
III. M OTION E STIMATION
To estimate the affine motion of a spherical object from
a motion blurred image, the direction and extent of motion
blur have to be identified first. A sequence of deblurred
images is then obtained according to those parameters,
followed by a circle fitting process to select the image with
the best motion deblurred result and the corresponding blur
extent.
A. Blur Parameter Identification
It is clear that the motion direction of a moving object
(appeared in the image) is the same as the direction of
motion blur. Thus, the intensity of high frequency components along this direction is decreased. As a result, the
derivative of the image in the motion direction should
suppress more of the image intensity compared to other
directions. To identify the motion direction relative to
the horizontal image scanlines, either spatial or frequency
domain approaches can be used [4]. However, most of
those methods consider the case that the whole image is
blurred, which is different from the motion blur caused by
an object moving in front of a static background. To solve
this problem, a gradient based method is applied only on
the subimage (containing the target object), as derived in
the previous section to estimate the motion direction [13].
For simplicity of blur extent identification, the angle between the identified motion direction and image horizontal
scanlines is used to rotate the image, so that the direction
of object motion is reduced to a one-dimensional case. In
this case, a commonly used spatially invariant model for
image blur is given by
g(x) = h(x) ∗ f (x)
B. Image Deblurring and Circle Fitting
Due to noise, non-uniform intensity of the background,
and other artifacts in real images, the blur extent estimates
obtained in the previous section usually contain several
pixels of error. To make the estimation more robust, a
circle fitting algorithm is employed to identify the object
and estimate the blur extent more accurately. First, a
sequence of motion deblurred images is created by Wiener
filter or blind deconvolution [15] using the estimated blur
parameters. The number of deblurred images to be created
depends on the size of the object appeared in the image and
(1)
where h(x) is a linear PSF, and f (x) and g(x) are the original and observed images, respectively [14]. Furthermore,
the one-dimensional PSF for the uniform linear motion can
be modeled as
1/R, |x| ≤ R/2
h(x) =
(2)
0, otherwise
where R is the extent of motion blur. That is, a step edge
becomes a ramp edge with a width of R due to the uniform
linear motion of the scene.
68
Fig. 4.
Motion deblurred images (top), edge images (middle), and circle fitting results (bottom).
the initial blur extent estimation. In the implementation,
typically 9 deblurred images (each image with one pixel
difference of motion deblurring) are generated for circle
fitting. For example, if the estimated blur length is K
pixels, then the images are deblurred with K − 4 to K + 4
pixels.
After a sequence of deblurred images is created, a circle
detection algorithm [16] is used to select the image with
the most circle-like object. The selected image is referred
to as the best deblurred image, and the corresponding
motion blur extent is considered as an accurate motion
length of the object appeared in the image. Fig. 4 shows
five motion blurred images and the circle fitting results.
In the experiments, the difference between the initial blur
extent estimation and the result after an additional circle
fitting procedure is less than 3 pixels in most cases. This
further ensures that 9 deblurred images are sufficient for
the identification of blur extent used for circle fitting.
is half of the angles between the rays passing through the
edges of the object. Moreover, x and y are given by
z x̂
f
x=
and
y=
z ŷ
f
(4)
from perspective projection, where f is the focal length of
the camera. If x̂l and x̂r represent the left and right edge
points of the object in the image, respectively, then θ and
α are given by
x̂l
tan θ =
(5)
f
and
tan(θ + 2α) =
x̂r
f
(6)
Thus, the 3-D coordinates of an object can be identified
from the captured image if its actual size is available.
z
IV. O BJECT S PEED M EASUREMENT
Q
d
2
The proposed method for speed measurements is based
on a pinhole camera model. 3-D coordinates and motion
direction of the target object in the camera coordinate
system are identified and then used for speed calculation.
P
φ
A. Camera Pose Estimation
z
As shown in Fig. 5, suppose the center of a spherical
object with diameter d is located at P = (x, y, z) in the
camera coordinate system and the corresponding image
point is p = (x̂, ŷ). Then the depth of the object can be
derived as
d cos(α + θ)
(3)
z=
2 sin α
θ
α
α
Image Plane
p q
f
x
O
where θ is the angle between the optical axis and the ray
passing through the left edge point of the object, and α
Fig. 5.
69
Camera pose estimation.
given by (7) have to be used to find the 3-D point P′ =
(x′ , y ′ , z ′ ) since the depth z ′ cannot be recovered by Eq.
(3). Suppose the angle φ is available, i.e.,
P′
d
P
tan φ =
z
p
u
z′ =
p′
β
f (z − x tan φ)
,
f − x̂′ tan φ
x′ =
z ′ x̂′
f
and y ′ =
z ′ ŷ ′
f
(10)
where f is the focal length of the camera.
If we consider a special case that the object is moving
in a direction parallel to the image plane, i.e., φ = 0 in
Figs. 5 and 6, then the displacement can be obtained from
similar triangles. Thus, Eq. (8) can be simplified as
x
f
O
v=
Fig. 6.
(9)
then the point P is given by
y
φ
z′ − z
x′ − x
Camera model for speed measurement.
zKsx
Tf
(11)
where K is the blur extent in pixel and sx is the CCD
pixel size of the camera. In this case, the position of the
object in the image is not required for speed measurement.
Similarly, the distance z can be derived as
To estimate the motion direction φ with respect to the
image plane, two image captures are used to identify the
locations of the object moving on the path. Suppose the 3-D
coordinates of the objects are (x1 , y1 , z1 ) and (x2 , y2 , z2 )
from the two images, then the angle is given by
z2 − z1
−1
φ = tan
(7)
x2 − x1
z=
fd
W sx
(12)
where d and W are the diameters of the object measured
physically and observed in the image, respectively. From
Eqs. (11) and (12), the speed v can be written as
It should be noted that the estimation of the angle φ can
be thought as a calibration stage and should be done prior
to speed calculation. At this stage, the shutter speed of the
camera should be set as fast as possible to avoid motion
blur.
Kd
(13)
WT
without the knowledge of the focal length and CCD pixel
size of the camera.
v=
V. E XPERIMENTAL R ESULTS
B. Formulation of Speed Measurement
The proposed speed measurement algorithms for spherical objects have been tested on real images from outdoor scenes with softball pitching. A system flowchart
is depicted in Fig. 7. For the first set of experiments,
the camera was placed approximately perpendicular to the
motion direction of the softball. Eq. (13) is used for speed
measurements with camera shutter speed T = 1/320 sec.
and softball diameter d = 97.45 mm. For the images shown
in Figs. 3 and 4, the blur extent and the size of the softball
are identified as 26 and 72 pixels, respectively. Thus, the
speed is found to be 40.54 km/hr. For the second set of
experiments, the camera was installed such that the image
plane is approximately 30 degrees off the moving direction
of the softball. The same camera parameters are adopted,
except that the shutter speed is first set as 1/800 seconds for
motion direction estimation. To make the motion direction
estimation more robust, instead of using Eq. (7), the angle
φ is given by line fitting on the xz-plane projections of
the 3-D coordinates obtained from five different images.
The angle φ is found as 32.5 degrees in the experiment.
For each speed measurement, Eqs. (3) - (10) are used to
compute the speed of the softball with the CCD pixel size
of 4.4 µm. Some results (16 out of 25 pitches) for both
In this work, we consider both a special case that the
object is moving parallel to the image plane and the general
case that the object is approaching or leaving the camera
at a certain angle. Speed measurement is formulated as the
object displacement in 3-D space divided by the camera
exposure time. As illustrated in Fig. 6, suppose the initial
and end positions of the object are P and P′ for a single
image capture, then the speed v is given by
d
||P′ − P||
=
(8)
T
T
where T is the shutter speed. Thus, if the motion blur is
identified and denoted as a displacement vector u, and the
center of the object in the deblurred image is denoted as p,
then the corresponding image points of P and P′ are given
by p and p′ = p + u, respectively. Suppose the identified
motion direction is an angle β with respect to the image
scanlines and blur extent is K pixels along the motion
direction, then we have u = (Ksx cos β, Ksx sin β), where
sx is the CCD pixel size of the camera.
For a given image point p = (x̂, ŷ), the 3-D coordinates
of P = (x, y, z) can be obtained from Eqs. (3) and (4).
However, Eq. (4) and the constraint on the motion direction
v=
70
TABLE I
S OFTBALL SPEED MEASUREMENT ( IN KM / HR ) FROM MOTION BLURRED IMAGES (B1, B2) AND A VIDEO - BASED APPROACH (V1, V2).
No.
B1
V1
B2
V2
1
40.5
40.9
38.9
37.1
2
34.0
37.2
34.7
39.1
3
36.9
38.9
34.8
36.5
4
33.1
34.2
52.8
48.1
5
40.1
40.4
42.2
35.4
6
36.4
38.2
50.3
44.3
7
41.7
41.0
49.2
51.5
8
45.5
45.3
46.6
46.6
13
42.1
39.5
54.8
50.2
14
41.0
43.0
40.3
37.0
15
39.3
43.5
51.6
46.3
16
42.1
46.2
42.8
41.0
R EFERENCES
[1] S. Pumrin and D.J. Dailey, “Roadside camera motion detection
for automated speed measurement,” in IEEE 5th International
Conference on Intelligent Transportation Systems, 2002, pp. 147–
151.
[2] A.L. Harvey and H.A. Cohen, “Vehicle speed measurement using
an imaging method,” in International Conference on Industrial
Electronics, Control and Instrumentation, 1991, pp. 1730–1733.
[3] M.M. Sondhi, “Image restoration: The removal of spatially invariant
degradations,” Proceedings of IEEE, vol. 60, no. 7, pp. 842–853,
July 1972.
[4] Y. Yitzhaky, I. Mor, A. Lantzman, and N.S. Kopeika, “Direct method
for restoration of motion blurred images,” Journal of the Optical
Society of America, vol. 15, no. 6, pp. 1512–1519, June 1998.
[5] H.J. Trussell and S. Fogel, “Identification and restoration of spatially
variant motion blurs in sequential images,” IEEE Trans. Signal
Processing, vol. 1, no. 1, pp. 123–126, January 1992.
[6] M.K. Ozkan, A.M. Tekalp, and M.I. Sezan, “Pocs-based restoration
of space-varying blurred images,” IEEE Trans. Signal Processing,
vol. 3, no. 4, pp. 450–454, July 1994.
[7] S.K. Kang, Y.C. Choung, and J.K. Paik, “Segmentation-based image
restoration for multiple moving objects with different motions,” in
International Conference on Image Processing, 1999, pp. 376–380.
[8] Lisa Gottesfeld Brown, “A survey of image registration techniques,”
ACM Comput. Surv., vol. 24, no. 4, pp. 325–376, 1992.
[9] R.C. Gonzalez and R.E. Woods, Digital Image Processing, 2nd
Edition, Prentice Hall, 2001.
[10] Z.Q. Wang, K.R. Rao, and J. Ben-Arie, “Optimal ramp edgedetection using expansion matching,” IEEE Trans. Pattern Analysis
and Machine Intelligence, vol. 18, no. 11, pp. 1092–1097, November
1996.
[11] M. Petrou and J.V. Kittler, “Optimal edge detectors for ramp edges,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no.
5, pp. 483–491, May 1991.
[12] R.C. Jain, R. Kasturi, and B.G. Schunck, Machine Vision, McGrawHill, 1995.
[13] H.Y. Lin and K.J. Li, “Motion blur removal and its application
to vehicle speed detection,” in International Conference on Image
Processing, 2004, pp. 3407–3410.
[14] M.R. Banham and A.K. Katsaggelos, “Digital image restoration,”
IEEE Signal Processing Magazine, vol. 14, no. 2, pp. 24–41, March
1997.
[15] M. Cannon, “Blind deconvolution of spatially invariant image blurs
with phase,” IEEE Trans. Acoustics, Speech, Signal Processing, vol.
ASSP-24, no. 1, pp. 58–63, 1976.
[16] T.C. Chen and K.L. Chung, “An efficient randomized algorithm for
detecting circles,” Computer Vision and Image Understanding, vol.
83, no. 2, pp. 172–191, August 2001.
Object Speed
Two Images
Target Identification
Environment Parameter
Estimation
Image Segmentation
Speed Measurement
no
Image Rotation
Circle Fitting
yes
Fig. 7.
12
50.2
49.1
53.4
34.0
The support of this work in part by the National Science
Council of Taiwan under Grant NSC-93-2218-E-194-024 is
gratefully acknowledged.
Most commonly used methods for object speed measurement adopt radar or laser based devices. They are
active devices and usually more expensive compared to a
passive camera system. In this paper, we have presented an
automatic speed measurement method of spherical objects
using a single image. Blur parameters of the motion blurred
image are estimated and then used to calculate the speed of
the object according to a pinhole camera model. Since our
approach is an image-based method, it can be implemented
with low cost, off-the-shelf digital cameras. Furthermore,
the camera settings (such as shutter speed, image resolution, focal length) can be easily adjusted to improve the
Initial Blur Length
Estimation
11
39.2
43.8
48.6
43.1
ACKNOWLEDGEMENTS
VI. C ONCLUSION
Horizontal
Motion Blur
10
40.5
41.1
49.3
41.2
accuracy of the system. In contrast to the commercially
available speed measurement devices, our approach has
virtually no limitation on the speed test range. In the
future work, more complicated camera model and different
methods of blur parameter estimation will be considered to
increase the robustness of the speed measurement results.
cases are shown in Table I. B1 and B2 represent the first
and second set of the experiments, respectively.
To find the relative error of our method compared to a
video-based approach, a video camera was used to record
an image sequence of each experiment. The speed is calculated by the distance traveled between two fixed locations
divided by the time difference. The results are shown in
Table I for both cases (V1 and V2). Most of the results from
our method and the video-based approach have less than
2% difference for the first set of experiments (where the
ball was moving parallel to the image plane), and less than
10% difference for the second set of experiments (where
the ball was moving about 30 degrees off the image plane).
Some results with large error for the second case could be
caused by incorrect angle φ for the motion direction, since
this angle might be different for each pitch.
Motion Blurred
Image
9
45.5
44.2
42.5
47.0
Image Deblurring
System flowchart for speed measurement.
71