Trajectory-Based Ball Detection
and Tracking with Aid of Homography
in Broadcast Tennis Video
Xinguo Yu, Nianjuan Jiang,
Ee Luang Ang
Visual Communications and Image Processing 2007
Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6508
Present by komod
Introduction
• The ball is the most important object
in tennis (and in many kind of sports)
• Very challenging problem
– Camera motion
– presence of many ball-like objects
– small size and the high speed of the ball
– Object-indistinguishable
Introduction
• Method
– Trajectory-based
• the ball is the “most active” object in tennis video
• previous work： A Trajectory-based ball detection
and tracking algorithm in broadcast tennis video,
Proc. of ICIP
– Homography
• Goal
– find projection locations of the ball on the
ground
– find landing positions
Introduction
Introduction
Feature Point Extraction
• Court Segmentation
– Find the court color range and paint all the
pixels in this range with a single color
– find the lines separating the audience from
the playing field
• detecting the change pattern of color for each row
and column of the image
– paint the audience area in the court color.
Feature Point Extraction
• Straight Line Detection
– gridding Hough transform
• Court Fitting
– Detect the net and use it as reference
– find the intersection of lines
Homography Acquisition
• Standard Frame
– whose lookat is the cluster center of all
lookats of all the frames in the
considered clip
– The lookat of frame is a point in the
real world that corresponds to the
center of the frame
Homography Acquisition
• Disparity Measure of Two Court Images
– For i = 1 to 9
• Measure Function
– Let Cstd be the court in the standard frame and Ctrn
denote the transformed court from the segmented
court in frame F
– For given H and F
Homography Acquisition
• Initial Matrix
– transforms an image point X' (x1', y2', 1) to a
point X (x1, y2, 1) in another image
– X = HX‘
• Tuning of Homography
– The homograph matrix computed based on
feature points
– A small hough space enclosing it
Homography Acquisition
• Tuning procedure
• Frame transform
Ball Location In Hitting Frame
• Hitting frame detection
– Find the sound emitted by the racket
hitting
• M. Xu et al, Creating audio keywords for
event detection in soccer video, In Proc. of
ICME
• Hitting racket detection
– Maybe player tracking
Ball Candidate Detection
• Object segmentation from standard frame
• Four sieve are used for non-ball object removal
– Court Sieve Θ1
• filter out audience area
• filter out court lines
– Ball Size Sieve Θ2
• filter out the objects out of the ball-size range
• homography from ground model to standard frame
• use a range of allowable ball sizes (estimate error)
– Ball Color Sieve Θ3
• filter out the objects with too few ball color pixels
– Shape Sieve Θ4
• filter out objects out of the range of width-to-height ratio
• 2.5 is suggested in previous paper
Ball Candidate Detection
– Each sieve is a Boolean function on domain Ο(F)
– The set of remaining objects is C(F)
• C(F) = {o : o ∈O(F), Θi(o)=1 for i = 1 to 4}
• Candidate Classification
– Three features are use
• Size, color, and distance from other objects
– The ball-candidates are classified into 3 Categories
Candidate Trajectory Generation
• No detail explanation in this paper
– X. Yu et al, Trajectory-based ball detection and
tracking of broadcast soccer video, IEEE Transactions
on Multimedia, issue 6, 2006.
• Candidate Feature Plots (CFPs)
– CFP-y
– CFP-l
• The algorithm is actually works on the CFP-l which are 3-D plots
Candidate Trajectory Generation
Trajectory Processing
• Trajectory Confidence Index
– Let T be a candidate trajectory
– and λ1,λ2,…,λm, be all properties of
trajectory T
– confidence index Ω(T)
Trajectory Processing
• Trajectory Discrimination
Trajectory Processing
• Ball Projection Location
– y = an3 + bn2 + cn + d.
• Ball Land Detection
– form a ball position function against
frame number i, y = f(i)
– find the maximum of f '(i) between
each pair of hittings
Experimental Results
• 5 clips
• extracted from mpeg2 704x576
• average time for acquiring ball
candidates
• ALGnew for a frame is 86.15s on a
P4/1.7Ghz PC with 512MB RAM
• ALGold is 19.21s
Experimental Results
BPL
Experimental Results
Experimental Results
• average discrepancy of all detected
balls from the groundtruth
previous result
Experimental Results
• frames with inserted 3D projected virtual content
• Homography in home surveillance video
Conclusion and Future Works
• The previous algorithm mainly alleviated the challenges
raised by causes besides camera motion
• The algorithm presented in this paper additionally
counteracts the challenges brought to us by the camera
motion
• The contributions of this paper are two-fold
– it develops a procedure to robustly acquire an accurate
homograph matrix of each frame
– it forms an improved version of ball detection and tracking
algorithm
• Two future works
– evolve the algorithm into an end-to-end system for ball
detection and tracking of broadcast tennis video
– analyze the tactics of players and winning-patterns, and
hence produce rich indexing of broadcast tennis video by
making use of the ball position
Any Question?
Thank You
Experimental Results
• 7 segments, total 120 s, mpeg1 video,
Men’s Final of FRENCH OPEN 2003
back