First Int'l Workshop on Image and Signal Processing and Analysis IWISPA
2000, June 14.-15. 2000, Pula, Croatia.
Computer Vision
System for Tracking
Players in Sports
Games
Janez Perš,Stanislav Kovačič, Faculty of Electrical
Engineering, University of Ljubljana, Tržaška 25, 1000
Ljubljana
[email protected], [email protected]
Outline
Introduction and motivation
Image acquisition
Camera calibration
Player tracking
Results
Conclusions
Introduction, Motivation
Human motion analysis in context of team sport - handball
Objective: to obtain trajectories:
...for every player, ...
...in the whole field,...
...in every instance of time.
Facts:
- Up to 14 players may be present on the court at
the same time.
- Large playing field (20x40 meters).
- Duration of the match is 60 minutes.
Problems:
- Large area to cover
- Rapid player motion
- Large amount of data to process:
25 fps * 3600 seconds = 90.000 images
Image acquisition
40 m
20 m
Camera setup:
- Two cameras, fitted with
103° lens were placed 10m
above the playing field.
camera
- Wide angle lens cause
significant distortion of
the acquired images.
Transfer to digital domain:
- Motion-JPEG compression
hardware.
- 25 fps and 384x288 pixel
resolution.
Result:12 GB of compressed
video data for a single
match.
camera
Camera calibration
Linear model was used to obtain scaling factors, camera
orientation and position.
Radial lens distortion is the most difficult problem.
Instead of polynomial approximation, we built a model of
radial distortion.
Results:
- court plane  image plane
R
R12 
1

r1  h  ln
 1 2 .
h
h 

- image plane  court plane
R1 
h (e
2

2 r1
h
e

) 1
r1
h
.
Player tracking - overview
An offline process.
Supervised by a human operator, who:
- Defines starting player positions before
tracking is started.
- Corrects tracking errors.
Post-processing of obtained trajectories is necessary to
reduce jitter. Gaussian filter can be used.
Three tracking methods were developed:
- Motion detection
- Template tracking
- RGB (color) tracking
Player tracking - 1
Motion detection by subtracting images:
1.
2.
3.
4.
5.
6.
Image of the empty playing court.
Current frame.
Difference image.
Thresholding.
Filtering by median filter.
Blob assignment.
Player tracking - 1
Drawback of motion detection algorithm is high
sensitivity to:
- light reflections,
- shadows
- non-uniform illumination.
There actually four players in the above scene!
Player tracking - 2
To exploit visible differences between the players and
background “objects”, we propose template tracking.
It is extremely difficult to build an accurate model of a
player, especially at low resolution.
We defined a set of templates , {Kj,j=1..14}, which
represent basic appearances of the player:
Feature vectors G and F are
derived from background image and
current frame at estimated player
16 16
position (ROI of 16x16 pixels). Fj   K j ( x, y)  I CUR ( x, y), ( j  1,..,14)
x 1 y 1
16 16
Vector H is calculated as mean
G j   K j ( x, y)  I BAC ( x, y), ( j  1,..,14)
average of past vectors F
x 1 y 1
obtained at true player positions.
Player tracking - 2
F (unknown object)
DFH
Simplified, two dimensional case:
DGF
H (player)
DGF and DHF are Euclidean distances
G (background)
Similarity measure S {S = [0..1]}
reflects the similarity between the
object in ROI and the predicted
appearance in H.
S
DFH
.
DGF  DFH
0.8
10
The position of the
player in image is
defined by the
position of local
minimum of measure S.
20
0.7
30
0.6
40
50
0.5
60
0.4
70
80
0.3
90
0.2
100
20
40
60
80
Drawback: possibility of adaptation to the background
objects.
100
Player tracking - 3
Color can be used to identify players without any need
for adaptation.
Players may wear black or dark-colored dresses, so we
chose the RGB color representation.
Problem: There are only a few pixels that correspond to
the true color of player’s dress.
10
2
20
4
30
6
40
8
50
60
10
70
12
80
14
90
16
100
10
20
30
40
50
60
70
80
90
100
2
4
6
8
10
12
14
16
Our ”Color tracking” algorithm simply searches for the
pixel that is the most similar to operator-defined RGB
value. Consequence: high amount of jitter.
Results
We tested three different methods:
A: Motion detection
B: RGB (color) tracking
C: Combination of color and template tracking:
- results of color tracking are used as a
reliable, but inaccurate estimate.
- template tracking is used to increase
accuracy and eliminate jitter.
- initial estimate for the next frame is
based on the output of the color tracking
algorithm to prevent adaptation to the
background objects.
Results
We measured the following:
- Number of required operator interventions on a
30 second test sequence.
- Processing time per frame with operator
interventions disabled.
- Path length of a single player.
- The mean position error for a single player.
We used the mean average of five paths, obtained by
manually by five human operators, as the true-position
estimate.
Operators were performing manual tracking at 2 frames
per second – computer generated results were subsampled
to ensure unbiased comparison.
Results
Method
A
B
C
O1
O2
O3
O4
O5
No. of interventions
45
12
14
X
X
X
X
X
Speed [sec/frame]
0.424
0.175
0.229
X
X
X
X
X
Mean position error [cm]
35.3
33.4
28.0
9.8
9.0
7.3
8.1
6.7
Path length [m]
73
82
75
76
78
76
77
77
Legend:
A: Motion detection
B: RGB (color) tracking
C: Combination of color and template tracking.
O1, O2, O3, O4, O5 – human operators
X – not available
Conclusions
Combination of color and template tracking algorithms
requires few operator interventions, and results
obtained contain low amount of jitter.
It is therefore the most appropriate for use in the
automated player tracker.
Operator supervision is still required.
Results, obtained by automated tracker are less accurate
than those obtained manually and have to be filtered to
reduce jitter.
However, obtained results can be used for further
analysis by sports experts.
Conclusions
Radial distortion
Pan-tilt camera assumption:
camera
C
R
dr  cos( )  dR,   arctg( ),
H
dr  cos(arctg(
r1
R
)) dR.
H
R1
 dr   cos(arctg(
0
0
R
)) dR.
H


dr

H
dR
0

X
R
court
Inverse formula:
Integration:
R
R 
1

r1  H  ln
 1
.
H

H 

2
1
2
R1 
h (e
2

2 r1
h
e

) 1
r1
h
.

X
Similarity measure S
S
DFH
.
DGF  DFH
F (unknown object)
DGF
Legend:
DFH
white = 1
black = 0
G (background)
H (player)
Manual vs. computer tracking
Method A
M ethod B
0
0
2
2
4
4
6
6
8
8
10
10
12
12
20
22
24
26
Method C
28
30
32
20
22
24
26
28
30
32
0
2
4
6
8
A
B
C
Legend:
Green - human results
Black - computer results
10
12
20
22
24
26
28
30
32