Tracking with CACTuS on
Jetson
Running a Bayesian multi object tracker on an
embedded system
School of Information Technology & Mathematical Sciences
September 2015
David Webb
Supervisor: David Kearney
Object Tracking Overview
What is object tracking?
Object tracking, also referred to as video tracking, is a sub
domain of computer vision, concerned with detecting,
correlating and tracing the path of an object(or objects) as
it travels, relative to the background, over multiple frames
of video (or a sequence of images).
Object Tracking Overview
How is object tracking used?
Real world applications of object tracking include
• Security and Surveillance
 Loitering
 Suspicious packages
• Business intelligence
 People counting
 Traffic
 Theft
Object Tracking Overview
How is object tracking used?
• Military
 Situational awareness
• Robotics
 Object manipulation
 Environment awareness
• Human Computer Interaction
 Gesture Recognition
 Gaze Tracking
Object Tracking Overview
What is involved in object tracking?
• Detection
 Determining the existence of an object at a given location (or pixel)
in an image
• Tracking
 Associating the location of an object over multiple images
• Recognition
 Classification of the nature of the object being tracked
Object Tracking Overview
Classic detection tracking recognition
• Classic algorithms suffer from ambiguity in measurements
due to noise
• In the presence of noise, classic trackers make hard and
fast assumptions about the nature of the target object
• This can lead to poor robustness, where the tracker is
unable to correctly maintain the track of an object
Object Tracking Overview
Competitive Attentional Correlation Tracker using
Shape (CACTuS)
• CACTuS is designed to avoid problems associated with
measurements taken in the presence of noise by using a
Bayesian approach to propagate uncertainty through the
tracking chain.
CACTuS Tracker
CACTuS Multiple Object Tracker
• Shape Estimating Filter (SEF)
 Each SEF is capable of tracking and describing one object
 Describes an object’s location, velocity and shape
 Uses a Bayesian approach to propagate uncertainty by
representing each state as a 2D array of probabilities
• The following operations are performed on each SEF
 Prediction
 Update
 Observation
CACTuS Tracker
CACTuS Multiple Object Tracker
• CACTuS Combines Multiple SEFs
 SEFs are compelled by a competitive mechanism to focus on
different objects
• Multiple SEFs in Parallel
 The each phase of each SEF is independent of each other SEF,
allowing for SEF’s to be computed in parallel, excluding the
competition phase
CACTuS Tracker
CACTuS Algorithm
• Computational Complexity
 SEFs perform the prediction phase using 2D convolution operations
on several 2D matrices
 The complexity of the convolution of a 2D matrix A with 2D matrix B
is approximately O(MNmn), where A is M x N and B is m x n
CACTuS Tracker
Demonstration
CACTuS Tracker
CACTuS Algorithm
• Current Implementations and Performance
 In the testing of the current CACTuS implementation, the following
performance figures were observed on our test PC
Number
of SEFs
Implementation
Time per frame
(ms) (average)
Prediction
phase time per
frame (ms)
16
MATLAB
150
69
16
C++ CPU
670
547
16
C++ OpenMP
139
107
16
C++ OpenMP + SSE
129
107
PC Specs: Intel Core i7 4790K 4.0GHz
16GB DDR3-1600 RAM
NVIDIA GeForce GTX960 2GB DDR5
NVIDIA Jetson Platform
NVIDIA Jetson TK1 DevKit
• Tegra K1 System on Chip (SoC)
 4 Plus 1 Quad Core ARM Cortex A15 CPU
 Kepler GPU with 192 CUDA Cores
• 2 GB memory
• 16GB eMMc storage
• Runs custom Ubuntu 14.04
NVIDIA Jetson Platform
Why the Jetson?
• As the majority of computational time is taken by the
convolution operations, and the convolution operation is
easily parallelised, it is expected that the GPU of the Jetson
platform will provide increased performance due to it’s
highly parallel nature
• The Jetson is also a convenient form factor and conforms to
a low power budget of only 10W
Tracking on Jetson with CACTuS
Project Goals
• Create an implementation of CACTuS that compiles and runs on the
Jetson platform
• Accelerate the implementation of CACTuS using CUDA
• Achieve a minimum of 10 frames per second of processing with 2 or
more SEFs on the Jetson platform
Performance Enhancement
Using CUDA to increase the performance of CACTuS
• Performance profiling was used to focus efforts.
Number
of SEFs
Implementation
Time per frame
(ms) (average)
Prediction
phase time per
frame (ms)
16
MATLAB
150
69
16
C++ CPU
670
547
16
C++ OpenMP + SSE
129
107
16
C++ CUDA Conv
65
24
16
C++ CUDA + OMP
29
15
16
C++ CUDA Predict
27
13
PC Specs: Intel Core i7 4790K 4.0GHz
16GB DDR3-1600 RAM
NVIDIA GeForce GTX960 2GB DDR5
Performance on Jetson
Benchmarking the performance of CACTuS on Jetson
• A hybrid CPU and GPU approach performed best, likely
due to the reduction of memory copy overheads
Number
of SEFs
Implementation
Time per frame
(ms) (average)
Prediction
phase time per
frame (ms)
16
MATLAB Desktop
150
69
16
C++ CPU Desktop
670
547
16
C++ CPU Jetson
808
530
16
C++ OMP Jetson
227
129
16
C++ CUDA Conv
16
C++ CUDA Predict
302
159
16
C++ Hybrid
193
79
Performance on Jetson
By reducing the number of SEFs, performance can be gained,
this is a trade off that reduces the number of targets that can
be tracked and the overall accuracy and track latency of the
tracker
Number
of SEFs
Implementation
Time per frame
(ms) (average)
Prediction
phase time per
frame (ms)
4
MATLAB Desktop
40
18
4
C++ OMP Desktop
35
33
4
C++ OMP Jetson
77
45
4
C++ CUDA Predict
89
41
4
C++ Hybrid
60
33
Conclusion
The performance of the platform is acceptable for marginal usage as a
tracking platform. Achieving 16 frames per second with 4 SEFs, the 10 fps
goal has been exceeded.
As a side note, running CACTuS with CUDA acceleration on the desktop in
under 30ms per frame for 16 SEFs, demonstrates Real Time processing of
the synthetic video, a feat which has not been previously achieved with this
version of the algorithm.