Human Detection in
Surveillance Applications
Ashish Desai
EE392J Final Project
Problem and Motivation



Had many petty vandals cause damage
to cars in my parking garage
Landlord added cameras, but did not
deter vandals
What if we could count and have
photos of all the people in the garage at
any given time?
Breakdown into Class Concepts



Simply, this is a foreground /
background segmentation problem
Perhaps use color based segmentation
Perhaps use motion based
segmentation
Methodology

Captured 25 second
video at 10 frames
per second of 1, 2,
or 3 people walking
around in my
apartment.
Methodology

Used Staufer and Grimson color based
segmentation





Used 5 gaussians in RGB space
Used maximum of 1.5 std. dev for classification
Used alpha = 0.7
Combined with connectivity requirements (each
pixel must touch 4 others)
Combined with size requirements (each group of
pixels must be larger than 100)
Methodology

Very simplified motion based segmentation

Used block matching between current and next
frame




16x16 block size, +16/-15 full search, SAD criterion
Eliminate blocks with fewer than 8 pixel
movement
Combine with size requirements (must be greater
than 4 blocks)
Note: did not use K-means clustering (and should
have used log search) to improve real-time
capabilities
Results of Individual
Segmentations

Color based segmentation



Worked pretty well (I spent a lot of time tweaking
this)
Had problems with large areas of occlusion,
shadows
Motion based segmentation



Purposely set to allow “false positives”
Simple methodology could not handle global
motion from person bumping the camera
Blinds (possible interlace artifact) caused
problems, aspect ratio (block-based)
Combinational Methodology



Take both of the initial methodologies and
create a confidence weighting (those pixels in
the center of a group have higher weighting)
Combine the two weights with more
preference to color weights (color had better
initial performance)
Apply threshold, connectivity and size
constraints.
Overall Block Diagram
Input Sequence (RGB)
RGB -> YUV
Motion Estimation
Color Gaussian Segmentation
Connectivity/ Size Requirements
Connectivity/ Size Requirements
Weighting
Weighting
Threshold
Output
Overall Results



Performed pretty well especially with lateral
movement and objects further from camera
Still had problems with large occlusions (from
color) and aspect ratio changes (from motion)
but better than respective individuals
Eliminated major shadow and global motion
issues by combining the two
Results - Videos
Future Enhancements
Improvements (if time allows)




Use affine parameters for motion based
Use log search (if still use block) for
more real time
Use K means clustering to segment,
rather than simple threshold
Use temporal knowledge (i.e. use object
identification from previous frame or
frames)