Vision based rock,
paper, scissors game
Patrik Malm
Standa Mikeš
József Németh
István Vincze
1/12
Task
Create a vision based rock, paper, scissor
game
Realtime webcam image-processing
2/12
Flowchart
3/12
Initiation
Smooth the image
Color-space conversion - L*u*v
The background is captured so that it can be
subtracted in the following segmentation
4/12
Hand segmentation
Normal Bayesian classifier
Mathematical morphology to decrease noise
Use labeling to find the largest coherent component
(or the two largest components in the 2 player
game)
Cut the hand at the wrist
5/12
Isolating the hand
Cut the arm where the size is 70% of the maximum projection
value
It works best for arms with vertical orientation
Improved solution using PCA which works for rotated arms as
well
6/12
Isolating the hand
The approximation of hand rotation is done using
principal component analysis (PCA)
It is computed using eigenvectors of the second-order
moment matrix (covariance matrix)
7/12
Hand tracking
Find the centre point of the hand in each
frame
Analyze the movement by looking at the
relative change in position over a few frames
The hand tracking is used to countdown
8/12
Gesture identification
Logarithmic values of three of the Hu set of
invariant moments
Bayesian classifier which then determines the
current hand gesture
9/12
Computer player
Computer choose randomly (current)
Markov chain to recognize the human
opponent’s game strategy (implemented but
not added)
10/12
Known issues
Skin color variations
Sensitivity to lighting
Background artifacts (skin or skin-like colored
objects)
11/12
Thank you for your attention
Bibliography
A.A. Argyros, M.I.A. Lourakis, Real time Tracking of Multiple
Skin-Colored Objects with a Possibly Moving Camera, in
proceedings of the European Conference on Computer Vision
(ECCV 04), Springer-Verlag, vol. 3, pp. 368-379, Prague, Chech
Republic,May,11-14,2004.
OpenCV Library Wiki, "http://opencvlibrary.sourceforge.net/".
12/12