Eigenfaces (2)
Face Recognition and Biometric Systems
Plan of the lecture
PCA – repeated
Back projection
Feature vectors comparison
Methods based on the Eigenfaces
Face Recognition and Biometric Systems
Eigenfaces
Feature extraction method
Redundant information reduction
(dimensionality reduction)
Two stages:


training
projection (feature extraction)
Possibility of back projection
Face Recognition and Biometric Systems
Eigenfaces: training
C00 ... C0n
...
... ...
Cn0 ... Cnn
Normalised
images
Covariance
matrix
Eigenfaces
Face Recognition and Biometric Systems
Eigenfaces: laboratory
Input data:

normalised images, number and size
Implementation:



covariance matrix
eigenvalues and eigenvectors (OpenCV)
output buffer – eigenvectors / eigenfaces
Testing:

dimensionality reduction
Face Recognition and Biometric Systems
Eigenfaces: feature extraction
K1
K2
K3
...
Scalar products between
normalised image and
eigenvectors
...
Feature vector
Face Recognition and Biometric Systems
Eigenfaces: feature extraction
 matrix can be cut to reduce dimensions

’
’’
Feature vector element is a scalar product:
wi  v i  x
T
w  ψ' x
T
Feature vector – cut projected vector x’
Face Recognition and Biometric Systems
Back projection: example
2-dimensional space:

eigenvectors:
2 2
[ ; ]
2 2

2 2
[
; ]
2 2
average vector [0, 0]
Vectors projection:

[3; 1], [-2; -2], [10, 9]
Back projection
Face Recognition and Biometric Systems
Back projection
Feature vector -> face image
N'
x P   wi  v i  μ
i 1
x P  ψ'w  μ
Projection error – difference between
original and recovered image
  x  xP
Face Recognition and Biometric Systems
Back projection: 2D
Face Recognition and Biometric Systems
Back projection: 2D
Face Recognition and Biometric Systems
Back projection: 2D
Face Recognition and Biometric Systems
Back projection: 2D
Face Recognition and Biometric Systems
Back projection: face image
Feature vector – face description

information reduction
Back projection: face image recovered
from feature vector

reduced information are lost
Projection error:



depends on similarity to the training set
2D example
face images
Face Recognition and Biometric Systems
Back projection: detection
Back projection of images:


face -> slightly modified face image
flower -> image similar to a face
Back projection error is higher for
non-face images
Can be used as a verifier

threshold of accepted projection error
Face Recognition and Biometric Systems
Feature vectors comparison
Similarity based on distance metric
1
S ( w1 , w 2 ) 
1  dist w1 , w 2 



Euclidean distance (norm L2)
Mahalanobis distance
angle between vectors
Classifier-based similarity

SVM, ANN
Face Recognition and Biometric Systems
Feature vectors comparison
Euclidean distance (L2 norm)

distance between two points
in Euclidean space
N'
dist (w1 , w 2 )   ( w1i  w2i )
2
i 1
Face Recognition and Biometric Systems
Feature vectors comparison
Mahalanobis distance

variance normalised in all directions
(a.k.a. whitening)
( w1i  w2i )
dist (w1 , w 2 )  
i 1
i
N'
2
 - eigenvalue
Face Recognition and Biometric Systems
Feature vectors comparison
Weak whitening:
( w1i  w2i )
dist (w1 , w 2 )  
i 1
i
N'
2
Eigenvalue filter:
i
dist (w1 , w 2 )   ( w1i  w2i )
2
i 1
(i  N ' )
N'
2
Face Recognition and Biometric Systems
Feature vectors comparison
Based on cosine of the angle
w1  w 2
dist (w1 , w 2 ) 
| w1 || w 2 |

feature vector length not taken into
consideration
Face Recognition and Biometric Systems
Feature vectors comparison
Classifiers



single vector can be classified
two or more classes
long training stage
One person – one class

training necessary when gallery is changed
Similarity between any two vectors


universal training
two vectors -> one vector
Face Recognition and Biometric Systems
K11
K12
...
K1n
K21
K22
...
K2n
SVM
The same
class
Different
classes
K11 - K21
K12 - K22
...
K1n - K2n
SVM
The same
class
Different
classes
Feature vectors comparison
Training set for classifiers:
1.
2.
3.
4.
5.
classified samples
intra-personal pairs
extra-personal pairs
differences within each pair
training with two sets:
intra-personal and extra-personal
Face Recognition and Biometric Systems
Feature vectors comparison
A pair of feature vectors:


many metrics, various results
metric as a separate feature extraction
method
Metrics fusion


weighted mean of single results
classifiers again
Testing necessary
Face Recognition and Biometric Systems
Eigenfaces – improvements
Main drawbacks:




holistic method
face topology not taken into account
statistical analysis of differences between
images in the training set
character of differences not taken into
account
Face Recognition and Biometric Systems
Example
Face Recognition and Biometric Systems
Example: PCA
Face Recognition and Biometric Systems
Example: PCA not helpful
Face Recognition and Biometric Systems
Example:
Linear Discriminant Analysis
Face Recognition and Biometric Systems
Thank you for your attention!
Next time:


Error function minimisation
Methods derived from Eigenfaces
Face Recognition and Biometric Systems