vii
TABLE OF CONTENTS
CHAPTER
TITLE
DECLARATION
1
2
PAGE
i
DEDICATION
iii
ACKNOWLEDGEMENT
iv
ABSTRACT
v
ABSTRAK
vi
TABLE OF CONTENTS
vii
LIST OF TABLES
xi
LIST OF FIGURES
xiii
LIST OF ABBREVIATIONS
xix
LIST OF APPENDICES
xxi
INTRODUCTION
1
1.1 Background of the Problem
1
1.2 Statement of the Problem
5
1.3 Objectives of the Study
7
1.4 Scope of the Study
7
1.5 Summary
8
LITERATURE REVIEW
10
2.1 Face Verification
10
2.1.1 Review of Face Verification methods
12
2.2 Challenges in automatic face verification
17
2.3 Illumination problem
17
2.4 Illumination Normalization methods
18
2.4.1 Illumination variation modeling
19
2.4.2 Preprocessing normalization methods
25
2.4.3 Photometric normalization methods
30
viii
2.4.4 Local information Approach
2.5 Proposed Approaches
2.5.1 The Discrete Cosine Transform (DCT)
36
36
2.5.1.1 The One- Dimensional DCT
37
2.5.1.2 The Two- Dimensional DCT
39
2.5.1.3 Properties of DCT
40
2.5.2 Discrete Wavelet Transform
41
2.5.2.1 Wavelet Transform in One Dimension
42
2.5.2.2 The Continuous Wavelet Transform
43
2.5.2.3 Wavelet Transform in Two Dimension
44
2.5.3 Classified Appearance-based Quotient Image
3
34
48
2.5.3.1 Self-Quotient Image
48
2.5.3.2 Classified Appearance-based Quotient
Image
49
2.6 Summary
50
RESEARCH METHODOLOGY
52
3.1 Gap of the study
52
3.2 Research Design
53
3.3 Theoretical Framework
55
3.4 Research Framework
55
3.5 Proposed Solution
57
3.5.1 Pre-Processing
57
3.5.2 Illumination normalization
57
3.5.2.1 Discrete Cosine Transform (DCT)
58
3.5.2.2 Discrete Wavelet Transform (DWT)
60
3.5.2.3 Classified Appearance - based Quotient
Image (CAQI)
61
3.5.3 Verification process
62
3.5.4 Score Fusing
63
3.5.4.1 Max-Rule
63
3.5.4.2 Min-Rule
64
3.5.4.3 Average-Rule
64
3.6 Evaluation of proposed method
65
3.7 Instrumentation
65
3.8 Assumptions and Limitations
66
ix
4
3.9 Summary
66
EXPERIMENTS AND RESULTS
67
4.1 Databases for Face Verification
67
4.1.1 Yale Face Database B
68
4.1.2 XM2VTS Database
71
4.2 Implementation of Pre-processing step
73
4.3 Implementation of normalization methods
73
4.3.1 Execution of Discrete Cosine Transform (DCT) in
Logarithm Domain
74
4.3.2 Implementation of Discrete Wavelet Transform
(DWT) in Logarithm Domain
76
4.3.3 Executing of Classified Appearance - based
Quotient Image (CAQI)
80
4.4 Implementation of Face Verification process
87
4.4.1 Principal Component Analysis (PCA)
88
4.4.2 Linear Discriminant Analysis (LDA)
89
4.5 System Assessment Criteria
90
4.6 Score Generation
92
4.6.1 Decision Making
4.6.1.1 Normalized Correlation Criterion
4.7 Implementation of Score Fusion
4.7.1 Justification of the Score Fusion
93
93
94
94
4.8 Evaluation Protocols in Face Verification
94
4.9 Findings
96
4.9.1 The results of verification on CAQI
96
4.9.2 Verification results on DCT
100
4.9.3 Verification results on Wavelet
106
4.9.4 Fusion Results
122
4.9.5 Evaluation of proposed method
124
4.9.5.1 Comparison of the results of fusion three
normalization methods with the results of
fusion each pair of these methods
125
x
4.9.5.2 Comparison of the results of proposed
technique with some other methods
4.9.6 Summary
5
130
131
CONCLUSION AND FUTURE RECOMMENDATION
133
5.1 Conclusion
133
5.2 Future Recommendations
135
REFERENCES
APPENDICES A-C
137
148-162
xi
LIST OF TABLES
TABLE NO.
TITLE
PAGE
4.1
Five subsets of Yale database B.(Chen et al., 2006)
69
4.2
The First Configuration of XM2VTS
72
4.3
The Second Configuration of XM2VTS
73
4.4
Verification results on CAQI for Yale database B
96
4.5
Verification results on CAQI for XM2VTS
98
4.6
Verification results on DCT for XM2VTS
101
4.7
Verification results on DCT for Yale B
104
4.8
Verification results on Wavelet for XM2VTS by
changing db in level 8.
107
Verification results on Wavelet for XM2VTS by
changing level in db12.
109
Verification results on Wavelet for Yale B database by
changing level in db10.
113
Verification consequences on Wavelet for Yale B by
changing db in level eight.
114
The outcomes of the verification process on the output
of three normalization techniques on XM2VTS database
by using selected components
119
The outcomes of the verification procedure with selected
components on the output of three normalization
techniques on Yale B database
120
4.14
The results of fusion procedure for XM2VTS.
122
4.15
The consequences of fusion procedure on Yale database
B.
123
The results of fusion DCT and DWT on XM2VTS.
125
4.9
4.10
4.11
4.12
4.13
4.16
xii
4.17
The results of fusion DWT and CAQI on XM2VTS.
126
4.18
The results of fusion DCT and CAQI on XM2VTS.
126
4.19
The outcomes of fusion DCT and CAQI on Yale B.
127
4.20
The consequences of fusion DCT and DWT on Yale
database B.
128
The consequences of fusion DWT and CAQI on Yale
database B.
128
4.21
xiii
LIST OF FIGURES
FIGURE NO.
1.1
TITLE
PAGE
Classifying of facial appearance (Nishiyama et al., 2008).
(a) "Diffuse reflection"; (b) "Specular reflection"; (c)
"Attached shadow"; (d) "Cast shadow"
6
2.1
Face verification flow (Short, 2006).
11
2.2
Example of the illumination variation from Yale
database B
18
2.3
1-D cosine basic function (N=8)
38
2.4
Two dimensional cosine basis function (N=8)
40
2.5
Calculate of 2-D DCT applying separability property
41
2.6
The block diagram of 2-D wavelet transform
46
2.7
The results of decomposition
47
2.8
The synthesis filter bank
47
3.1
Gap of the study
53
3.2
The general diagram of proposed methodology
55
3.3
The details of research framework
56
4.1
Examples of the images of Yale Database B with four
various appearance due to illumination variation.(a)
"Diffuse Reflection"; (b) "Specular Reflection"; (c) "Cast
Shadow"; (d) "Attached Shadow"
68
4.2
Example of subset 1 of Yale database B
69
4.3
Example of subset 2 of Yale database B
69
4.4
Example of subset 3 of Yale database B
70
xiv
4.5
Example of subset 4 of Yale database B
70
4.6
Example of subset 4 of Yale database B
70
4.7
Samples of the images of XM2VTS database with four
various appearance due to illumination variation.(a)
"Diffuse Reflection"; (b) "Specular Reflection"; (c) "Cast
Shadow"; (d) "Attached Shadow"
72
Zigzag sequence for produce vector of DCT coefficients.
(Joshi, 2006)
74
Compensated logarithm images with various Ddis on Yale
Database B: (a) main image; (b) Ddis=3; (c) Ddis=5; (d)
Ddis=10; (e) Ddis=20; (f) Ddis=25;
(g) Ddis=45.
75
Compensated logarithm images with various Ddis on
XM2VTS:
(a) main image; (b) Ddis=3; (c) Ddis=5; (d)
Ddis=10; (e) Ddis=20; (f) Ddis=25;
(g) Ddis=45.
75
Block diagram of implementation of Discrete Wavelet
Transform
76
Implementation of DWT method in various levels using
wavelet by Daubechies order 2. (a) one-level
decomposition;(b) three-level decomposition.
77
The results of using DWT normalization method on Yale
database B with different levels and db8. (a) original
image; (b) level 1 ; (c) level 3 ; (d) level 5 ; (e) level (7).
78
The results of using DWT normalization method on Yale
database B with different dbs and level 8. (a) Original
image; (b) db 3; (c) db 5; (d) db 9;
(e) db 12.
78
The results of using DWT normalization method on
XM2VTS with different levels and db8. (a) Input image;
(b) level 1; (c) level 3; (d) level 5;
(e) level 7.
79
The results of using DWT normalization method on
XM2VTS with different dbs and level 8. (a) Original
image; (b) db 3; (c) db 5; (d) db 9; (e) db 12.
79
4.17
Criterion for sorting of photometric factors.
81
4.18
Flowchart of CAQI implementation
81
4.19
The details of CAQI implementation (Nishiyama et al.,
2008)
82
Flowchart of the photometric linearization procedure for all
input images.(Mukaigawa et al., 2006)
83
4.8
4.9
4.10
4.11
4.12
4.13
4.14
4.15
4.16
4.20
xv
4.21
4.22
4.23
4.24
4.25
4.26
4.27
4.28
4.29
4.30
4.31
4.32
4.33
4.34
4.35
4.36
Estimation of the coefficients by using center of gravity
method.
84
Some examples of applying CAQI on Yale database B.
(a) input images; (b) consequences
86
Example of using CAQI on XM2VTS; (a) original image
(b) result
87
(a) The manner of distribution of clients and imposters to
select the threshold; (b) The carve of Receiver Operating
Characteristic (ROC).
92
Distribution of clients and imposters to select the threshold
for verification procedure on CAQI for Yale database B.
97
The ROC carve for verification process on CAQI for Yale
Database B
97
Distribution of clients and imposters to select the threshold
for verification procedure on CAQI for XM2VTS database
99
The ROC carves for verification process on CAQI for
XM2VTS database.
99
Variation HTERE versus varying discarded coefficients
for XM2VTS database.
101
Variation HTERT versus changing discarded coefficients
for XM2VTS database.
102
The distribution of clients and imposters to estimate the
threshold for verification procedure on DCT for XM2VTS
database.
102
The ROC carves for four different values of eliminated
coefficients for XM2VTS database.
103
Variation HTERE versus varying eliminated coefficients
for Yale B database
104
Variation HTERT versus varying eliminated coefficients
for Yale B database
105
The distribution of clients and imposters to select the
threshold for verification process on DCT for Yale B
database.
105
The bar chart of variation of HTERE versus changing the
number of discarded DCT coefficients for Yale B database.
106
xvi
4.37
4.38
4.39
4.40
4.41
4.42
4.43
4.44
4.45
4.46
4.47
4.48
4.49
4.50
4.51
4.52
4.53
Variation HTERE versus varying db for XM2VTS
database.
108
Variation HTERT versus changing db coefficients for
XM2VTS.
108
Variation HTERE versus varying level for XM2VTS
database.
109
Variation HTERT versus changing level for XM2VTS
database.
110
The distribution of clients and imposters to select the
threshold for verification process on DWT for XM2VTS
database.
110
The ROC carves for four different values of levels in
db12 for XM2VTS database.
111
The bar chart verification error rate against varying of
level for XM2VTS database.
111
The verification error rate against varying the number of
dbs for XM2VTS database.
112
Variation HTERE versus varying level for Yale B
database.
113
Variation HTERT versus changing level for Yale
database B.
114
Variation HTERE versus varying db in level8 for Yale B
database.
115
Variation HTERE versus varying db in level8 for Yale B
database.
115
The distribution of clients and imposters to select the
threshold for verification process on DWT for Yale B
database.
116
The identification error rate versus varying the amount of
db for Yale database B.
117
The verification error rate against varying the number of
level and db10 for Yale B database.
117
The ROC carves for three normalization methods in Yale
database B.
118
The evaluation results of VRE for three normalization
methods on XM2VTS.
119
xvii
4.54
The evaluation results of VRT for three normalization
methods on XM2VTS.
120
4.55
The comparison results of VRE for Yale B.
121
4.56
The comparison results of VRT for three normalization
on Yale B.
121
Comparison results of fusion processes on XM2VTS
database.
(a) HTERE, (b) HTERT.
123
Comparison results of fusion processes on Yale B
database. (a) HTERT, (b) HTERE.
124
The comparison results of fusion two illumination
normalization methods and fusion three illumination
normalization techniques on XM2VTS.
127
The comparison results of fusion two illumination
normalized methods and fusion three illumination
normalization techniques on Yale B.
129
The comparison result of proposed method with HE,
HF, AS, DCT, Wavelet, and CAQI on XM2VTS database
(Short, 2006).
130
The comparison result of proposed method with "HE",
"Linear subspace", "Cones-attached", "Illumination ratio
images", and "Quotient illumination relighting" on Yale
B Database(Chen et al., 2006).
131
Variation FARE versus changing discarded coefficients
for XM2VTS database.
149
Variation FRRE versus changing discarded coefficients
for XM2VTS database.
149
Variation FART versus changing discarded coefficients
for XM2VTS database.
150
Variation FRRT versus changing discarded coefficients
for XM2VTS database.
150
Variation FARE versus changing discarded coefficients
for Yale database B.
151
Variation FRRE versus changing discarded coefficients
for Yale database B.
151
Variation FART versus changing discarded coefficients
for Yale database B.
152
4.57
4.58
4.59
4.60
4.61
4.62
A.1
A.2
A.3
A.4
A.5
A.6
A.7
xviii
A.8
Variation FRRT versus changing discarded coefficients
for Yale database B.
152
B.1
Variation FARE versus varying db for XM2VTS database.
154
B.2
Variation FRRE versus varying db for XM2VTS database.
154
B.3
Variation FART versus varying db for XM2VTS database.
155
B.4
Variation FRRT versus varying db for XM2VTS database.
155
B.5
Variation FARE versus varying the number of level for
XM2VTS database.
156
Variation FRRE versus varying the number of level for
XM2VTS database.
156
Variation FART versus varying the number of level for
XM2VTS database.
157
Variation FRRT versus varying the number of level for
XM2VTS database.
157
Variation FARE versus varying the value of db for Yale
database B.
158
Variation FRRE versus varying the value of db for Yale
database B.
158
Variation FART versus varying the value of db for Yale
database B.
159
Variation FRRT versus varying the value of db for Yale
database B.
159
Variation FARE versus varying the number of level for
Yale database B.
160
Variation FRRE versus varying the number of level for
Yale database B.
160
Variation FART versus varying the number of level for
Yale database B.
161
B.6
B.7
B.8
B.9
B.10
B.11
B.12
B.13
B.14
B.15
xix
LIST OF ABBREVIATIONS
HE
-
Histogram Equalization
LBP
-
Local Binary Patterns
WT
-
Wavelet Transforms
SH
-
Spherical Harmonics
QI
-
Quotient Image
DCT
-
Discrete Cosine Transform
HF
-
Homomorphic Filtering
CAQI
-
Classified Appearance-based Quotient Image
FLD
-
Fisher Linear Discriminant
MAP
-
Maximum A Posterior
QIR
-
Quotient Illumination Relighting
GIIS
-
Generic Intrinsic Illumination Subspace
MQI
-
Morphological Quotient Image
PCA
-
Principal Component Analysis
AHE
-
Adaptive Histogram Equalization
BHE
-
Block-based Histogram Equalization
LTP
-
Local Ternary Patterns
LDCT
-
Local Discrete Cosine Transform
DWT
-
Discrete Wavelet Transform
IRM
-
Illumination Reflectance Model
RLI
-
Rule of Lighting Invariance
IVIW
-
Improved Variable Illumination on Wavelet
HMV
-
Homomorphic Vertical filtering
HMH
-
Homomorphic Horizontal filtering
SQI
-
Self-Qutient Image
LDA
-
Linear Discriminant Analysis
EGFC
-
Ensemble based Gabor Fisher Classifier
xx
ANN
-
Artificial Neural Networks
HMM
-
Hidden Markov Models
LEM
-
Line Edge Map
SVM
-
Support Vector Machine
MCS
-
Multiple Classifier Systems
MRF
-
Markov Random Field
FAR
-
False Acceptance Rate
FRR
-
False Rejection Rate
EER
-
Equal Error Rate
HTER
-
Half Total Error Rate
GT
-
Global Threshold
xxi
LIST OF APPENDICES
APPENDIX
A
TITLE
PAGE
Plots of variation of FARE, FRRE, FART, and FRRT
versus varying the value of eliminated coefficients for
DCT method for XM2VTS and Yale database B
B
144
Plots of variation of FARE, FRRE, FART, and FRRT
versus varying the value of db and number of level for
C
XM2VTS and Yale database B
149
List of Publications
159