Gaussian Classifiers
•
Let’sThe
imagine
the Bayesian
Decision Rule (BDR) for the case of
Gaussian
classifier
two multivariate
in this case Gaussian case:
$ 1
"
exp#& ( x & *i )T % i&1 ( x & *i )!
( 2
'
(2+ ) d | % i |
1
PX |Y ( x | i ) )
• the BDR
i * (x ) ) arg max,log PX |Y (x | i ) . log PY (i )i
• becomes
0 1
3 2
1
2
& log(
l (2+ )d %i . log
l PY (i )1
2
4
i * (x ) ) arg max /& (x & *i )T %i&1 (x & *i )
i
11
’11
Tuesday, November 8, 2011
Arshia Cont: Survey of Machine Learning
57
Gaussian Classifiers
The Gaussian classifier
this can be written as
i * (x ) & arg min!d i (x , $i ) % # i "
!" " "
#
!"$%&"'"()(#*
+,-.!/"!"#$%$&'(
i
with
d i (x , y ) & (x ' y )T (i'1 (x ' y )
# i & log(2) )d (i ' 2 log PY (i )
the optimal rule is to assign x to the closest class
closest is measured with the Mahalanobis distance di(x,y)
to which the # constant is added to account for the class
prior
12
’11
Tuesday, November 8, 2011
Arshia Cont: Survey of Machine Learning
58
Gaussian Classifiers
’11
Tuesday, November 8, 2011
Arshia Cont: Survey of Machine Learning
59
Gaussian Classifiers
•
In detail:
!
"
T −1
i (x) = arg min (x − µi ) Σ (x − µi ) − 2 log PY (i)
i
! T −1
"
T −1
T −1
T −1
= arg min x Σ x − x Σ µi − µi Σ x + µi Σ µi − 2 log PY (i)
i
! T −1
"
T −1
T −1
= arg min x Σ x − 2µi Σ x + µi Σ µi − 2 log PY (i)
i


∗

 T −1
1
T
−1
= arg max 
µi Σ x − µi Σ µi + 2 log PY (i)


i
& '( ) &2
'(
)
T
wi
’11
Tuesday, November 8, 2011
wi0
Arshia Cont: Survey of Machine Learning
60
Gaussian Classifiers
The Gaussian classifier
!"#$%"&"'(')*
+,-.!/"!"#$%$&'(
in summary,
i * ( x) ! arg max g i ( x)
i
• with
g i ( x) ! wiT x " wi 0
wi ! # $1%i
1 T
wi 0 ! $ %i # $1%i " log PY (i )
2
• the BDR is a linear function or a linear discriminant
15
’11
Tuesday, November 8, 2011
Arshia Cont: Survey of Machine Learning
61
Gaussian Classifiers
Group Homework 0
Find the Geometric equation for the
hyperplane separating the two classes
for the linear discriminant of the Gaussian
classifier.
Hint: This is the set such that
’11
Tuesday, November 8, 2011
Arshia Cont: Survey of Machine Learning
62