Matakuliah
Tahun
Versi
: H0434/Jaringan Syaraf Tiruan
: 2005
:1
Pertemuan 11
OPTIMASI KINERJA
1
Learning Outcomes
Pada akhir pertemuan ini, diharapkan mahasiswa
akan mampu :
• Menjelaskan konsep optimasi kinerja.
2
Outline Materi
• Steepest Descent Method.
• Contoh Aplikasi.
3
Basic Optimization Algorithm
xk + 1 = xk + k pk
or
xk =  xk + 1 – x k  = kpk
kp k
xk +1
xk
pk - Search Direction
k - Learning Rate
4
Steepest Descent
Choose the next step so that the function decreases:
Fx k + 1   F xk 
For small changes in x we can approximate F(x):
T
F xk + 1  = F xk + x k   F xk  + gk x k
where
g k   Fx 
x = xk
If we want the function to decrease:
T
T
g k xk =  kg k pk  0
We can maximize the decrease by choosing:
pk = –gk
x k + 1 = xk – k g k
5
Example
2
2
F x  = x1 + 2 x1 x 2 + 2x 2 + x1
x 0 = 0.5
 = 0.1
0.5
F  x  =

F x 
 x1

F x 
 x2
=
2x 1 + 2x2 + 1
g0 =  F x 
2x 1 + 4x 2
= 3
x = x0
3
x 1 = x 0 – g 0 = 0.5 – 0.1 3 = 0.2
0.5
3
0.2
x2 = x1 – g1 = 0.2 – 0.1 1.8 = 0.02
0.2
1.2
0.08
6
Plot
2
1
0
-1
-2
-2
-1
0
1
2
7
Newton’s Method
T
1 T
F  xk + 1  = F  xk +  xk   F  xk  + g k  x k + --  xk A k x k
2
Take the gradient of this second-order approximation
and set it equal to zero to find the stationary point:
gk + Ak xk = 0
–1
x k = – Ak g k
xk + 1 = xk – A–k 1 gk
8
Example
2
2
F x  = x1 + 2 x1 x 2 + 2x 2 + x1
x 0 = 0.5
0.5
F  x  =
x1 =

F x 
 x1

F x 
 x2
0.5
22
–
0.5
24
–1
=
2x 1 + 2x2 + 1
2x 1 + 4x 2
g0 =  F x 
= 3
x = x0
3
A= 22
24
3
0.5
1 – 0.5 3
0.5
1.5
–1
=
–
=
–
=
3
0.5
– 0.5 0.5 3
0.5
0
0.5
9
Plot
2
1
0
-1
-2
-2
-1
0
1
2
10
Non-Quadratic Example
4
Fx  =  x2 – x1  + 8x 1 x2 – x1 + x2 + 3
1
x =
Stationary Points:
– 0.42
0.42
2
x =
– 0.13
0.13
F(x)
F2(x)
2
2
1
1
0
0
-1
-1
-2
-2
-1
0
0.55
– 0.55
3
x =
1
2
-2
-2
11
-1
0
1
2