Matakuliah
Tahun
Versi
: H0434/Jaringan Syaraf Tiruan
: 2005
:1
Pertemuan 15
ADAPTIVE RESONANCE THEORY
1
Learning Outcomes
Pada akhir pertemuan ini, diharapkan mahasiswa
akan mampu :
• Menjelaskan mengenai jaringan Adaptive
Resonance Theory ( ART ).
2
Outline Materi
• Arsitektur ART
• Layer 1 ART
• Layer 2 ART
3
Basic ART Architecture
4
ART Subsystems
Layer 1
Normalization
Comparison of input pattern and expectation
L1-L2 Connections (Instars)
Perform clustering operation.
Each row of W1:2 is a prototype pattern.
Layer 2
Competition, contrast enhancement
L2-L1 Connections (Outstars)
Expectation
Perform pattern recall.
Each column of W2:1 is a prototype pattern
Orienting Subsystem
Causes a reset when expectation does not match input
Disables current winning neuron
5
Layer 1
6
Layer 1 Operation
Shunting Model
1
d n (t)
1
+ 1
1
2:1 2
1
- 1 - 1 2
 --------------- = – n (t) +  b – n (t)   p + W a t  – n (t) + b  W a  t 
dt
Excitatory Input
(Comparison with Expectation)
1
+
Inhibitory Input
(Gain Control)
1
a = hardlim  n 
 1, n  0
+
hardlim  n  = 
 0, n  0
7
Excitatory Input to Layer 1
2:1 2
p + W a  t
Suppose that neuron j in Layer 2 has won the competition:

2:1 2
2:1
2:1
2:1
W a = w 2:1
w
w
w


2
1
2
j
S
0
0
2:1
= wj
(jth column of W2:1)
1

Therefore the excitatory input is the sum of the input pattern
and the L2-L1 expectation:
2:1 2
2:1
p +W a = p +wj
8
Inhibitory Input to Layer 1
Gain Control
-
1
2
 W a t 



11 1
1
- 1
W = 11
11 1
The gain control will be one when Layer 2 is active (one
neuron has won the competition), and zero when Layer 2 is
inactive (all neurons having zero output).
9
Steady State Analysis: Case I
1
dn i
1
+ 1
1 
 -------- = – n i +  b – ni  p i +
dt

S
2
S
2
2:1 2 
1
- 1
2
w
a
–

n
+
b

a
 i j j  i
 j

j= 1
j=1
Case I: Layer 2 inactive (each a2j = 0)
1
dn i
1
+ 1
1
 -------- = – n i +  b – n i  p i 
dt
In steady state:
+ 1
0 = –
1
ni
+ 1
+ b –
1
ni  pi
= – 1 +
1
p i n i
+ 1
+ b pi
1
ni
b pi
= -------------1 + pi
Therefore, if Layer 2 is inactive:
1
a = p
10
Steady State Analysis: Case II
Case II: Layer 2 active (one a2j = 1)
1
d ni
1
+ 1
1
2:1
1 - 1
--------- = – n i +  b – ni   pi + wi j  –  n i + b 
dt
In steady state:
0 = –
1
+ 1
ni +  b
= –  1 + pi +
1
2:1
1 - 1
– n i   pi + wi j  –  n i + b 
2:1
1
+ 1
2:1
- 1
wi j + 1 n i +  b  p i + wi j  – b 
+ 1
2:1
- 1
b  pi + wi j  – b
1
n i = -----------------------------------2:1
-----------2 + p i + w i j
We want Layer 1 to combine the input vector with the expectation from
Layer 2, using a logical AND operation:
+ 1
- 1
b

2

–
b 0
1
2:1
n i<0, if either w i,j or pi is equal to zero.
+ 1
- 1 + 1
b

2


b  b
+ 1 - 1
n1i>0, if both w2:1i,j or pi are equal to one.
b – b 0
Therefore, if Layer 2 is active, and the biases satisfy these conditions:
a1 = p  w 2:1
j
11
Layer 1 Summary
If Layer 2 is inactive (each a2j = 0)
1
a = p
If Layer 2 is active (one a2j = 1)
a1 = p  w 2:1
j
12
Layer 1 Example
 = 1,
+b1
= 1 and
-b 1
2:1
W
= 1.5
=
11
01
0
1
p =
Assume that Layer 2 is active, and neuron 2 won the competition.
1
dn 1
1
1
2:1
1
0.1  --------- = – n1 + 1 – n1  p 1 + w1  2  –  n 1 + 1.5 
dt
1
1
1
1
1
d n1
1
--------- = – 30n1 – 5
dt
1
dn 2
1
--------- = – 40n 2 + 5
dt
= – n1 + 1 – n1  0 + 1  – n 1 + 1.5  = – 3n 1 – 0.5
1
dn 2
1
1
2:1
1
0.1  -------- = – n 2 +  1 – n 2   p2 + w 2 2  –  n 2 + 1.5 
dt
1
1
1
1
= – n 2 +  1 – n 2   1 + 1  –  n 2 + 1.5  = – 4n2 + 0.5
13
Example Response
0.2
1
1
–40t
n 2 t  = --  1 – e

8
0.1
0
-0.1
1
1
– 30t
n1  t  = – ---1 – e

6
-0.2
0
0.05
0.1
2:1
p  w2
0.15
1
= 0  1 = 0 = a
1
1
1
0.2
14
Layer 2
15
Layer 2 Operation
Shunting Model
2
d n (t )
2
 --------------- = – n (t )
dt
On-Center
Feedback
+ 2
2
+
2
2
Adaptive
Instars
2
? +  b – n (t )   W f (n (t )) + W
1:2 1
a 
Excitatory
Input
Off-Surround
Feedback
2
- 2
-
2
2
2
–  n (t) + b   W  f (n (t))
Inhibitory
Input
16
Layer 2 Example
 = 0.1
b = 1
b = 1
+ 2
- 2
1
1
 10 n 2 ,
f (n ) = 
 0,
2
W1:2 =
1:2 T
 1w 
1:2 T
 2w 
n0
n0
= 0.5 0.5
1 0
(Faster than linear,
winner-take-all)
2
dn 1(t )
 2 2
2
2
1:2 T 1 
2
2 2
 0.1  -------------- = – n 1(t ) +  1 – n 1 (t)  f (n 1 (t)) + 1 w  a  – n 1 (t) + 1  f (n 2 (t))
dt


2
dn 2 (t)
 2 2
1:2 T 1 
2
2
2
2 2
 0.1 -------------- = – n 2 (t) +  1 – n 2(t )  f (n 2(t )) +  2 w  a  –  n 2(t ) + 1  f (n 1(t )) .
dt


17
Example Response
1
1:2 T 1
 2w  a
2
n2 t
0.5
 1w
a = 1
1
1:2 T 1
 a
2
a = 0
0
1
0
-0.5
2
n1 t
-1
0
0.05
0.1
t
0.15
0.2
18
Layer 2 Summary

 1,
2
ai = 
 0,

if iw
1:2 T
1
1:2 T 1
 a = max  jw
 a 
otherwise
19
Orienting Subsystem
Purpose: Determine if there is a sufficient match between the
L2-L1 expectation (a1) and the input pattern (p).
20
Orienting Subsystem
Operation
0
dn (t )
0
+ 0
0
+ 0
0
- 0 0 1
 -------------- = – n (t ) +  b – n (t )  W p – n (t) + b  W a 
dt
Excitatory Input
+
S
0
W p = aaap = a
1

pj = a p
2
j=1
Inhibitory Input
-
S
1
W a = bbba = b
0 1
1
1
a j t
= b a
1 2
j=1
When the excitatory input is larger than the inhibitory input,
the Orienting Subsystem will be driven on.
21
Steady State Operation
1 2

0 = – n +  b – n  a p  – n + b  b a 


0
+ 0
= – 1 + a p
2
0
2
+b a
1 2
0
- 0
0
+ 0
2
2
- 0
1 2
- 0
n + b  a p  – b  b a
+ 0
1 2

b a p  – b b a 
0
n = --------------------------------------------------------------2
1 2
1 + a p + b a 
Let
0
n 0
+ 0
- 0
b = b = 1
1 2
if
a
a
-------------  --- = r
2
b
p
Vigilance
RESET
1
2:1
Since a = p  w j , a reset will occur when there is enough of a
2:1
mismatch between p and w j .
22
Orienting Subsystem Example
 = 0.1, a = 3, b = 4 (r = 0.75)
p =
1
1
1
a =
1
0
0
dn (t )
0
0
0
1
1
 0.1  -------------- = – n (t ) +  1 – n (t)  3  p 1 + p 2   –  n (t ) + 1  4  a 1 + a2 
dt
0
dn (t)
0
-------------- = – 110n (t) + 20
dt
0.2
0
n  t
0.1
0
-0.1
-0.2
0
0.05
0.1
t
0.15
0.2
23
Orienting Subsystem
Summary

 1,
0
a = 
 0,

if  a
1 2
 p
2
 r
otherwise
24