Neural Networks
Chapter 4
Joost N. Kok
Universiteit Leiden
Hopfield Networks
• Optimization Problems (like Traveling
Salesman) can be encoded into Hopfield
Networks
• Fitness corresponds to energy of network
• Good solutions are stable points of the
network
Hopfield Networks
•
Three Problems
1. Weighted Matching
2. Traveling Salesman
3. Graph Bipartitioning
Hopfield Networks
• Weighted matching
Problem:
– Let be given N points
with distances dij
– Connect points
together in pairs such
that the total sum of
distances is as small as
possible
Hopfield Networks
• Variables: nij (i<j) with values 0/1
• Constraint: Sj nij = 1 for all i
• Optimize: Si<j dij nij
Hopfield Networks
• Penalty Term approach: put constraints in
optimization criterion
• Weights and thresholds of Hopfield
Network can be derived from


H n    dij nij   1   nij 
2 i 
i j
j


2
Hopfield Networks
• Travelling Salesman Problem (TSP):
Given N cities with distances dij .
What is the shortest tour?
Hopfield Networks
• Construct a Hopfield network with N2 nodes
• Semantics: nia = 1 iff town i on position a in
tour
1
L   d ij nia n j ,a 1  n j ,a 1 
2 i , j ,a
Hopfield Networks
• Constraints:
n
ia
a
 1, i
n
ia
 1, a
i
1
H   d ij nia n j ,a 1  n j ,a 1 
2 i , j ,a
2
2

 


 
  1   nia    1   nia 
2  a 
i
i 
a

 
Hopfield Networks
• 0/1 Nodes
• Nodes within each row connected with
weight –
• Nodes within each column connected with
weight –
• Each node is connected to nodes in columns
left and right with weight –dij
• (Often) continuous activation
Hopfield Networks
postion
city
1
2
3
4
A
0
1
0
0
B
1
0
0
0
C
0
0
0
1
D
0
0
1
0
Hopfield Networks
• Graph bipartitioning: divide nodes in two
sets of equal size in such a way as to
minimize the number of edges going
between the sets
• +1/-1 Nodes
• 0/1 Connection matrix Cij
Hopfield Networks
L   Cij Si S j
(ij )
S
i
i
0
Hopfield Networks


H   Cij Si S j     Si 
( ij )
 i 
H  N   wij S i S j
(ij )
wij  Cij  2
2