EXAMPLE :
Principle of Optimality
and Dynamic
Programming
1.4
1.3
1.4
Dynamic
Programming
Dynamic Programming
• Search for best path
• Dynamic programming principle
– If B is on the shortest path from A to C, then the
path from A to B that lies on the path from A to C
is the shortest path from A to B
…
A
B
…
C
Dynamic Programming
• If DP principle holds, then efficient search can be
implemented
• When at node B in search you can prune all paths to
B that are not the shortest
• You should not expand these paths
…
A
B
…
C
Applications to Handwriting Recognition
• Address Interpretation – USPS
• Word Recognition
– ZIP + Street No. Hypotheses 
– “Small” Lexicons for Word Rec
1. Problem Statement and History
2. Picture of Parsed Address Block
3. Word Recognition Problem
4. Results for NIST OCR Tests with refs
Character Ambiguity in Handwriting Recognition
3 modules for
character class
membership
assignment
Handwritten Word Recognition S ystem
word image
segmentation
Baseline module:
MLP
Mixed Linear Programming
primitiv es
lexicon
dynamic
programming
matching
primitiv es
and
unions of primitiv es
character class
membership
assignment
New module:
SOFM and MLP
Self-Organizing Feature Maps
New module:
SOFM and FI
fuzzy integral
aggregation
match scores
between word
image and lexicon
strings
Fuzzy Inferfence
DP Approach to handwriting
recognition
Segmentation-Based Handwritten Word Recognition
Search Problem – How do we put the pieces together?
DP Approach
• Let I be a word image
• L = {L1, L2, ..., LT} be a lexicon of strings
representing all possible identities of I.
• Word recognition requires finding the string in
L that best matches I.
• Search – DP, Heuristic
DP Approach
Let Lm = c1c2...cn. Let the primitives of I be S1, S2, ..., Sp , (ordered from left to right). Denote
the set of all legal unions of primitives by
k
UI = { Sjk :
Sjk =
S
h
, j  k, and
h j
k
 S is legal }.
h
h j
A segmentation of length n of I is a sequence of elements of UI of the form
S=
Sk0+1,k1,
Sk1+1,k2, ..., Skn-1+1, kn
where k0 = 0 and k n = p.
Let Sn denote the set of all segmentations of I of length n. For each character class c, let x c(s)
denote the neural network output corresponding to class c given s UI as input. A match score
between Lm and a segmentation S  Sn can be computed as
1 n
segmatch ( L m , S) =  xci ( S ki 1 1, ki ) .
n i 1
DP Approach
For i = 1, ..., w
For j = i, ..., p
1
IF i = 1, (that is we are matching against the first character) THEN
2
IF Legal(S1j) THEN
dp(1,j) = ( S1j, C1)
3
4
ELSE
dp(1,j) = -?
5
6
ENDIF
7
ELSEIF Legal(Sij) THEN
8
dp ( i , j )  max
k
9
dp ( i  1, k )   ( S k 1 , j , C i ) | i  k 
j and Legal
Sk j  is TRUE 
ELSE
dp(i,j) = -?
10
11
ENDIF
12
ENDIF
13
IF dp(i,j) > -? THEN
14
j 2  arg max
k
15
dp (i  1, k )   (S k 1, j , C i ) | i  k 
path(i,j) = (i - 1, j2)
16
ENDIF
ENDFOR
ENDFOR
j and Legal
Skj  is TRUE 
DP Approach
Table 1. Dynamic Programming Array for Match of Richmond to "Richmond"
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
"R"
11
38
53
45
44
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
"i"
0
23
27
71
80
52
53
54
52
51
55
0
0
0
0
0
0
0
0
0
0
0
0
0
0
"c"
0
0
0
38
90
132 108 108
86
86
87
63
63
63
63
62
65
65
0
0
0
0
0
0
0
"h"
0
0
0
0
42
95
144 147 157 193 189 166 162 141 145 121 100
95
85
77
76
0
0
0
0
"m"
0
0
0
0
0
50
104 154 156 167 202 208 219 227 261 263 272 258 178 177 216 231
0
0
0
"o"
0
0
0
0
0
0
74
114 169 163 179 206 241 261 236 236 279 309 312 290 281 280 281
0
0
"n"
0
0
0
0
0
0
0
83
122 178 181 190 223 261 269 269 275 285 322 337 352 370 356 321
0
"d"
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
458
Table 2. Dynamic Programming Path Array for Match of Richmond to "Richmond"
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
"R"
0
0
0
0
0
0
0
1
8
8
6
9
11
11
12
12
13
16
15
15
19
20
20
19
20
"i"
0
1
1
3
3
5
5
5
5
5
5
5
7
7
14
10
16
16
16
15
15
19
16
21
19
"c"
0
0
2
3
3
5
5
5
5
5
5
7
7
7
8
11
11
11
12
14
14
16
21
19
21
"h"
0
0
0
3
4
5
6
6
6
6
6
6
6
8
8
10
11
11
15
15
15
16
16
17
21
"m"
0
0
0
0
4
5
6
7
8
9
10
10
10
11
11
10
10
11
12
13
15
15
16
21
19
"o"
0
0
0
0
0
5
6
7
8
9
10
11
12
13
14
14
16
16
16
15
17
17
17
19
21
"n"
0
0
0
0
0
0
6
7
8
9
9
9
11
13
14
14
13
17
18
18
17
18
18
19
23
"d"
0
0
0
0
0
0
0
7
8
9
10
10
11
12
13
13
16
17
17
17
20
20
20
22
22