Multiple Agents
for
Pattern
Recognition
Louis Vuurpijl
http://hwr.nici.kun.nl/~vuurpijl
Contents
• The problem: handwriting recognition
• PR1: The traditional solution
– (some) solutions at the NICI
• PR2: Multiple classifiers
– (some) solutions at the NICI
• PR3: Could MAPR be a solution?
– our current achievements
Sources of variation
(Schomaker, Plamondon et al ’00)
Affine
transforms
Neurobiomechanichal
Variations
Style
Variations
Order
Variability
The more writers, the more….
Is this a problem?
• Shrihari ICDAR’01: “No problem”
Handwriting is individual, so can be
used in court (as a fingerprint)!
• IWFHRxx, ICDAR, IJDAR... “Yes!”
We have 99+ digit recognition, 98+
character recognition and 90+ for
isolated words........
Pattern Recognition (I)
Raw data
X(t),Y(t),P(t)
Class labels
Pattern Recognition (I)
Preprocessing
Raw data
Segmentation
X(t),Y(t),P(t)
Feature Extraction
Class labels
Classification
Solutions at the NICI
• More than 20 years of experience in
– Handwriting production (Thomassen, van
Galen, Meulenbroek, Maarse, Schomaker, et al)
– Handwriting recognition (Schomaker, Teulings,
Vuurpijl, et al)
• Keywords:
– Use knowledge about human handwriting
– Specialization and......
– Fusion
The basis of handwriting
X(t),Y(t),P(t)
Va(t)
UNIPEN data
Zooming in on writing styles
UNIPEN styles: print, cursive, mixed
The lean recognition machine
Clustering on strokefeatures (IWFHR’96)
Specialization boosts recognition performance,
while reducing computational and memory requirements
Finding structure in
diversity (ICDAR’97)
Hierarchy in character
shapes
Allograph prototypes
dScript: a MAPR system (’00)
dScript: 9 classifiers
•
•
•
•
•
Neural networks (MLP, Kohonen)
Nearest neighbour & clustering
Structural/geometrical
Support vector machines
Hidden markov models
Fusion through classifier combination and
Multiple agents (IWFHR’98,’00,’02)
Pattern Recognition (II)
Raw data
Classifier 0
Classifier 0
Classifier 0
Classifier 0
Classifier 0
Classifier 0
Classifier 0
Classifier 0
Classifier 0
Classifier(i)
Classifier
Combination
Class labels
Classifier combination (van Erp’00,’02)
Changing contexts....
- static architecture
- what if “Go 551”
was intended?
Dynamic PR
Through extra heuristic information
Dynamic PR
Through extra features (add PENUP)
Features determine
how you look at data
Dynamic PR
Knowing when to use which feature
Dynamic PR
Knowing when to use which feature
Dynamic PR
Knowing when to use which feature/algorithm
•Through a knowledge base of PR
•Through a library of PR modules
•Through negotiation protocols
Knowing how to use which PR module
Pattern Recognition (III)
Raw data
MAPR
Class labels
What is an agent?
(Wooldridge and Jennings, 1995)
A software system with:
• Goals: ``What do you want?'' or ``What can you do?''
– I can solve 0-6 conflicts
• Beliefs and reasoning: ``How do you realize this goal?''
– I solve this 0-6 conflict using modules PR1 and PR2
– and features #84 and #96, extracted by FE(i) and FE(j)
• Assertions with confidence:
– Based on my experience and these features I belief this
input is a ``6'' with confidence 0.9.
– I have been correct in 90% of the cases in the past.
Agent framework
MAPR: Our goal
• A distributed intelligent agent framework,
• with PR modules, symbolic equivalents
and PR language.
• Driven by problem constraints
• and with learning capabilities.
Our current MAPR system
• Distributed processing over internet using
sockets.
• Interfaces to KQML and Jatlite.
• Agents know about the environment.
• Agents know about the available PR
modules and data.
• Agents interact with other agents.
• Agents to detect problems and conflicts
A proof of concept
Trained recognition system hclus
– 15557 digits from UNIPEN
– 7778 train, 7779 test (95.9%)
1-7
7-2
5-3
1-2
1-2
4-1
4-6
30
27
25
23
23
22
14
5-8
1-8
0-9
8-0
8-0
4-8
14
12
12
11
11
10
1-7, 7-2, 4-6
conflicts
solved
 97%
But......
• This is all hard-wired
if confidence top[0] too low
then solve(top[0],top[1],...)
solve(1,7,2) =
best(1-7,1-2,7-2)
Current research on MAPR
• Knowledge base
• PR language & implementation
• PR negotiation mechanisms
Conclusions
• Online HWR is still unsolved
• MCS can improve recognition rates, but.....
– Hard-wired PR modules
– Examples where dynamic PR is needed
• MAPR is a new paradigm that exploits knowledge
about when to use which features or algorithms
But how to implement shared access to knowledge?
And how to perform agent-like negotiations ?