A Pattern Matching Approach to Map
Cognitive Domain Ontologies to the
IBM TrueNorth Processor
CCAA 2017
Nayim Rahman1, Tanvir Atahary1, Tarek Taha1,
and Scott A. Douglass2
1Electrical
2711
and Computer Engineering Department,
University of Dayton
HPW/RHAC, Cognitive Models and Agents Branch,
Air Force Research Laboratory
1/34
Overview
• Computerized agent-based decision aids
• Represent knowledge in databases
• Mine this knowledge to find solutions to specific
problems
• Use computer clusters to accelerate this mining
2/34
Cognitively Enhanced Complex
Event Processing
•
−
−
Cognitively Enhanced Complex Event Processing
(CECEP) Architecture:
Models and agents are specified in research modeling
language (RML)
RML has been developed using the Generic Modeling
Environment (GME)
•
Consists of the following central net-centric
components:
−
soaDM: an associative memory application that allows
RML models and agents to store and retrieve declarative
knowledge.
soaCDO: a knowledge representation and mining
application that allows RML models and agents to store
and exploit domain knowledge.
Esper: a complex event processing framework that allows
RML models and agents to base actions on context
assessment and procedural knowledge. Procedural
knowledge is represented in RML behavior models and
processed using pattern matching and event abstraction
capabilities provided by Esper.
−
−
IO “Adapters”
EventOutputStreams
3/34
Cognitively Enhanced Complex
Event Processing
•
Cognitively Enhanced Complex Event Processing
(CECEP) Architecture:
−
Declarative Knowledge: specified as events and
relations
−
−
Procedural Knowledge: specified as behavior models
Domain Knowledge: specified in CDOs and
processed by constraint-satisfaction framework
Adapters : includes a number of IO “Adapters” or
event input and output streams, allow models and
agents specified in RML to be integrated into
software-based instructional systems
−
IO “Adapters”
EventOutputStreams
4/34
Cognitive Domain Ontology
Domain knowledge represented
as a data structure called
“Cognitive Domain Ontology”
(CDO).
IO “Adapters”
EventOutputStreams
CDO
5/34
Example CDO
Sports
Ball
Large
Medium
SizeConstraints:
•Small
•NotMedium
•NotLarge
SizeConstraints:
•Medium
•NotSmall
•NotLarge
Small
SizeConstraints:
•Large
•NotSmall
•NotMedium
6/34
CDOs Model Domains and Have Constraints
ball
XOR
baseball
football
sport
basketball
XOR
small
medium
size
XOR
large
white
orange
color
brown
Constraints:
If(sport==baseball)then(size=small)and(color=white)
If(sport==football)then(size=medium)and(color=brown)
If(sport==basketball)then(size=large)and(color=orange)
7/34
Track CDO
8/34
Track CDO Constraints
Name
C1
C2
C3
C4
C5
C6
Specification
iff ews_choices is arinc_564 then model_choices is b_747
iff ews_choices is apq_120 then model_choices is f_4
iff ews_choices is apg_63 then model_choices is f_15
iff ews_choices is foxfire then model_choices is mig_25
if model_choices is f_4
then threat_choices is assumed_hostile hostile assumed_friendly or friendly
type_choices is strike
if model_choices is f_15
then threat_choices is assumed_friendly or friendly
threat_choices is not (assumed_hostile or hostile)
type_choices is strike
C7
if model_choices is b_747
then threat_choices is friendly or assumed_friendly
threat_choices is not (assumed_hostile or hostile)
type_choices is commercial
C8
if model_choices is mig_25
then threat_choices is assumed_hostile or hostile
threat_choices is not (assumed_friendly or friendly)
type_choices is strike
C9
if speed is between 350 and 550,
altitude is between 25000 and 36000
then model_choices is b_747
threat_choices is friendly
type_choices is commercial
9/34
CDO as a CSP Tree
CDOrepresentation
CSPtreerepresentation
10/34
HPC Approach 1: Exhaustive Search
• Exhaustivedepthfirstsearch
• Highlyparallel
Sport=baseball
Size=small
Color=white
0
1
Constraints:
If(sport==baseball)then(size=small)and(color=white)
If(sport==football)then(size=medium)and(color=brown)
If(sport==basketball)then(size=large)and(color=orange)
14
Sport=football
Size=medium
Color=brown
Sport=baseball
Size=small
Color=orange
11/34
Results
1E+07
Speeduprelative
to1Xeoncore:
RuntimeofsingleandmulticoreCPUsandasingle
GPU
1XeonCore
Executiontimeinsec
1E+06
1E+05
8Xeoncores
IntelXeon
(1core)
100x
1NvidiaGPU
4
1E+04
1E+03
1E+02
IntelXeon
(4cores)
1E+01
1E+00
1E+08
1E+09
1E+10
1E+11
1E+12
1E+13
100
Numberofleafnodes
NVIDIAGPU
(448cores)
12/34
CDO Scaling
DatabasedonsyntheticCDOswithn brancheshavingn entitieseach.
1E+13
9E+12
Numberofleafnodes
1E+12
3E+11
1E+11
1E+10
1E+10
1E+09
4E+08
1E+08
2E+07
1E+07
1E+06
8E+05
1E+05
5E+04
1E+04
6
7
8
9
10
11
12
13
Treelevel
ExampletreefromaCDOwith3levels:
13/34
CDO Run Times: Exhaustive Search
TreeLevel
CPUruntime
1GPUruntime
10
42 m
20 s
11
22h
10m22s
12
30days*
5.75h
13
3years*
9days*
14
75years*
1years*
15
2250years*
42years*
16
675centuries*
1898years*
*estimatedruntimes
14/34
HPC Approach 2: Informed Search
Beforepruning
Afterpruning
Nosolutionforthatparticularconstraint
--Pruned18outof27childnode
Possiblesolutionspace
--Only9childnode
15/34
HPC Approach 2: Informed Search
Beforepruning
Afterpruning
Nosolutionforthatparticularconstraint
--Pruned24outof27childnode
Possiblesolutionspace
--Only3childnode
16/34
Path based Forward Checking Algorithm
Thisalgorithmhasthreesteps:
• Pre-processingstep: simplifiesconstraintswhichinturn
removesanyinterdependenciesbe-tweenbranches.
• Pathgenerationstep: prunesthesearchspacebasedon
constraints andmakeapathfromroottoleafnode.
• Solutiongenerationstep:listsallpossiblesolutionsina
compactmannerbasedonthepathgeneratedinstep2.
17/34
Path Generation
1 Root
Path:
SP
Level 0
3
7
C
5
4 G
V1=0
V2=0
6:
Exp
M
Level 4
Level 5
12 E
D
CP
SP
Level 2
Level 3
SP
SP
Level 1
Root-A-C-G-H-Exp-D-J-P-U-B-E-V3=2.45-F-varied
B 11
2 A
CP
CP
N
F
CP
V3=2.45
J 8
H
13
14:
Varied
K
L
W
CP
P 9
XOR
U 10
Q
R
S
Solution based on the path:
Root is SubPart event and it's corresponding choice is below
A is SubPart event and it's corresponding choice is below
C is SubPart event and it's corresponding choice is below
G is 'Variable' event and values associated with it is below
V1 is 0 V2 is 0
H is a ChoicePoint event and it is expanded
H can be M or H can be N
D is a ChoicePoint event and it is J
J is a ChoicePoint event and it is P
P is a ChoicePoint event and it is U
Number of solution found from branch 1 = 2
B is SubPart event and it's corresponding choice is below
E is 'Variable' event and values associated with it is below
V3 is 2.45
F is a ChoicePoint event and it is varied
F can be K or F can be W
Number of solution found from branch 2 = 2
Total possible number of solution= 2×2=4
18/34
Performance (1 Xeon core)
ScarCDO
1E+7
Runtime(ms)
1E+6
~1Milliontimesspeedup.
Requireshighpower
computingresources.
1E+6
Sherlock-allresults(ms)
Sherlock-1stresult(ms)
Forwardchecking-allresults(ms)
Speedup(generatingallresults)
1E+5
1E+5
1E+4
1E+4
1E+3
Speedup
1E+8
1E+3
1E+2
1E+2
1E+1
1E+1
1E+2
1E+3 1E+4 1E+5 1E+6
Numberofsolutions
1E+7
1E+1
1E+8
19/34
IBM TrueNorth
• 4096 cores
• 256 neurons / core
• 100mW
20/34
Spiking Neurons
• TrueNorth System is spikes-in, spikes-out
• Internal processing is based on spiking
neurons
21/34
Cores and Connectivity
22/34
Mapping CDOs to Pattern Form
Constraints:
If(sport==baseball)then(size=small)and(color=white)
If(sport==football)then(size=medium)and(color=brown)
If(sport==basketball)then(size=large)and(color=orange)
Inputs
Output
Sport:baseball
Size:small
=>Color:white
Sport:football
Size:medium
=>Color:brown
Sport:baseketball
Size:large
=>Color:orange
23/34
Mapping CDOs to Pattern Form
Constraints:
If(sport==baseball)then(size=small)and(color=white)
If(sport==football)then(size=medium)and(color=brown)
If(sport==basketball)then(size=large)and(color=orange)
Inputs
Output
Sport:baseball
Size:small
=>Color:white
Inputs
Output
Sport:football
Size:medium
=>Color:brown
Sport:baseketball
Size:large
=>Color:orange
Sport
Baseball
Football
Basketball
Size
Small
Medium
Large
Color
White
Brown
Orange
24/34
Mapping CDOs to Pattern Form
Inputs
Output
Sport
Baseball
Football
Basketball
Size
Small
Medium
Large
Color
White
Brown
Orange
Output
Inputs
Sport
Size
Small
Medium
Color
Baseball
Football
Basketball
Large
White
Brown
Orange
1
0
0
1
0
0
1
0
0
0
1
0
0
1
0
0
1
0
0
0
1
0
0
1
0
0
1
25/34
Lookup Table with Neurons
•
Neurons spike only for a specific pattern
•
Neuron set to fire for input 1100:
a=1
b=1
c=0
d=0
a=1
b=1
c=0
d=0
+1
+1
0
0
0
0
-1
-1
a=1
b=0
c=0
d=0
a=1
b=0
c=0
d=0
+1
+1
0
0
0
0
-1
-1
a=1
b=1
c=1
d=0
a=1
b=1
c=1
d=0
+1
+1
0
0
0
0
-1
-1
a=0
b=1
c=1
d=1
a=0
b=1
c=1
d=1
+1
+1
0
0
0
0
-1
-1
+2/2
+1/2
+1/2
-1/2
SPIKE
Nospike
Nospike
Nospike
26/34
Multiple Patterns in a Crossbar
•
Each neuron in a crossbar can detect a different input pattern
a=1
b=1
c=0
d=0
a=1
b=1
c=0
d=0
+1
0
+1
+1
+1
+1
0
0
0
0
0
0
-1
+1
0
0
0
0
-1
-1
-1
-1
-1
0
+2/2
SPIKE
0/1
+2/3
Nospike Nospike
27/34
CDOs in Neural Form
Baseball=0
Football=0
Basketball=1
Small=0
Sport
Baseball
Football
Basketball
Size
Small
Medium
Large
Color
White
Brown
Orange
Medium=0
Large=1
Baseball=0
Football=0
Basketball=1
Small=0
Medium=0
Large=1
+1
0
0
0
+1
0
0
0
+1
+1
0
0
+1
0
0
0
0
+1
0
-1
-1
-1
-1
0
-1
-1
0
0
-1
-1
-1
0
-1
-1
-1
0
-2/2
White
-2/2
+2/2
Brown
Orange
28/34
Larger CDO: Vehicles
root
14constrains
12classes
SP
country
mileage
CP
type
year
hp
price
model
CP
Class
1
Choicepoints
Country
Mileage
Type
Year
HP
Price
Model
Entities
'japan', 'usa', 'south korea', 'germany'
'25_above/35_above', '25_below/35_above',
'25_below/35_below'
'family sedan', 'mid-size suv', 'luxury coup'
2016
'160-180', '181-200', '201-220', '221-240',
'240-above'
$21000-$23000', '$29000-$32000', '$38000$43000'
Next Slide output Classes.
2
3
4
5
6
7
8
9
10
11
12
Content
honda_accord, nissan_altima,
mazda_mazda6
toyota_camry, subaru_legacy
chevloret_malibu
ford_fusion
chrysler_200
kia_optima, hyundai_sonata
honda_pilot,
toyota_highlander
dodge_durango, ford_explorer
audi_a5
bmw_4series
lexus_rc
inifinit_q6
29/34
Vehicle CDO into Lookup Table
• 38 input axons. (19 input elements.)
• 12 neurons. (12 classes).
+1
+1
0
0
0
0
0
0
+1
+1
+1
+1
+1
+1
+1
-1
-1
0
0
-1
-1
-1
0
-1
Sedan
SUV
Luxury
2016
Japan
201-220HP
181-200HP
160-180HP
Class1
Class2
Class12
30/34
Memristor TCAM
•
Four memristors are required to
store a bit
– There will always be some current
through the memristors set to ROFF
– Biases are used to negate this effect
0
ROFF
RON
1
-1
ROFF
x2
x2
RON
RON
ROFF
1
-1
ROFF
xN
xN
ROFF
RON
RON
ROFF
1
-1
1
-1
RON
RON
ROFF
- +
(a)
vcol
vout
t=
0
×
×
x1
x1
RON
ROFF
1
-1
ROFF
x2
x2
ROFF
RON
ROFF
1
-1
ROFF
xN
xN
ROFF
ROFF
ROFF
ROFF
1
-1
1
-1
t2
tN
ROFF
RON
ROFF
- +
(b)
t1
...
RON represents a strong connection
ROFF represents no connection
t=
x1
x1
...
•
•
Input
Complement
Positive Bias
Negative Bias
...
–
–
–
–
0
1
bias (Vb)
vcol
vout
31/34
CECEP using Memristor TCAMs
• Memristor based TCAM: High area efficiency, and
hence high amount of knowledge per chip
TCAMSystem
Configuration
x1
x1
x1
1
-1
xN
xN
xN
1
-1
1
-1
bias
- +
Vout,1
- +
Vout,2
- +
Vout,j
Kim
CMOS
Huang
CMOS
Zheng
RRAM
Xu
MRAM
512×144 256×144 256×144 256×144
Memristor
256×144
Technology(nm)
130
65
180
180
45
SearchTime(ns)
4.8
.38
2.3
8
1
EnergyMetric
(fJ/bit/search)
0.59
0.165
3
7.4
2.15
Area(mm2)
2.55
0.434
0.167
1.548
0.0012
NormalizedArea
Factor
255
173
8.7
80.6
1
- +
Vout,M
32/34
CDOs as CNNs on TrueNorth
• Each network was trained for 8000 iterations
• Testing set = Training set
Layers
14
13
12
9
7
6
5
3
2
Learning Rate Average
Accuracy
20, 2, 0.2
99.98%
20, 2, 0.2
99.97%
20, 2, 0.2
99.98%
20, 2, 0.2
99.99%
10, 1, 0.1
99.99%
10, 1, 0.1
99.99%
10, 1, 0.1
99.97%
10, 1, 0.1
99.99%
10, 1, 0.1
98.37%
Cores
218
127
106
82
74
62
56
36
7
Power
(mW)
52.7
51.6
51.3
51.0
50.9
50.8
50.7
50.4
50.0
33/34
Conclusion
• CECEP cognitive agent can be mapped to
pattern matching form
• Can be implemented on IBM TrueNorth
processor
• Can be mapped to memristor crossbars
• Future work: Look at how to solve (as opposed to
look up solutions) using neurons
34/34