A Biopsychically Inspired
Cognitive System for
Intelligent Agents in
Aerospace Applications
Xudan Xu, James Graham and Basawaraj
Advisor : Dr. J. Jim Zhu, Dr. J. A. Starzyk
Ohio University
Jun /19/2012
Introduction(1)

Challenges in Extraterrestrial
Exploration:




Uncertainty/Unknown Environment
Complexity
Time Urgency/Response Urgency
Rapidly Changing/Dynamic Situation
Introduction(2)

Current state-of-the-art Machine Intelligence

Remotely commanded / human supervised
telecommunication






Limited by
Bandwidth;
Latency;
Distortion;
Noise;
Power/Energy
Pre-programmed :





Static knowledge
Supervised machine learning
Inability to set goal autonomously
Inability to imagine, innovate and improvise
Lack of distinct cognitive traits, personalities , autonomous cooperation
Introduction(3)

NSF CFP 11556. Dec 5, 2011 [Information and Intelligent Systems
(IIS): Core Programs]:
“… with the capability for situation awareness, allows for greater degree of
uncertainty in terms of reasoning systems and produces greater robustness
and adaptability in planning algorithms in dealing with unexpected
interruptions and rapidly changing objective.”

AFOSR , BAA-AFOSR-2011-01. Dec 5, 2011.
The system should “not only ascertain their descriptive validity and neural
plausibility or feasibility, but also deepen understanding of mathematical
characterizations of principles of adaptive intelligence”.
Our answer
A Biopsychically Inspired Cognitive System
for Intelligent Agents
Outline of Main Results
The Central Nervous System
The Proposed Cognitive System Architecture
The Key Algorithms for the cognitive system
Simulation
Conclusion
Central Nervous System (1)
Name
Specific
Simple
Spinal cord
Mid brain
Brain
Medulla
stem
Hind
brain
Fore
brain
Abstract
Complex
Function
Transmission Bus
Vision, Sound
Balance; equilibrium
……
……
Cerebellum
……
Motion coordination
……
Cerebral
cortex
Executive center
Thalamus
Information center
Limbic
system
Memory management
Hippocampus
……
……
Central Nervous System (2)
Cerebral Cortex
Lobe
Function
Frontal lobe
Decision Making; Planning,
abstraction;
Motion planning and
monitoring center;
Parietal lobe
Sensory Info integration;
Interpretation,;
end effectors manipulation
Occipital lobe
Visual processing
Temporal lobe Auditory processing
Central Nervous System (3)

CNS is





Hierarchical;
Survival driven; penalty-reward motivated learning
A network of massive concurrent computing
devices, instead of a single processor
advantage of a network of billions of simple and
slow computing units
“survival of the fittest” -- naturally optimized
Proposed Cognitive System
Architecture (1)
Biopsychically Inspired



Biology  Hardware
Psychology  Software/Algorithms
Salient Features



Hierarchical;
Modeler; Every unit is a computing device; It is a
network of concurrent computing devices;
Motivated learning;
Proposed Cognitive System
Architecture (2)
“Forebrain”
Central Execution
Perception Focus
Loop4
Language
“Temporal
Lobe”
Decision Making and Planning
“Frontal Lobe”
Learning
reinforce query
Working
Memory
Limbic System
Motivation & Goal
(Pain/ Reward Emotion)
Memory Manager
“Hippocampus”
“Sensors and Processors”
Long-term Memory
SM
EM
IMU
Smell and Taste
Somatic Sensors
Vision
Hearing
…
…
Inhibition
“Thalamus”
“Occipital Lobe”
“Temporal Lobe”
Procedural
Memory
Health Management
“Autonomic Nervous System”
Motion Control
(Control)
“Cerebellum”
Loop2
Reflex Action
“Spinal Cord”
Loop1
Tele communication
Motion Monitoring and Coordination
(Guidance)
“Motor Cortex”
Loop3
Sensory
Processing
(Navigation)
“Spinal Cord”
Internal
Sensor
+
Agent
Agent Motor System
Agent Bodily
Functional System
Agent
Environment
External
Memory
E M: Episodic Memory
S M:Semantic Memory
Dynamic Environment
Adversarial
Agents
Friendly Agents
Proposed Cognitive System
Architecture (3)
“Sensors and Processors”
“Spinal Cord”
Proposed Cognitive System
Architecture (4)
“Forebrain”
Key Algorithms (1)

Units




Motivation &Learning(M&L) :
Long-term Memory (LTM):
Decision Making &Planning (DM&P) :
Salient Features





Self-motivated rather than directed goal setting and
pursuit;
Evolved rather than programmed cognition;
Capable of unsupervised learning;
Able to improvise, innovate and imagine;
Possess distinct personality and unique abilities
Key Algorithms (2)
DM&P
M&L
LTM
Key Algorithms (3)
DM&P Unit
Task_Info
Decision
Making
Action_Info
Task
_d
ue
Planning
ata
Plan
Action
Monitoring
ery
q
Qu& goal)
n_
g
a
l
nin ht
Long-term Memory Unit P
lann. spotlig
P
t
Semantic
Procedural
(At
Memory
Memory
Goal
Reference
lts
)
re
su
Spot
At
ten
tio
nD
ata
(C
Q
om
pa u e
rin ry
g
Attention
Sem_Rein
Proc_Rein
Action
Motivation & Learning Unit
Attention
Switch
Learning
Goal
Reference
Plan Executed
Pain State
(List of pain levels)
List of objects
Environment
Model
Motivation
Environment State
(Condition Resource)
Primitive Pain
Model
Key Algorithms(4) – M&L

Functionality :
 Perceive the environment
 Evaluate the primitive pain and the abstract
pain / motivation
 Derive motivation
Key Algorithms (5) – M&L
Pain Model:
B  [ Bi ]   log 2
RK
i
RPi
Pabstract i  wBP i Bi
rp n

;
since reset
 Psat
Pprim  f  n    rp n   p
0;
when reset by agent action

Key Algorithms (6) – M&L
Learning

wBP : how important the resource is to the action
1
B  BP( B  wBP )
wBP i
 wBP i1  B
if A


  wBP i1 (1  BP) if B


 wBP i1 (1  BN ) if C
A : if pain is reduced by current action;
B : if pain is not reduced by current action;
C : if the action is not taken;
The weight between the pain and the
untaken action reduced gradually.
0.9
0.8
0.7
0.6
wBP

0.5
0.4
0.3
0.2
0.1
0
0
1
2
3
4
5
frequency
6
7
8
BP  0.08
BN  0.0001
9
10
Key Algorithms (7) – M&L
For more details :
 J. A. Starzyk, J. T. Graham, P. Raif, A-H. Tan. "Motivated
Learning for the Development of Autonomous Systems" .
Cognitive Systems Research. 14 (2012), pp. 10-25.
 J. A. Starzyk, "Motivation in Embodied Intelligence," in Frontiers
in Robotics, Automation and Control, I-Tech Education and
Publishing, Oct. 2008, pp. 83-110.
Key Algorithms (8) – LTM

Functionality : (Semantic, procedural)



Storing knowledge acquired by observations and
experiences
Formulating plans and actions with the past
observations and experiences;
Optimizing the memory for future use with
resulting changes in the environment.
Key Algorithms (9) – LTM
Object


pain level
the strongest link.
A connection between every node in
different group
Pain
Action
Key Algorithms (10) – LTM


Visual saccade : the spotlight moves from the object
with most salient features to the one with least salient
features.
Comes up with the strongest link



Wpo_22 >Wpo_21> Wpo_23
O2O1O3
O2 is the winner
Object
Pain
P1
O1
Wpo_21
O2
Wpo_22
P2
Wpo_23
O3
P3
Key Algorithms(11) – LTM

The weight from O2 to P2 is



Wop_22= w
OA_2i
* wAP_i2
wOA_21 wAP_12 > wOA_22 wAP_22
Winner takes all: A1
Object
Action
Pain
P1
O1
A1
P2
O2
A2
O3
P3
Key Algorithms(12) – LTM
Learning :
Aj
Pi
Other wAP & wPO
Inhibition
wAjPi
wPiOi
wPA   A min   A  wPA , wPA 
wPO  O min  O  wPO , wPO 
Ai  wPA .wPAi /  wPAi
Oi  wPO .wPOi /  wPOi
If the current spotlight is not taken into consideration, the next new
saccade is operated with the previous winner inhibited.
Key Algorithms(13) – LTM


Limitations of current implementation :
 Need to store all possible actions
 Cost of performing an action is not considered
 Only one step action is considered
 Objects can’t be indentified unambiguously
 The connection between object and pain is static for now
For more details:



J. A. Starzyk, J. T. Graham, P. Raif, A-H. Tan. "Motivated Learning for the
Development of Autonomous Systems" . Cognitive Systems Research. 14
(2012), pp. 10-25.
J. A. Starzyk, "Motivation in Embodied Intelligence," in Frontiers in Robotics,
Automation and Control, I-Tech Education and Publishing, Oct. 2008, pp. 83110.
J.A. Starzyk, “Mental Saccades in Control of Cognitive Process”, Int. Joint
Conf. on Neural Networks, San Jose, CA, July 31 - August 5, 2011.
Key Algorithms (14) – DM&P


Executive Intelligence center.
Functionality



Attention Switching : Attention
spotlight;
Decision Making : Choose
current task;
Planning : Develop procedure
for current task;
Key Algorithms (15) – DM&P

Current spotlight?




Unexpected change (Sudden)
Avert harm (Surviving)
Pursue Gains (Thriving)
Preliminary
Task Selection




Based on current spotlight
Evaluation of all the potential tasks
Prioritization of all the effective tasks
Selection the task with the highest priority is the
selected task
Key Algorithms (16) – DM&P

Evaluation :
Ui  wP  p  wC  c  wT  Tfin
Δp
pain reduction by task
wC
task cost weight
Δc
cost of resource consumption
by task
wP
task pain reduction weight
Tfin
percentage completion of
task
wT
task completion weight
biological
psychological
Key Algorithms (17) – DM&P

Prioritization



Time window
Urgency : urgent  non-urgent
Utility: High  low
Current time =1
Key Algorithms (18) – DM&P

Planning




"Relinquish",
"Defer",
"Continue"
"Resume"
Simulation

An extraterrestrial exploration scenario



Resources : Base, Energy pill, Sun, Shelter, Big rock, Medium
rock and Small rock. The agent is equipped with a cooler and
heater to adjust its internal temperature.
Behavior: Use, Go to, Repair, Turn on, Turn off, Break, Build,
Explore.
Motivation :


Primitive pains :Damage, Hunger, Low Temperature, High Temperature,
Mission, and Curiosity.
Abstract pains : Base, Energy pill, Sun, Heater, Cooler, Big rock, Medium
rock, Shelter, and Small rock.
Conclusion & Future work (1)

A biopsychically inspired cognitive system





Self-motivated rather than directed goal setting and pursuit;
Evolved rather than programmed cognition;
Capable of unsupervised learning;
Able to improvise, innovate and imagine;
Possessing distinct personality and unique abilities;
Conclusion & Future work (2)






The further integration work is expected and the code
will be ported into C++ and NeoAxis
How to handle complexity of cognitive control systems
Cognitive control law (decision making, planning and
execution)
Dynamic modeling of the cognitive process
Psychologically inspired DM&P algorithm :
GGB,STM.
Time perspective : Organizers; Crammers; Relaters;
Visioners.
Simulation

An extraterrestrial exploration scenario



Resources : Base, Energy pill, Sun, Shelter, Big rock, Medium
rock and Small rock. The agent is equipped with a cooler and
heater to adjust its internal temperature.
Behavior: Use, Go to, Repair, Turn on, Turn off, Break, Build,
Explore.
Motivation :


Primitive pains :Damage, Hunger, Low Temperature, High Temperature,
Mission, and Curiosity.
Abstract pains : Base, Energy pill, Sun, Heater, Cooler, Big rock, Medium
rock, Shelter, and Small rock.
T!
&
Q?