Design of Web Agents Inspired by Brain
Research
Maya Dimitrova, Hiroaki Wagatsuma * and
Yoko Yamaguchi*
Institute of Control and System Research –
Bulgarian Academy of Sciences
* Laboratory for Dynamics of Emergent Intelligence,
RIKEN Brain Science Institute, Japan
Content
1. Introduction
2. Information retrieval studies based on cognitive
heuristics (problem definition)
3. Theta phase coding in the brain and episodic
memory
3.1. The Theta phase model
3.2. Relevance to episodic/autobiographical memory
4. Autobiographical memory in Web context: An
example
5. Conclusions and future work
30/09/2007
ADBIS'07 Varna
2
Introduction
 Modeling agents’ own experience – their
reconstructed past - is essential to optimize agent
behavior (i.e. agents for Web knowledge search,
retrieval and presentation)
 We propose to base this optimization on recent studies
and models of the brain – focusing on the generality
of processing in the animal and human, not on their
difference
 We have focused on “episodic” memory and the
underlying hippocampal mechanisms of processing as
closest to the “story-telling” approach of agent design
30/09/2007
ADBIS'07 Varna
3
Information retrieval studies based on
cognitive heuristics
 Problem statement:
- Current Web page classifiers perform excessive amount of
processing to reach simple classificatory results
- Whenever the classificatory task is multi-dimensional, agent
performance deteriorates dramatically
- There is the database or storage problem – where models have
always been least analogous to human memory
- Human memory processing is fast, heuristical and much more
efficient in the multi-dimensional case, than in the sequential
processing case (contrary to the machine) etc. (recent finding)
- There is always a point in looking into recent brain studies –
for their fast and accelerated development as empirically-based
science
30/09/2007
ADBIS'07 Varna
4
Information retrieval studies based on
cognitive heuristics (2)
 The current study:
- Presents the main ideas of the Theta phase precession
theory as proposing a memory model, different from
the mainstream rate-coding models
- Operationalizes “episodic memory” for the purpose of
design of memory-optimized Web search agent
- Provides an example to illustrate the described
process in a human Web search process and some
relevant conclusions for agent future development
30/09/2007
ADBIS'07 Varna
5
The agent (1)
 In earlier work we
designed a Javaservlet (a wrapper
to a search engine)
to diagnose the
level of expertise
and detail of a
corpus of on-line
available Web
pages
 Weak point – what
to store in the
database and in
what form
Types (sample, test
set) of documents
Results of the online
classification on a specific
criterion
Adjustment of the
parameters of the classifier
Document
of type
Lexical
Database
Document
of type
Classifier
Document
of type ∆
∆
(heuristics and
methods for
classification)
Document
of type Ω
Ω
..
.
Example of classified
documents on one
criterion
..
.
∆
Example of classified
documents on two
criteria
Legend
∆
Ω
30/09/2007
Process of comparison for
validity of the results
∆
- expert level
- detail
- illustration
- explanation
ADBIS'07 Varna
Ω
..
.
6
The agent (2)
 It employed
pattern
classification
based on syntactic
analysis for
diagnosing the
amount of detail
 And cognitive
heuristics based on
memory research
for the expert level
of the text
Types (sample, test
set) of documents
Results of the online
classification on a specific
criterion
Adjustment of the
parameters of the classifier
Document
of type
Lexical
Database
Document
of type
Classifier
Document
of type ∆
∆
(heuristics and
methods for
classification)
Document
of type Ω
Ω
..
.
Example of classified
documents on one
criterion
..
.
∆
Example of classified
documents on two
criteria
Legend
∆
Ω
30/09/2007
Process of comparison for
validity of the results
∆
- expert level
- detail
- illustration
- explanation
ADBIS'07 Varna
Ω
..
.
7
The agent (3)
 In following history
of interaction,
optimization is
needed much the
way natural
cognitive systems
perform - learn,
abstract, remember
and forget
 We use the
‘autobiographical
memory’ metaphor
to apply to such
kind of Web agents
and the Theta phase
precession theory to
model it
30/09/2007
Types (sample, test
set) of documents
Process of comparison for
validity of the results
Results of the online
classification on a specific
criterion
Adjustment of the
parameters of the classifier
Document
of type
Lexical
Database
Document
of type
Classifier
Document
of type ∆
∆
(heuristics and
methods for
classification)
Document
of type Ω
Ω
..
.
Example of classified
documents on one
criterion
..
.
∆
Example of classified
documents on two
criteria
Legend
∆
Ω
∆
- expert level
- detail
- illustration
- explanation
ADBIS'07 Varna
Ω
..
.
8
Current MATLAB GUI version
of the agent interface
 The difference of
the MatLab version
from the Java
version is that the
Java-servlet
processed also the
hyperlink structure
of the Web page
 The current version
is simplified for
algorithmic and
modeling reasons
30/09/2007
ADBIS'07 Varna
9
Assumptions of the current study:
 Assumptions:
- User interaction with the present-day Web is an
autobiographical experience (remembering
sequences of events like personal-life episodes)
- Memorizing events in a human-like memory
system undergoes constant synthesis as any
other learning process and therefore can be
implemented in autonomous agent on the Web
30/09/2007
ADBIS'07 Varna
10
Operational definitions in the current study:
 Definition of Episodic Memory:
- The system of processing episodes of Web search and
transforming them into autobiographical events of
successful/unsuccessful Web navigation to reach the
needed goal
- Every Web page viewed by the user is a complex
perceptual and semantic stimulus, i.e. an episode of
Web search
- What is stored in episodic memory from the rapid
viewing of a succession of pages is a set of fragments
of the search episode rather than the entire pages and
their exact order of viewing
30/09/2007
ADBIS'07 Varna
11
Hypotheses of the current study:
- The theoretical hypothesis is that episodic memory
extracts meaningful cues from the fragments of the
page-retrieval episodes, rather than random perceptual
or semantic cues
- “Meaningful” we define as having anticipatory value
(i.e. useful in the future)
- The tested hypothesis is that the mechanisms of event
synthesis from fragments of page-retrieval episodes are
the same in personally-significant and neutral
situations, that is – universal for learning per se
- These effects are reproduced by both the human and
the agent in the current study
30/09/2007
ADBIS'07 Varna
12
Theta phase coding in the brain and
episodic memory
 Theta phase theory models rhythmic patterns of activity of
assemblies of neurons in the brain and their role in
complex cognitive tasks (theta rhythm 4-10 Hz)
 Theta phase modulation is associated with learning and
memory in rats and humans A mechanism of pattern
compression has been observed in hippocampal neuron
assemblies, which is called theta phase precession
 Modelling hippocampal nerve cirquit by the Theta phase
precession theory (Yamaguchi, Yamaguchi et al.) reinstates
complex learning effects observed in human episodic
memory like object-context integration, single-trial
learning and time-space contextualization
30/09/2007
ADBIS'07 Varna
13
Neuro-realistic model of episodic encoding
and retrieval
 (a) Main modules of the hippocampus – ECII, ECIII, CA3,
CA1, DG
 (b) Entorhinal input, emerging from the sensory system and
representing the vector time series produced by behavior
 (c) when the rat runs to the right, the phase shift in firing within
each theta rhythm cycle occurs in place cells 1 to 4, which are
activated sequentially
30/09/2007
ADBIS'07 Varna
14
Properties of the phase model
 (The recurrent connections in the CA3 layer of the
hippocampus form asymmetric connections so that
units with prior activations are connected to those
with subsequent activations
 These activations are a consequence of the
coincidence between their phase differences and the
time window of the asymmetric synaptic plasticity
30/09/2007
ADBIS'07 Varna
15
Model of hippocampal nerve circuit
 (a) mechanism of theta phase coding
 (b) phase shift in firing within each theta rhythm cycle
30/09/2007
ADBIS'07 Varna
16
Relevance to modeling memory:
 Partial cuing triggers pattern completion in the CA3 and its
transmission to the CA1 and EC. Thus a stored pattern in
the hippocampus can be sent to the cortical area to replay
the input pattern
 The temporal difference between encoding and retrieval in
the network parallels behavioral tests
 The retrieval is ten times faster than the behavioral input in
the encoding stage and the retrieval time scale is similar to
that of the compressed pattern of theta phase precession
 The retrieved activity can cause further synaptic plasticity
in other cortical areas for memory consolidation or
memory transfer from the hippocampus to the cortex
30/09/2007
ADBIS'07 Varna
17
Importance of theta phase precession
theory to the present study:
 The emergence of rhythmic patterns in the
absence of memory to cause its formation is
a plausible candidate for a neural correlate
of the formation of complex and durable
cognitive effects governing future behavior
i.e. autobiographical memories
 We have aimed to give a relevant example
to illustrate these effects
30/09/2007
ADBIS'07 Varna
18
How to find an Art Gallery on the Web
with incomplete initial information?
 Participants:
- Motivated
- Neutral
 Search goal:
- Example of new concept learning, similar to
artificial concept learning: “Kurenai Kai”
 Search task:
- Google search from incomplete initial information
(The name of the gallery “Kurenai Kai” not
known apriory)
30/09/2007
ADBIS'07 Varna
19
Results and discussion:
Table 1. User performance on Web search with incomplete initial information about
the search goal
User
Motivated
Neutral
Searches
Random
Meaningful
23
3-5
19
3-5
Time
Approximate
30 min
30 min
Our study of Web search has replicated the mechanism of
performance of the theta phase precession model :
- The Web paths resemble random search until a meaningful path is
composed
-We observe that meaningful paths are composed, rather than
encountered
30/09/2007
ADBIS'07 Varna
20
Web search: “Kurenai Kai”
 Left path represents the nature of the search with incomplete initial information
 Right path represents the shortest path with well-remembered concept term
30/09/2007
ADBIS'07 Varna
21
Remembering meaningful fragments of search episodes
Left path represents episodic memory guided retrieval based on a set of meaningful cues and “insight”
The pages to the right represent the mappings to the respective initial cues (none of them are identical)
30/09/2007
ADBIS'07 Varna
22
Conclusions from the Web search study:
 Users base subsequent retrieval of the search path on
remembering the ‘events’ of their search, which are the
meaningful fragments of knowledge inside the Web pages,
not the exact ‘page-retrieval’ episodes
 We propose that the autonomous Web agent behaves in
analogy with the event-based (autobiographical) strategy
that Web users are applying for retrieval of useful
information, which is some state of knowledge with
anticipatory value (useful in the future)
 Next will be tests of the agent for optimized storage of
fragments of knowledge contained in the Web page, rather
than of snapshots of sequences of Web pages
30/09/2007
ADBIS'07 Varna
23
Agent reinstatement of the search task
Fragment
extraction and
recombination
of textual
elements as a
process of
encoding
search events
30/09/2007
ADBIS'07 Varna
24
Temporal evolution of episodic events by the neuro-cognitive agent
30/09/2007
ADBIS'07 Varna
25
Conclusions and Future Work
 We presented one approach towards the design of
Web agents based on recent knowledge from brain
research
 We have tried to demonstrate how the theoretical
framework of theta phase coding endows real-time
process of memory formation of Web search
experiences and the retrieval process of effective
routes to the targets through the experienced sites
 The proposed approach focuses on bridging the
gap between neuroscience and Web technologies
 It is currently being applied to retrieval of
cardiological knowledge from the Web
30/09/2007
ADBIS'07 Varna
26
Acknowledgement
This work is supported by:
NSRF of the Bulgarian Ministry of Education
and Science as part of the Research Project
№ MI - 1509/2005
”Multimodal User and Sensor Interface in a
Computer-Aided System for Cardiological
Diagnosis and Intervention”
30/09/2007
ADBIS'07 Varna
27
Thank you
for your attention
30/09/2007
ADBIS'07 Varna
28