Information Binding with
Dynamic Associative Representations
ARAKAWA, Naoya
Imaging Science and Engineering Laboratory, Tokyo Institute of Technology
Abstract. This paper proposes an explanation of human cognitive functions in terms of association. In this proposal, representation is not static
but is achieved by dynamic retrieval of patterns by means of association. With this dynamic representation, information binding in cognitive
functions such as physical world recognition, planning and language understanding is illustrated.
Keywords: association, generativity, dynamic representation, the binding problem
1
Introduction
This paper addresses the issue of representation in human cognitive functions
in terms of association and attempts to show the benefit of seeing cognitive
representation as dynamic associative processes with examples of perception,
planning and language understanding. In this paper, association designates the
process of obtaining a pattern from another through learning. As discussed in
[4], patterns here can be sub-symbolic vector representations formed by nonsupervised learning.
The reason why association is taken to be the basis for explaining cognitive functions here is two-fold: 1) cognitive functions are often accounted for
with sub-symbolic information processing and association is considered to be
its abstraction; 2) while sub-symbolic information processing is formalized with
models such as neural networks and Bayesian models, its technical details could
obscure the essence of over-all functions in certain discussions. Having said that
association is an abstraction of sub-symbolic information processing, the following are reasons why sub-symbolic information processing is important for
explaining cognitive functions. First, for models of brain functions such as neural networks are normally assumed to be sub-symbolic, their incorporation would
be essential for biologically natural or realistic cognitive modeling. Second, while
learning is an essential cognitive function, most modern-day learning models are
sub-symbolic. Finally, sub-symbolic information processing would evade thorny
issues in cognitive science such as the issue of classical categories brought forth by
cognitive linguists such as Lakoff [7] and the frame problem. Associative memory
█2
could evade the issue of classical categories, since sub-symbolic associative models (e.g., neural models) can represent prototypical and fuzzy categories. Certain
frame problems for intelligent agents to find relevant information could also be
evaded, as sub-symbolic associative memory could retrieve most relevant information (association) for given patterns first (by means of a competitive process
among‘ neurons, ’for example) (see [5] for a debate).
However, there are numerous problems with using association with cognitive
modeling. First of all, it is not clear if association alone is enough for explaining
dynamic cognitive functions such as planning or language understanding, as
association is nothing more than mapping between patterns. Secondly, as it was
argued in the computationalism vs. connectionism debate (see [2]), it is not
clear how connectionist/associationist models account for generativity of, say,
linguistic structure. Moreover, associationism also bears the so-called binding
problem for explaining the integration of various pieces of information (see [12]
and 2.1 below).
Given the aforementioned pros and cons, my motivation is to explain human
cognitive functions as an associative processes by addressing the problems raised
above.
2
The Proposal
The basic idea of the proposal is that associative representation is better conceived dynamically as traversal of association paths, if we are to address the
issues of information binding and generativity. Note that the issues are not
problematic in the case of classical symbolic models (e.g., [9]) or symbolic/subsymbolic hybrid cognitive architectures (e.g., [1] [10]), in which the composition/binding of information can be done symbolically by means of variable binding. However, when we regard the human brain as a sub-symbolic associative
machine, the apparent lack of binding mechanism becomes problematic for explaining human cognitive functions. Therefore, a (sub-) motivation here is to
explain cognitive functions with an associative mechanism without bringing in
variable binding. While the motivation is mainly explanatory, the proposal may
entertain technological merit: a cognitive system may be constructed with subsymbolic learning mechanisms without generating symbolic rules for the sake
of simplicity, although whether such a system has a practical advantage would
depend on the case.
In the sections to follow, the proposed method will be explained more in
detail by using concrete examples. First, physical world (scene) recognition is
discussed as the basic case of the binding problem. Then, discussions of planning
and language understanding will follow.
2.1
Physical world recognition
Recognizing the physical environment involves recognizing objects, their locations and features such as motion. When a cognitive system has to perceive
multiple objects at once, the binding problem occurs: the system has to extract
█3
features from each entity and ascribe them to the proper item. The problem is
acute with parallel information processing presumably performed in our brain;
how can pieces of information represented in parallel be integrated in a coherent representation? For example, seeing a blue circle and a red rectangle, how
does a brain integrate the representations of blue, red, circle and rectangle as
those of a blue circle and a red rectangle in a particular relative location? In the
current binding problem literature, synchronicity is often mentioned as a possible solution to the problem ([12][3], for example). However, synchronicity alone
does not solve the problem. In the first place, it does not answer the question
of how different objects are represented in the brain. Secondly, considering that
visual scenes are perceived with saccades, there remains a suspicion that visual
perception is rather dynamic than a synchronic process.
An associative system could cope
with the (perceptual) binding problem
in the following manner. Suppose that
the system obtains a feature pattern
of a physical object as well as a pattern representing the coordinates1 of
the object and that it obtains the information of an object at a time. In
this situation, the system shifts ‘ attention ’ among objects to recognize Fig. 1 Association among figure repremore than one object2 . The associa- sentations
tive retrieval of object representation Each figure represents feature patterns for
could be driven by cues representing the figure. The numbers represent cue patorientations (such as 15 degrees left) terns for relative degrees and distances.
and relative distances (Fig. 1). In this
scheme, a scene (containing more than one object) may not be simultaneously
represented, but object representations may be retrieved one-by-one by means
of association in short-term memory or working memory and bound together as
a coherent scene representation. What assures the information binding here is
not synchronicity but the potential or mechanism to bring all relevant representations together.
2.2
Planning
A planning mechanism by means of association could be conceived in a rather
straightforward way as described by the following (pseudo) algorithm:
1
2
Information on the location of physical objects can be obtained via integrating various modalities such as vision, touch, vestibular sense and the sense of eye rotation.
The idea is in line with the Feature Integration Theory [14] but formulated in a more
abstract level.
4
█
if the pattern representing the current situation is not desirable according to
certain criteria, then
associate the current situation pattern (CSP) with an action pattern
(APT), which represents either a directly executable action or a plan.
if CSP and APT are not associated with (do not predict) a pattern
representing a satisfactory situation, then
associate another APT with CSP by suppressing the current APT.
end
if APT represents a learned plan (PL), then
LP: associate APT with a sequence of APTs (APT1, APT2,…).
if the situation pattern associated with (predicted from) CSP and a
sequence consisting of APTi (i=1,2,…) is evaluated as unsatisfactory,
then
abort and suppress PL.
end
if APTi is a learned plan, then
retrieve a sub-sequence for APTi recursively (⇒ LP:).
end
if all the sub-sequences consist of directly executable actions and the
patterns of situations associated (predicted) with the entire APTs and
CSP are satisfactory, then
the entire action (a new plan) is ready to be executed.
end
end
While such an algorithm
is a variation of classical
planning algorithms, some
notes can be given for
that to be realized associatively. In planning, the output sequence could have any
length and plan hierarchy
could have any depth. In
other words, a plan is a cognitive representation having
a combinatory/generative nature. A combinatory representation can be represented
Fig. 2 Planning by means of association
with an associative network,
Ovals are patterns representing the indicated
which actually is a dynamic
content. Blue solid lines represent associative
process of recalling (associrelations.
ating) patterns. In case of a
plan, the entire plan can be represented by retrieving its parts one by one by
means of association. A representation of an action in a plan may be retrieved
5
█
from a representation of the preceding action with a cue pattern representing
the temporal relation succeeding. Here, note that in the sample algorithm above,
a plan is formed (information is bound) with association but without having
recourse to rules with variables.
A plan hierarchy (Fig. 2) would be represented by associating a pattern
representing a learned plan with patterns representing actions or sub-plans3 .
Constructing hierarchical plans requires back-tracking, which, in turn, requires
a mechanism that suppresses an unwanted pattern and retrieves another relevant
pattern (by means of association).
2.3
Language understanding
Understanding language involves syntactic parsing and associating syntactic
structures with semantic representations. As a sentence in a human language
can be indefinitely long, the corresponding associative syntactic/semantic representation can be indefinitely large. Again, such representation has the combinatory/generative nature. While an associative network may not be able to
keep simultaneous activation of patterns for a large representation, the entire
representation could be retrieved piece by piece.
As for semantic representation,
computational models known as
semantic networks (e.g., [11]) are
considered to be associative. A
node of a semantic network represents an individual or type of physical objects, events, situations, time
or location.4 Semantic networks
normally have labeled edges to represent relations between nodes. A
labeled node may be realized in
an associative network as a patFig. 3 A regular phrase structure
tern representing a node and a cue
Blue solid lines can be associative relations,
pattern representing the type of
where ovals represent patterns for syntactic
an edge or relation. For example,
categories.
the edge or relation representing
bigger-than may be realized as association from a pattern representing an individual and a (cue) pattern representing bigger-than to the representation of
another individual.
With regard to parsing, the syntax of human language has recursive structure, i.e., a structure may be embedded in another structure. Such recursive
3
4
[13] can be an exemplary work in this line.
A node in an associative network may accrue meaning by means of non-supervised
learning so that they can be interpreted as a node of a ‘ semantic ’ network. In
fact, a semantic network alone does not give the semantics to its components. See
discussion in [6].
█6
structure would be constructed by a mechanism similar to the one discussed
in the section of planning above, where the construction of sentential structure
would be the goal. The system could traverse a syntactic tree by associating
daughter constituents with the parent node and associating one node to its sibling nodes with cue patterns representing such as left and right (Fig. 3).
Finally, mapping from
syntactic structure to semantic structure must be
taken into consideration.
A pattern representing
a syntactic constituent
such as a clause, verb
and noun may be associated with a pattern representing situation, event
and object respectively.
Here, syntactic relations
may be associated with
Fig. 4 Syntactic and Semantic Patterns
patterns representing semantic relations. For example, an English verb phrase consisting of a verb and
a following noun (having the accusative relation) would be associated with the
representation of an event, an object and the theme relation between them (Fig.
4).
3
Conclusion
The associative representations proposed herein have shared characteristics. In
the three cases, representation is not static but is achieved by dynamically retrieving patterns by association. The associative retrieval is accompanied by cue
patterns such as those indicating spatial and temporal directions and semantic/syntactic relations. The proposed representations also cope with the issues
of dynamicity and generativity and the binding problem. As for dynamicity,
planning and language understanding discussed above are dynamic and the representations proposed here are all dynamic. As for generativity, representations
in planning and language understanding are generative and those in physical
world recognition can also be generative, as a scene is composed of many objects.
The issue of binding without variable was addressed in physical world recognition as well as in language understanding and planning, as these processes must
also bind pieces of information together into coherent representations.
The proposal here apparently requires empirical studies. In particular, if
the illustrated mechanisms are to serve as certain functions, association should
be properly controlled. So, while the author plans to implement experimental
systems for corroborating the proposal, the issue of the (executive) control [8]
shall be seriously addressed.
█7
Acknowledgements
I would like to express my deep gratitude to the comments from the referees of
this paper and also from Quentin Quarles, without which the paper did not form
the current shape.
References
[1] ACT-R: http://act-r.psy.cmu.edu
[2] Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: A critical
analysis. Cognition 28(1), 3–71 (1988)
[3] Fuster, J.M.: Cortex and mind: Unifying cognition. Oxford University Press (2005)
[4] Gärdenfors, P.: Conceptual spaces: The geometry of thought. MIT press (2004)
[5] Haselager, W., Van Rappard, J.: Connectionism, systematicity, and the frame
problem. Minds and Machines 8(2), 161–179 (1998)
[6] Johnson-Laird, P.N., Herrmann, D.J., Chaffin, R.: Only connections: A critique
of semantic networks. Psychological Bulletin 96(2), 292–315 (1984)
[7] Lakoff, G.: Women, fire, and dangerous things: What categories reveal about the
mind. University Of Chicago Press (1990)
[8] Miyake, A., Shah, P.: Models of working memory: Mechanisms of active maintenance and executive control. Cambridge University Press (1999)
[9] Newell, A.: Unified theories of cognition. Harvard University Press (1994)
[10] OpenCog: http://wiki.opencog.org/w/The Open Cognition Project
[11] Quillian, M.R.: Semantic memory. In: Minsky, M. (ed.) Semantic information
processing, pp. 227–259. MIT Press (1968)
[12] Revonsuo, A.: Binding and the phenomenal unity of consciousness. Consciousness
and cognition 8(2), 173–185 (1999)
[13] Subagdja, B., Tan, A.H.: A self-organizing neural network architecture for intentional planning agents. In: Proceedings of The 8th International Conference on
Autonomous Agents and Multiagent Systems-Volume 2. pp. 1081–1088. International Foundation for Autonomous Agents and Multiagent Systems (2009)
[14] Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cognitive
psychology 12(1), 97–136 (1980)