In: Proceedings of the GALA ’97 Conference on Language Acquisition (1997), pp 393–398. Edinburgh, UK: HCRC
A Constructivist Neural Network Learns the Past Tense of English Verbs
Gert Westermann
Centre for Cognitive Science
University of Edinburgh
2 Buccleuch Place, Edinburgh EH8 9LW
[email protected]
Abstract
A constructivist neural network is presented that models the acquisition of the past tense of English verbs.
The network constructs its architecture in response
to the learning task, corresponding to neurobiological
and psychological evidence. The model outperforms
other connectionist and symbolic models in learning
and in displaying psychologically realistic learning
and generalization behavior. It is argued that the success of the network is due to its constructivist nature,
and that the distinction between fixed architecture and
constructivist models is fundamental. Given this distinction, constructivist systems constitute better models of cognitive development.
1. Introduction
The acquisition of the English past tense has in the
past years become a touchstone for different theories of language acquisition and of cognition in general. Different theories and models have not only been
used in the debate between proponents of symbolic
and connectionist accounts of language learning, but
have also raised the question whether regular and irregular past tense forms necessarily rely on different
mechanisms for their formation or whether a single
mechanism can account for both. While most connectionist accounts (Rumelhart & McClelland 1986,
MacWhinney & Leinbach 1991, Plunkett & Marchman 1993) generally argue that a homogeneous architecture is sufficient for both forms, hybrid theories (Pinker 1991) explain regular cases with a rule
and irregulars with an associative memory. There exist, however, modular connectionist (Westermann &
Goebel 1995) and homogeneous symbolic (Ling &
Marinov 1993) models of inflection acquisition.
What is common to most of these models is that
they rely on a fixed, pre-defined architecture, an assumption which, as argued below, is unrealistic and
poses severe problems for their usefulness as models of cognitive development. In this paper, a constructivist neural network for learning the past tense
is described that builds its architecture in response
to the learning task. The network is compared with
three other implemented models of past tense acquisition: the original pattern associator (Rumelhart & McClelland 1986, R&M), the improved backpropagation
network by MacWhinney & Leinbach (1991) (M&L)
which took the extensive criticism of the R&M model
into account, and the Symbolic Pattern Associator
(SPA, Ling & Marinov 1993), which took up the challenge posed by M&L to present an implemented symbolic system (rather than just a theory) for past tense
acquisition. It is shown that the constructivist neural network presented here outperforms the existing
connectionist and symbolic models both in learning
the task and in the display of psychologically realistic
learning and generalization behavior. It will argued
that such a model can help bridging the gap between
symbolic and connectionist, and modular and homogeneous theories of inflection acquisition.
The rest of this paper is organized as follows: in
section 2 the argument is made that constructivist development is a necessary condition for realistic models of cognitive development. In section 3 a specific
constructivist neural network algorithm, Supervised
Growing Neural Gas, is described that was used for
the simulations in this paper. Section 4 is concerned
with the experimental setup, and in sections 5, 6, and
7, the results of the simulations are analyzed with respect to learning, a U-shaped learning curve, and generalization performance, respectively. These results
are then discussed in section 8.
2. Why Constructivist Learning?
Cognitive development has recently been argued to
closely correlate with the structural development of
the cortex, with an increase in structural complexity
leading to an increase in cognitive capacities (Quartz
& Sejnowski 1998, Johnson 1997). In order to understand the principles of cognitive development it is
therefore important to take the mechanisms of brain
development into account. Recent work in this area
has provided evidence that the development of cortex is activity dependent on different levels (see e.g.,
Van Ooyen 1994): activity can determine the rate and
direction of dendritic and axonal growth and the formation of synapses (e.g., Quartz & Sejnowski 1998).
Stabilization and loss of these synapses is also activity
dependent (Fields & Nelson 1992). It has further been
shown in transplantation and rewiring studies that cortical areas are not innately prespecified to assume a
certain functionality, but readily adapt to process afferent signals from different domains (O’Leary 1989).
Further, cortex remains flexible to a certain degree
3. The Supervised Growing Neural Gas
Algorithm
There exist now a great number of constructivist neural network algorithms, most of which have been designed to overcome the shortcomings of fixed architecture networks (the need for choosing a predefined
architecture, slow learning time, uniform allocation of
resources for tasks of varying complexity). For the
cognitive simulations described here, a modified version of the Supervised Growing Neural Gas (SGNG)
algorithm (Fritzke 1994) was used because it incorporates constructive and regressive events which depend
on the learning task and because it provides mechanisms to produce outputs based on both the structure
and on the identity of input signals, conforming to
both neurobiological and psychological evidence.
Activation
throughout life, with dendritic density increasing until
a late age (Uylings et al. 1978). These results indicate
that neural development proceeds in a constructivist
way, with the neural organization of the brain being
modified through constructive and regressive events
by complex interactions between genetic predispositions and environmental inputs.
Cognitive development which is based on cortical development will thus proceed in the same constructivist way, where activity dependent architectural
modifications lead to increasingly complex cognitive
representations.
Most significantly, research in learning theory
(Baum 1989, Quartz 1993) has shown that incorporating activity dependent structural modification into
a learning system is not just a way to tune performance, but leads to entirely different learning properties of that system, evading many of the problems that
are associated with fixed-architecture systems. Such
constructivist systems can overcome Fodor’s paradox
(Fodor 1980) which claims that no new representations can be learned and thus argues for innate representations (Quartz 1993), and they can overcome the
prohibitive time complexity of even simple learning
tasks in fixed-architecture systems (Baum 1989).
Any model which aims to capture the essential
properties of human cognitive development must take
these results into account: cognitive models should
therefore employ neural networks which, like the
brain, adapt their architecture in a way specific to the
learning task. These models can be called constructivist networks, reflecting their proximity to the constructivist developmental theories of Piaget in which
structural modification of the learning system occurs
in response to environmental input.
In this paper a constructivist neural network model
is employed for the simulation of past tense acquisition and compared with previous past tense models.
This will allow to empirically assess the suitability of
constructivist networks in the modeling of cognitive
development.
Input x Position wc
Figure 1: A Gaussian activation function which acts as
a receptive field to near inputs (viewed from the side
and from the top).
The SGNG algorithm constructively builds the hidden layer of a radial basis function (RBF) network.
Such an RBF network is different from the more
common backpropagation networks in that the hidden units do not have a sigmoid but a Gaussian, ‘bellshaped’ activation function (see figure 1). This allows
each hidden unit to be active only for inputs within a
certain range (as opposed to being active for all inputs
above a certain threshold, as with sigmoidal units) and
it can thus be viewed as a receptive field for a region
of the input space. All input vectors obtain a position in this space (determined by their values), and the
hidden units are placed at different positions to cover
the whole space. Hidden units will be activated by an
input if it falls within their receptive fields, and the
closer the input is to the center of the field, the more
the unit will be activated.
The problem in building RBF networks is to decide
on the number and positions of the hidden units, because inputs falling into a common receptive field will
lead to similar outputs. The SGNG algorithm solves
this problem by building the hidden layer and adding
units when and where they are needed. The algorithm
starts with just two hidden units. When an input is presented to the network, the hidden unit which is closest to this input (i.e., the winning unit) together with
its direct topological neighbors are moved towards the
input signal – this prevents hidden units to remain in
regions of the input space where no inputs occur; hidden units which never win eventually die off. The
activation from the hidden units is propagated to the
output and the output error is calculated. This error
is added onto a local counter variable of the winning
unit, and the weights between the hidden and output
units are adjusted (e.g., with the delta rule). A new
hidden unit is inserted when the performance of the
network no longer improves in the current architecture (i.e., when the error decreases less than a certain
value within a certain number of epochs). The new
unit is inserted next to the hidden unit which has accumulated the highest error (note that only winning units
can accumulate error). The idea here is that a winning
unit which produces a high output error is inadequate
(because it covers inputs with conflicting outputs), and
more structural resources are needed in that area. On
insertion of a unit the sizes of the receptive fields are
shrunk so that they slightly overlap with each other;
this in effect leads to a more “fine-grained” resolution
in that area of the input space.
Initial
Final
00111100
0011
00111100
0011
0011
11001100
00111100 11001100
111
0011 000
111 000
00111100
00111100
00111100
Figure 2: Covering of the two-dimensional input
space by receptive fields at the beginning (left) and
the end (right) of learning.
Figure 2 shows a hypothetical start and end state in
a two-dimensional input space. While initially only
two receptive fields cover the whole of the space, at
the end hidden units have been inserted with different
densities across the space to account for the specific
learning task.
brought
b
r
O
t
111111
000000
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
000000
111111
0000000
0000000
0000000
0000000
0000000
0000000
0000000
0000000
0000000
0000001111111
111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
0000000
1111111
b
r
I
4. Experiments
For our simulations, we borrowed the corpus from
MacWhinney & Leinbach (1991) which consists of
1,404 stem/past tense pairs of English verbs. This corpus was also used by Ling & Marinov (1993) in their
SPA to allow direct comparisons between models.
The verbs were transcribed using UNIBET and, following MacWhinney & Leinbach (1991), represented
in a templated format containing slots for consonants
and vowels. Table 1 shows examples for the templated
phonological encoding of some verbs. Each phoneme
was represented by ten features, such as voiced, labial,
dental for consonants, and front, center, high for vowels. A template consisted of 18 slots, resulting in a
180-bit featurevector for the representation of each
verb.
Output Layer
(Template)
Hidden Layer with
Gaussian Units
111111
000000
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
000000
111111
0000000
0000000
0000000
0000000
0000000
0000000
0000000
0000000
0000000
0000001111111
111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
00000001111111
1111111
0000000
1111111
When similar verbs lead to similar outputs, however,
(e.g., look and cook with looked and cooked),
no new receptive field will be inserted there and one
such field will cover different verbs without producing output error. Thus, the internal structure of the
network will adapt to reflect the learning task, and observing this adaptation can lead to insights into the
past tense learning process.
The next section describes the specific simulations
that were undertaken with the SGNG network model.
Input Layer
(Template)
N
Input
bring
Template
explain
Template
point
Template
recognize
Template
shake
Template
br-I-N-----------CCCVVCCCVVCCCVVCCC
---I-ksp--l--e-n-CCCVVCCCVVCCCVVCCC
p--2-nt----------CCCVVCCCVVCCCVVCCC
r--E-k--I-gn-3-z-CCCVVCCCVVCCCVVCCC
S--e-k-----------CCCVVCCCVVCCCVVCCC
Output
brought
explained
pointed
recognized
shook
br-O-t-----------CCCVVCCCVVCCCVVCCC
---I-ksp--l--e-ndCCCVVCCCVVCCCVVCCC
p--2-nt-I-d------CCCVVCCCVVCCCVVCCC
r--E-k--I-gn-3-zdCCCVVCCCVVCCCVVCCC
S--U-k-----------CCCVVCCCVVCCCVVCCC
Table 1: Some examples for the template-encoding of
verb pairs.
bring
Figure 3: The initial SGNG network modified with
direct input-output connections. All layers are fully
connected.
Figure 3 shows the whole SGNG network. For
the cognitive simulations described here, the original
SGNG network was extended with direct connections
between the input and the output layer. These connections allow the past tense to be produced through a
direct structural transformation of the input stem. By
contrast, the (growing) hidden layer acts as a memory:
it produces an output based on the identity and not
the structure of the input verb. Initially, though, similar input verbs fall into the same receptive fields even
when they require different outputs (e.g., hear and
fear requiring heard and feared, respectively).
This problem is overcome in the training of the network through the insertion of new receptive fields in
the area of such verbs, and eventually similar verbs
with dissimilar past tense forms will be discriminated.
From the original corpus of 24,802 tokens, 8,000
tokens were randomly extracted according to the frequency of their past tense forms. The resulting training corpus thus consisted of 8,000 tokens (57.2% regular, 42.8% irregular), corresponding to 1,066 types
(88.4% regular, 11.6% irregular).
Training of the SGNG network proceeded in a nonincremental fashion: the whole training set of 8,000
stem/past tense pairs was presented to the network in
random order at each epoch. Hidden units were inserted depending on the learning progress (see section 3), and the network was tested for its performance
on the training set prior to each insertion.
5. Learning Results
After 912 epochs, the network produced 100% of the
irregular and 99.8% (all but two) of the regular past
tense forms correctly. At that point the network had
inserted a total of 400 hidden units. On average, there-
fore, each of the 400 hidden unit receptive fields covered 2.67 verbs.
Verb types
R&M
420
M&L
1,650
SPA
1,038
Constructivist
1,066
Percentage correct
Total
Regulars
Irregulars
97.0
98.0
95.0
99.3
100.0
90.7
99.2
99.6
96.6
99.8
99.8
100.0
Table 2: Performance on training of the four compared
models (extended from Ling & Marinov 1993).
Table 2 shows a comparison of the training results
of the different models. While all models performed
almost equally well on regular verbs, the constructivist network clearly outperformed the other models
on irregular verbs. This result indicated that the ability to add structure where needed was advantageous
in that it allowed the network to allocate resources
specifically for learning the irregular past tense forms.
This was confirmed by an analysis of the hidden layer:
while on average 2.7 regular verbs clustered to the
same hidden unit (with the maximum of 16 regular
verbs in one receptive field), this number was only 1.1
for irregular verbs. This result clearly shows the advantage of constructivist learning over the learning in
fixed-architecture systems: resources are not evenly
distributed a priori to handle all cases, but they can be
specifically allocated for the more difficult, or exceptional cases, whereas fewer resources are needed for
the easy, regular cases.
6. U-shaped Learning Curve
A plausible model of past tense acquisition should
follow the documented course of acquisition in children, that is, the U-shaped learning curve which has
been extensively studied (see e.g., Marcus et al. 1992):
while children initially produce a number of irregular verbs correctly, they subsequently overregularize
the same verbs and only in a final step produce them
correctly again. This phenomenon has theoretically
been explained with the inappropriate application of a
linguistic rule, but connectionist theories have argued
that it can arise due to regularities in a subtly changing
speech environment of the child.
The constructivist model described in this paper displayed a U-shaped learning curve for many of the irregular verbs: A period of overregularization was preceded by a phase of correct production of the past
tense form; this was the case e.g., for knew, sat,
made, took and said. Often, all forms were produced (irregular past, stem + regular ending, irregular past + regular ending; e.g., knew – knowed
– knewed – knew). For other, less frequent verbs
(e.g., wet, sell, cost), the first sampled past tense
form was overregularized. This corresponds to data
on overregularization in children (Marcus et al. 1992):
While in the corpora of Adam, Eve, and Sarah several
past tense forms were produced correctly before their
first overregularization, this was not always the case.
It is unclear, however, whether this is due to a lack of
speech samples, but presumably a child would overregularize an infrequent verb in its first usage.
The network displayed psychologically plausible
behavior in more specific aspects as well: on average,
the more frequent verbs were less often overregularized than less frequent ones. Further, clusters of irregular verbs acted as protection from overregularization:
the verbs ring, sing, and spring were overregularized only in 2.9% of all cases. By contrast, the
verbs hang, slide, and bear, which had a comparable token frequency, had an average overregularization rate of 15.4%.
The constructivist network model was thus successful in modeling the left side of the U-shaped learning
curve, i.e., a correct production of past tense forms
before their subsequent overregularization, and its performance corresponded to the details of children’s past
tense learning.
How does the U-shaped learning in the constructivist network occur? Since the verb set was held constant throughout training, the change in network performance could only be a consequence of the internal
reorganization of the network architecture. Initially,
the network had only two hidden units which were
of little use since they each covered about half of all
verbs with their varied past tense forms, and the network therefore had to rely on the direct input-output
connections for producing the past tense forms. Given
these restrictions the network initially learned to produce the past tense forms of the frequent irregulars
(because they were so frequent) and of many regular
verbs (because there were so many of them). When
learning stagnated and no more past tense forms could
be produced correctly, the network gradually grew its
hidden layer, adding more receptive fields which were
then used to account for more past tense forms. The
output forms were produced through a ‘collaboration’
of the direct connections with the newly established
hidden unit connections. The growing process lead
to a phase in which representations were already relocated to the hidden layer, but the few receptive fields
were large and included verbs that required different past tense forms, thereby leading to errors even
for verbs that had initially been produced correctly
through just the direct input-output connections. This
phase corresponds to the overregularization stage in
children. It is evident that with this mechanism, different verbs would be overregularized at different times,
depending on whether they had been allocated an individual receptive field. This process of internal reorganization of the network’s representations becomes
evident in figure 4, which shows the learning curves
for the regular and irregular past tense forms, but also
how many of these forms were still produced correctly
when the hidden layer was lesioned and only the direct
input-output connections were used.
10
9
8
human
constructivist
6
4
3
2
P
% correct
8
7
11001100
111
000
000
111
000
111
000
111
000
111
000
111
000
111
I
D
Irregulars
111
000
000
111
00111100111
000
000
111
11001100
111
000
000
111
6
5
111
000
000
111
000
111
4
3
2
1111
0000
0000
1111
0000
1111
111
000
000
111
111
000
000
111
000
111
1
0
P
I
D
Regulars
Figure 5: Generalization of the constructivist network
to different classes of pseudo-verbs, in comparison
with humans, the SPA, and R&M’s network (extended
from Ling & Marinov 1993). P = Prototypical, I = Intermediate, D = Distant.
40
regulars
regulars with lesioned hidden layer
irregulars
irregulars with lesioned hidden layer
20
9
R&M
111
000
000
111
000
111
0
60
10
SPA
000000000000000000000000000000
111111111111111111111111111111
111
000
000000000000000000000000000000
000111111111111111111111111111111
111
000
111
000000000000000000000000000000
111111111111111111111111111111
000
111
000
111
000000000000000000000000000000
0011111111111111111111111111111111
000
111
000
111
000000000000000000000000000000
111111111111111111111111111111
000000000000000000000000000000
111111111111111111111111111111
000
111
000000000000000000000000000000
111111111111111111111111111111
000
111
000
111
11001100
000
111
000
111
000
111
000
111
000
111
000
111
1
80
11001100
111
000
000
111
000
111
7
5
100
111
000
000
111
000
111
000
111
000
111
0
0
100
200
300
400
500
Epoch
600
700
800
900
1000
Figure 4: The learning curves for the regular and irregular past tense forms in the intact network and with a
lesioned hidden layer.
Initially, with few hidden units, lesioning the hidden layer did not lead to a strong decrease in the network performance: with or without the hidden layer,
initially about 20% of the irregular and 60% of the
regular past tense forms were produced correctly. As
the hidden layer grew, however, lesioning lead to a
marked decrease in performance, and at around epoch
200, when the network had constructed 91 hidden
units, deletion of these units resulted in no more irregulars and only 7.2% of the regular verbs to be produced correctly. This confirmed the collaboration of
the two pathways, direct input-output and via the hidden units, in producing the past tense forms of most
verbs, and it showed that even the representations of
initially correct verbs were transferred from the direct
connections into the growing hidden layer, leading in
many cases to the temporary wrong production of initially correct past tense forms. The internal reorganization of the network due to a constructivist adaptation
of its structure could thus account for the unlearning
of initially correct outputs and for the U-shaped learning curve in the acquisition of the English past tense.
7. Generalization
The network was also tested on its generalization to
pseudo-verbs. As Ling & Marinov (1993) pointed out,
testing the generalization ability of a model on existing verbs is misleading because irregular verbs are
by their nature unpredictable, and in line with Ling
& Marinov (1993) we therefore used the set of 60
pseudo-verbs which had been devised by Prasada &
Pinker (1993) and had been tested by them on human
subjects. These verbs consisted of blocks of ten which
were prototypical, intermediate and distant with respect to existing regular and irregular verbs.
The results of the generalization experiments are
shown in figure 5. The generalization performance of
the constructivist network was similar to that of human subjects for both regular and irregular cases. It
performed similar to the SPA, but better than the R&M
network model.
8. Discussion
The experiments reported here show empirically that
a constructivist neural network can model the acquisition of the English past tense more closely than other,
fixed-architecture networks. This is due to the fact that
a constructivist network is capable of adding structure
when and where needed, thereby adapting to the specific learning task, and to the resulting internal reorganization of representations which lead to the U-shaped
development which is also found in children’s learning. These results, together with those from learning theory (see section 2), indicate that constructivist
learning is superior to learning in fixed-architecture
systems. In fact this is also true for the symbolic
SPA: this model builds a decision tree in response
to the learning task and therefore also constitutes a
constructivist system. It is likely that the fact that
the SPA outperformed both R&M’s and M&L’s neural
network model is not based on it being symbolic, but
is due to its constructivist nature. It seems therefore
that the dichotomy constructivist/fixed-architecture is
more fundamental than the symbolic/subsymbolic distinction which previous past tense models have aimed
to emphasize. Direct comparisons between symbolic
and subsymbolic models can thus only be made fully
within or without the constructivist framework, and,
as seen in this paper, models within the constructivist
framework conform better to evidence from neural
and cognitive development.
Comparing the constructivist network with the constructivist symbolic SPA indicates, however, that the
network constitutes a more realistic psychological
model: it both learns better than the SPA and it explains the U-shaped learning curve more realistically.
In the SPA, U-shaped learning was achieved by the ex-
plicit manipulation of a learning parameter that controlled how many times a verb had to be seen to be
memorized as an exception – if it occurred less often, it was overregularized. Besides “hard-wiring” the
theory that children possess such a variable parameter and using the resulting U-shaped learning curve
as evidence for just that theory, leading to a circular
argument, this procedure also established an unrealistically direct relationship between the frequency of a
verb and its overregularization. In the constructivist
network, however, U-shaped learning arose as a direct
outcome of the learning algorithm due to the internal
reorganization of the network architecture.
The constructivist network contradicts the view that
connectionist learning implies a homogeneous architecture, which is often held for connectionist past
tense models. Although learning was based, as in conventional fixed-architecture networks, on the complex
interactions of many simple units and on the gradual adjustment of connection weights, the constructivist network developed a “pseudo-modular” architecture where more space was given to the harder,
irregular cases, and where a memory in the form of
hidden unit receptive fields developed in addition to
the direct input-output connections. Goebel & Indefrey (in press) and Westermann & Goebel (1995)
had shown how learning in (fixed architecture) modular connectionist networks modeled cognitive development more closely than homogeneous architectures,
and the present paper shows how a similar modular architecture can develop in a constructivist framework.
The results obtained with the present and with
previous past tense models thus suggest an extension of the common symbolic/connectionist distinction by the dimensions of modular/homogeneous and
fixed-architecture/constructivist. Given this threedimensional classification matrix, the present paper
indicates that connectionist modular constructivist
systems constitute the most realistic models of cognitive development in the child.
Future work will address an extension to the SGNG
algorithm: in its present form, it only learns to discriminate between similar inputs requiring different
outputs (such as hear and fear) but has in its hidden layer no mechanism of integrating different inputs requiring similar outputs, such as note and decide. Further research will also involve assessment
of the neurobiological plausibility of the constructivist
growth process and its modification to that end.
Such research might then contribute to the understanding of the connection between neural and cognitive development, an area which is currently only
beginning to be addressed.
9. Acknowledgements
This research was supported by the ESRC (award
no. R00429624342) and by the Gottlieb Daimler-und
Karl Benz-Stiftung (grant no. 02.95.29).
References
Baum, E. B. (1989), ‘A proposal for more powerful learning
algorithms’, Neural Computation 1, 201–207.
Fields, R. D. & Nelson, P. G. (1992), ‘Activity-dependent
development of the vertebrate nervous system’, International Review of Neurobiology 34, 133–214.
Fodor, J. (1980), Fixation of belief and concept acquisition, in M. Piattelli-Palmarini, ed., ‘On Language and
Learning: The Debate between Jean Piaget and Noam
Chomsky’, Routledge & Kegan Paul, London and Henley, pp. 143–149.
Fritzke, B. (1994), ‘Fast learning with incremental RBF networks’, Neural Processing Letters 1, 2–5.
Goebel, R. & Indefrey, P. (in press), The performance of
a recurrent network with short term memory capacity
learning the German s-plural, in P. Broeder & J. Murre,
eds, ‘Cognitive Models of Language Acquisition’, MITPress, Cambridge, MA.
Johnson, M. H. (1997), Developmental Cognitive Neuroscience, Blackwell, Oxford, UK; Cambridge, MA.
Ling, C. X. & Marinov, M. (1993), ‘Answering the connectionist challenge: A symbolic model of learning the past
tenses of English verbs’, Cognition 49, 235–290.
MacWhinney, B. & Leinbach, J. (1991), ‘Implementations
are not conceptualizations: Revising the verb learning
model’, Cognition 40, 121–157.
Marcus, G. F., Pinker, S., Ullman, M., Hollander, M., Rosen,
T. J. & Xu, F. (1992), ‘Overregularization in language
acquisition’, Monographs of the Society for Research in
Child Development, Serial No. 228, Vol. 57, No. 4.
O’Leary, D. D. M. (1989), ‘Do cortical areas emerge from a
protocortex?’, Trends in Neuroscience 12, 400–406.
Pinker, S. (1991), ‘Rules of language’, Science 253, 530–
535.
Plunkett, K. & Marchman, V. (1993), ‘From rote learning to
system building: Acquiring verb morphology in children
and connectionist nets’, Cognition 48, 21–69.
Prasada, S. & Pinker, S. (1993), ‘Generalization of regular and irregular morphological patterns’, Language and
Cognitive Processes 8(1), 1–56.
Quartz, S. R. (1993), ‘Neural networks, nativism, and the
plausibility of constructivism’, Cognition 48, 223–242.
Quartz, S. R. & Sejnowski, T. J. (1998), ‘The neural basis of
cognitive development: A constructivist manifesto’, Behavioral and Brain Sciences 21.
Rumelhart, D. E. & McClelland, J. L. (1986), On learning past tenses of English verbs, in D. E. Rumelhart &
J. L. McClelland, eds, ‘Parallel Distributed Processing,
Vol. 2’, MIT Press, Cambridge, MA, pp. 216–271.
Uylings, H. B. M., Kuypers, K., Diamond, M. C. & Veltman,
W. A. M. (1978), ‘Effects of differential environments on
plasticity of dendrites of cortical pyramidal neurons in
adult rats’, Experimental Neurology 62, 658–677.
Van Ooyen, A. (1994), ‘Activity-dependent neural network
development’, Network 5, 401–423.
Westermann, G. & Goebel, R. (1995), Connectionist rules of
language, in ‘Proceedings of the 17th Annual Conference
of the Cognitive Science Society’, Erlbaum, pp. 236–241.