American Journal of Psychology and Behavioral Sciences
2015; 2(2): 33-40
Published online March 10, 2015 (http://www.openscienceonline.com/journal/ajpbs)
A Review of Computational Classical Conditioning
Models
Mehmet Emin Tagluk1, Omer Faruk Ertugrul2, *
1
2
Department of Electrical and Electronic Engineering, Inonu University, Malatya, Turkey
Department of Electrical and Electronic Engineering, Batman University, Batman, Turkey
Email address
[email protected] (M. E. Tagluk), [email protected] (O. F. Ertugrul)
To cite this article
Mehmet Emin Tagluk, Omer Faruk Ertugrul. A Review of Computational Classical Conditioning Models. American Journal of Psychology and
Behavioral Sciences. Vol. 2, No. 2, 2015, pp. 33-40.
Abstract
Classical conditioning (CC), which is a basic learning phenomenon, used to explain basic emotions, such as fear, phobia, and
reflexes. It was introduced by Pavlov in 1927, and since then it has been investigated in psychological, behavioral, memory,
neuroscience, and neurobiology perspectives. It has been used for investigating consumer behavior, conditional fear acquisition
and extinction, response extinction and robotic control. Additionally, a large number of methods were proposed to model the CC
learning stage. Unfortunately, none of them could model all outcomes of CC. In this study, these computational models, their
usages and also the papers about their comparisons are reviewed. It obvious from this review that there is a high requirement to a
model, which has a capability to model each feature of CC.
Keywords
Classical Conditioning, Pavlov, Computational Model, Behavioral Learning
1. Introduction
The classical conditioning (CC) is at the heart of our
understanding of human learning. CC, which was first
demonstrated experimentally by Pavlov in 1927 [1] named as
a conditioned reflex, is based on the association of two stimuli;
a stimulus (S), which evokes either no or a weak response,
usually unrelated to the response that eventually will be
learned, and an unconditioned stimulus (US), which
consistent a response called the unconditioned response (UR)
[2].
In CC, a temporal association between S and US is learned
[4] by presenting the S before the US. After a successful
conditioning, a specific response R will be observed whatever
the presence or absence of the US [5] and the probability to
observe and the strength of CR increases over multiple
training sessions [2], i.e. can be defined as been strengthening
the associative link [6]. Finally, an association is formed
between S and US as a result of the time series pairing (S, US)
and S is termed as CS, anymore. Similarly, this association can
be weaker while having CS with non-occurrence of the
expected US followed for a long time [7]. Additionally,
behavioral learning theories, in which learning is defined as a
change in behavior that occurs as a result of experience, are
based on findings about CC.
The major outcome of CC is about the main knowledge
about human. According to Descartes and the most of other
thinkers in the past several centuries, the human body is a
marvelous machine and all of its functions or properties are
unknown. Therefore they only interested in mind. On the other
hand, Pavlov [1] showed that body is a living machine not
only a machine or robot, which being touched in the world, by
experiments, and this is similar to Aristotle view. For example,
Miguez et al. reviewed the results of stress, and they showed
that it can produce conditioned analgesia or conditioned
hyperalgesia, which can be explained by CC [8], Lindquist et
al. investigates the effect of ethanol in CC delay time by using
the results of experiments on rats [9] and Dalla and Shors and
Brom et al. reviewed the sex differences in learning processes
[10, 11]. These examples showed that, the human body is a
living machine and his learning is also depending on the
physical properties of him.
CC is dealing with learned reflex; therefore it becomes a
very important issue to understand human or animal behavior,
to control behaviors or emotions of them. Classical
conditioning can be seen as a basic learning scheme; the
learning experiments are sometimes done by human subject
[12, 13, 14], but, generally with animals [13, 15, 16, 17, 18,
American Journal of Psychology and Behavioral Sciences 2015; 2(2): 33-40
19]. Additionally, it is important to recognize that the
performance of animals in a conditioning experiment based on
the memory and emotion processes of that animal. Therefore
the results of experiments provide a rich knowledge that can
be used for studying basic learning, memory, and emotion
processes in animals.
The past decade, researchers have shown an increased
interest in CC and they investigated this stage in various
perspectives, such as: psychological [20], behavioral [21],
long and short term memory [22], neuroscience behind CC
[23], neurobiology [20, 21, 24], the role of the Cerebellum in
CC [25], CC of motor responses [26], the activity of EEG
signals in theta band during CC [18, 27], biological neurons [2,
7, 28, 29, 30] and also in Pulsed Neural Chip [31, 32].
Drew et al. investigate the effect of psychological operant
and classical on Central Pattern Generator Neural Circuit
modeled by Hodgkin-Huxley. They reported that the
simulations failed to fully reproduce empirical observations
[28]. Moustafa et al. proposed a biologically plausible model,
which is able to simulate the behavioral effects: including
latent
inhibition,
acquired
equivalence,
sensory
preconditioning, negative patterning, context shift effects, the
effect of the number of training trials on blocking and
overshadowing [30]. Magri et al. improved the biological
neural network of McGregor to assess the psychological
classical conditioning in biological network [33].
As a summary, the CC cannot explain the human thoughts;
it is dealing with the emotional responses [34], reflexes [35],
phobia and superstitions [36]. New CR cannot be produced by
CC, only the probability of occurrence of a CR and its strength
can be changed [34]. However, the researchers were showed
that a S pairing cannot be created for all types of S, it depends
on the requirements of the organism [37]. The strength of R
and more permanent pairings is correlated with the
requirements and expectation of the organism [34, 37].
Additionally, the CC learning also depends to experiences of
organism [38] and the statistical dependency of S [39].
The purpose of this paper is to review the researches,
applications about CC. This paper has been divided into five
parts. The first part deals with the CC learning stage, the
second part is about its applications, the third one is about the
proposed computational models, the fourth part reviewed the
comparison papers and the last one concluded the paper.
2. Applications of CC
There is a large volume of published studies describing the
role of CC in consumer behavior [40, 41, 42, 43], network
security by dynamic policy-based management [44, 45],
robotic control [5, 7, 29], response extinction in rats [46] and
conditional fear acquisition and extinction [15, 16, 19, 47, 48].
Additionally, the effect of CC in obesity is also investigated
[49].
Allen and Madden (1985) reported that classical
conditioning is an important tool for understanding and
producing advertising effects, and their study is a general
review of the theories of classical conditioning in consumer
34
behavior [40]. These types of consumer studies mentioned
that the consumer behavior can be manipulated by CC and
experiments were evaluated on this [41, 42, 43]. Guoshi et al.
developed a biophysical network model of the lateral
amygdala neurons to investigate the underlying mechanisms
for acquisition and extinction of conditioned fear. They used
Hodgkin-Huxley formalism to model neurons and Hebbian
type synaptic plasticity to model the learning process. They
reported that the network model, they proposed, can replicate
the neuronal behaviors successfully [47]. Additionally, several
attempts have been made to investigate fear conditioning by
using human subjects [14, 49] and animal experiments;
Yoshida et al. used goldfish by using fear-related classical
heart rate in their experiments [15], Yamada made his
experiments on mice [16] and Baldi and Bucherelli used rat
subjects in their experiment [19]. In 2007, Erasmus
demonstrated CC by using RealNeuron networks based on
associative learning. They reported that the RealNeuron
networks are simple and can be easily implemented to
standard discreet-logic integrated circuits [7]. Salotti and
Lepretre presented that the results of CC in robot training are
still very far from what can be obtained with animals. They
proposed a new model that integrates both classical and
operant conditioning mechanisms, which may be a source of
inspiration to control the basic interaction mechanisms of
robots [5]. Lehmann showed the applicability of CC by the
Edinburgh Classical Conditioning Pulsed Neural Chip that
implements a drive-reinforcement learning algorithm for
continuous-time pulsed networks with nonlinear synapses. He
reported that the circuit shows good results [31, 32]. However,
a major problem with this kind of application is that these
applications cannot model each particular feature of the CC,
because of the proposed methods.
3. Computational Models
Furze analyzed the machine learning applications of CC in
his PhD thesis [50]. In [51], Ludvig et al. presented the
reinforcement learning methods. Dawson published a book
about connectionism in CC [52]. Additionally, Schmajuk
published two books about mechanisms and computational
methods in CC [53, 54]. Furthermore, there is a large and
growing literature in CC, but unfortunately, there is not any
proposed method that can explain all features of CC yet [55],
e.g. spontaneous recovery. One observer, Balkenius and
Morén, have already drawn attention to that, the CC can be
seen a very simple and basic phenomena, but unfortunately,
there is not any model that capable of explaining all parts of
CC for the last century. Therefore, they pointed out that the
CC is one of the hardest questions in learning [55]. Lack of
enough experimental results because of the ethical rules, has
existed as a main reason of this problem, but which may have
important outcomes, for instance, Chester defined classical
conditioning as a form of temporal learning that may be useful
in intelligent control [56]. Several studies modeling CC have
been carried out, such as: LA Pyramidal Cell based on the
Hebbian learning method [47], Bayesian method [57, 58],
35
Mehmet Emin Tagluk and Omer Faruk Ertugrul:
A Review of Computational Classical Conditioning Models
Hidden Markov method [59], Bayesian confidence
propagating neural networks (BCPNNs) that employed
Hebbian learning [60] and artificial neural network based
methods [4, 44, 45, 61].
Gershman and Niv presented a Bayesian based method,
which is based on the idea of that animals learn the hidden
structure of the environment from their experience and they
employ this internal model of the environment to make
predictions about unobserved or future variables [58].
Courville et al. proposed a framework based on Bayesian
model to explain how animals cope with uncertainty about
contingencies in CC including acquisition, negative and
positive patterning, and forward and backward blocking [57].
Austermann and Yamada demonstrated the CC learning stage
by using a robot that can learn by rewards. In this method,
there is a cooperative learning through the robot and its user
[59].
Additionally, some special computational models were
developed to define the CC stage [62], which can be
categorized into two parts: reinforcement and artificial neural
network based approaches, and they were explained in next
section briefly.
3.1. Reinforcement Learning Based Models
The popular reinforcement learning based models [51] are
Sutton-Barto (SB) [63], SOP [64], TD [65], DR [66],
Balkenius Modeli (BM) [67] and CP [68] models [6, 55]. The
output in reinforcement learning method is calculated as
follows:
the weight adjustments are:
∆
∑
=
=
(1)
̅
,
Table 1. Eligibility trace ( ̅ ).
̅
+1 =
̅
DR
CP
SOP
̅
=
̅
&
+1 =
̅
̅
̅
+1 =
+1 =
+1 =
+1 =
̅
|
#&
#
+
̅
̅
+
+
|
−
−
−
+% 1− ̅
+ 1−
̅
−
−&
−1 ,
−1 ,
>0
≤0
∗
+%
̅
+1 =
+1 =
+1 =
#,
,
̅
#&
∗
∗
∗
+1 −
−
+
−
−1
−
+% 1− ̅
+ %# 1 − ̅
−1
∗
−1
−1 ∗
−&
−&
(7)
(8)
−1
(9)
(10)
%, % and %# are learning coefficients inside
3.2. Artificial Neural Network Based Models
The popular artificial neural network based model is Klopf
[69] model. The other models are policy management model
[PM] [44, 45], adaptive threshold learning model (ATL) [4,
70], and our model, ET [71]. The equations are listed in Table
3.
Table 3. Artificial Neural Network Based Methods
(
Equation
Klop
f
∆
=
=∆
* +=
PM
(
|
−)
− |∆
, -. * + + ,/0 1.
∆, = ∗ 2 ∗ * + ∗ * +
− 4
1−
0< − 4
26
2 =3
− 4
1.5 −
6< −
6
+∆
.
−<
;
=
+:
2;
.
−<
< +∆
.
−<
.
=<
+ =
2= .
−<
.
?@A ?BA
=> 1−%
?@A ?BA -.
.
= tanh ?@A
+%
−
OP N = O N +
N =
Q
N−∆
O N+
4
.
.
(12)
−<
−<
−<
>0
−<
<0
-.
+ ?@A 1.
− ∆ C.
.M N ∗RO N
+
100S
(11)
≤6
?BA -. + ?@A
+ I 4JK ∆L . − 1
.M N = . N − . N − ∆
ET
(6)
̅
&
where ,
0 and 1.
(4)
(5)
̅
SOP
ATL
− −1
̅
CP
−1 +
+%
TD
(3)
−
−
SB /
DR
Equation
SB /
TD
).
Equation
(2)
where,
denotes weights, ∆ , shows weight adjustment,
is input, ̅ denotes eligibility trace of
,
is
reinforcement term which is depend on the relation between
reinforcement output ( ) and real output ( ) [6]. The
eligibility trace and reinforcement terms are different in each
method and they are summarized in Table 1 and 2.
,
Table 2. Reinforcement Term (
>0
<0
(13)
1.
(14)
is learning coefficient in Klopf model [69]. T is cache
cycle, j is time instant, US is the initial value (x0), CS is input
(xi) and c is a small coefficient [44, 45]. ; , 2; , = , 2= are
American Journal of Psychology and Behavioral Sciences 2015; 2(2): 33-40
learning coefficients, % forgetting coefficient
shows
eligibility, Hi is the threshold and Si is output in the ATL model
[4]. In ATL, both threshold values and weights are updated.
The basic properties of CC, such as: delay and trace
conditioning, different ISI effects, blocking, overshadowing,
and compound stimulus, were demonstrated in their study [4].
The (0 < < 1) and (0 < < 1) are association and
extinction coefficients [71].
In [44, 45] the typical network security cases are presented
to simulate how network management policies, which are
described in this study as a conditioned reflex, for a dynamic
learning process based on classical conditioning. They
simulated the acquisition, extinction, reacquisition, secondary
conditioning phenomena in network security policies.
Therefore the proposed approach is dynamic and it can be
employed for self-management, which is a better policy
scheme than using static rules. The ATL method has the ability
of modeling trace and delay conditioning, acquisition,
extinction, reaquisition, blocking, conditioned inhibition,
secondary conditioning and facilitation, but it cannot model
spontaneous recovery [71].
36
Table 4. Comparison of CC models [55]
Feature
Trace
Conditioning
Delay
Conditioning
ISI Curve
S Shaped
Acquisition
Extinction
Reacquisition
Blocking
Secondary
Conditioning
Spontaneous
Recovery
Conditioned
Inhibition
Facilitation
SB
TD
Klopf
Balkenius
SD
+
+
+
+
+
-
+
±
+
+
-
+
-
±
+
±
+
+
-
+
+
+
+
+
±
+
+
+
+
+
+
±
±
+
+
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
Malaka and Hammer reviewed real CC models;
Sutton/Barto (1981), SOP (Wagner, 198l), temporal difference
(Sutton & Barto, 1990), the drive-reinforcement (Klopf, 1989)
and CP (Malaka, 1995). The results of their comparison are
sorted in Table 5 [6].
4. Comparison of CC Models
Table 5. Comparison of CC models [6]
Chester compared Sutton-Barto (1981), Klopf (1989) and
Grossberg-Schmajuk (1989) models in his study. The models
he assessed are based on Hebbian learning, but they differ in
how events are remembered. He reported that the models he
compared fall far short of being good learners of temporal
associations. Sutton-Barto, Klopf and the Hebbian models
associate CS with R without the US and the Klopf model is a
better associator than the Sutton- Barto model. Also, he
presented that the Grossberg- Schmajuk model is more suited
to learning temporal associations than the others, on the other
hand, it cannot be respond with a sharp memory of a transient
event and recover quickly after the situation has returned to
normal [57].
Cheung et al. demonstrated the real time models of CC,
such as; the unsupervised spatio-temporal single-neuron
models of Sutton-Barto (1981, 1982), Rescorla-Wagner
(1972), and Klopf (1988, 1991). They reported that Klopf
model can be used for demonstrating CC features such as
delay and trace conditioning, inter-stimulus interval effects,
second-order conditioning, conditioned inhibition, extinction,
reacquisition,
backward
conditioning,
blocking,
overshadowing, compound conditioning, and discriminatory
stimulus effects [70].
Balkenius and Moren compared the most popular CC
models, which are Sutton-Barto (1981), temporal difference
(Sutton-Barto, 1987), Klopf (Drive-Reinforcement, 1982),
Balkenius (1995, 1996, 1998), Schmajuk-DiCarlo (1992)
models [72]. These comparisons are based on the applicability
of the CC features, such as: acquisition, inter-stimulus interval
effects, extinction, reacquisition, blocking, conditioned
inhibition and secondary conditioning. The results of their
comparison are sorted in Table 4 [55].
Positive Forward
Conditioning
Inhibition Backward
Conditioning without
Background
Inhibition Backward
Conditioning with
Background
Extinction
Blocking
Delay Conditioning
Conditioned Inhibition
Unpaired Inhibition
Second Order
Conditioning
Complete Serial
Compound
Conditioning
SB
DR
TD
CP
SOP
CP/
TD
+
+
+
+
+
+
+
+
-
-
+
+
+
+
+
-
+
+
+
+
+
-
+
+
+
+
-
+
+
±
+
-
+
+
+
+
-
+
+
±
+
±
+
+
+
+
±
+
±
+
+
-
+
-
-
+
+
+
+
Additionally, Vogel et al. [73] described the quantitative
models of Pavlovian conditioning and Malaka [74] analyzed
CC models, such as: Sutton and Barto (1981),
Drive-reinforcement (Klopf, 1989; Klopf and Morgan, 1990),
Temporal difference (TD) models (Sutton and Barto, 1990),
Sometimes opponent processes (SOP) (Wagner, 1981;
Donegan et al., 1989; Wagner and Donegan, 1989) and CP
model (Malaka et al., 1995a, b; Malaka and Hammer, 1996)
with respect to their mathematical basis. The comparison
results that he obtained were summarized in Table 6 and 7
[74].
37
Mehmet Emin Tagluk and Omer Faruk Ertugrul:
A Review of Computational Classical Conditioning Models
Table 6. Eligibility term [74]
Conditioning paradigm
Positive Forward Conditioning
Inhib. backward conditioning (IBC)
IBC with backgroundb
Conditioned inhibition
Convergence of multi-trial learning
Monotonicc ∆wi (xi )
Extinction (short CS)
Extinction of long CSs
and
Extinction of CI possibled
Non-extinction of CI possibled
Unpaired inhibitione
Delay-conditioning (DC) ineffective for long CS
One-trial effectiveness in DCg
Discrimination/generalization proportional to CS overlaps
Differential conditioning
Compound conditioningh
Over shadowingh
Second-order conditioning
Blocking
CSD prediction
CSC prediction
SB/TD
+
+
+
+
+b
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
where a Only for a short range of ISIs possible, b multi-trial
learning can be described by trial-based models, c highest xi
lead to largest weight changes, d extinction and non-extinction
of conditioned inhibitors depend on the restriction of y in the
reinforcement term, e is modeled as inhib. backward
DR
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
SOP
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
±
+
+
CP
+
+
+
+
+b
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
CP-CP
+
+
+
+
+b
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
CP+SB
+
+
+
+
+b
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
conditioning with long negative ISI, f only possible with
background and appropriate eligibility, g for long CSs, the
learning result after multiple trials is the same as after one trial
and h depends on stimulus overlaps and initial weights, with
SOP eligibility additionally ISI-dependent.
Table 7. Reinforcement Term [74]
Conditioning paradigm
Positive Forward Conditioning
Inhib. backward conditioning (IBC)
IBC with backgroundb
Conditioned inhibition
Convergence of multi-trial learning
Monotonicc ∆wi (xi )
Extinction (short CS)
Extinction of long CSs
and
Extinction of CI possibled
Non-extinction of CI possibled
Unpaired inhibitione
Delay-conditioning (DC) ineffective for long CS
One-trial effectiveness in DCg
Discrimination/generalization proportional to CS overlaps
Differential conditioning
Compound conditioningh
Overshadowingh
Second-order conditioning
Blocking
CSD prediction
CSC prediction
SB/TD
+
+-a
+-a
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
where a Only for a short range of ISIs possible, b multi-trial
learning can be described by trial-based models, c highest xi
lead to largest weight changes, d extinction and non-extinction
of conditioned inhibitors depend on the restriction of y in the
reinforcement term, e is modeled as inhib. backward
conditioning with long negative ISI, f only possible with
DR
+
±f
+
+
+
+
+
+
+
+
±f
+
+
+
+
+
+
+
+
+
+
SOP
+
+
+
+
+
+
+
+
+
+
+
+
+
+
±
+
+
CP
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
CP-CP
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
±
-
CP+SB
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
background and appropriate eligibility, g for long CSs, the
learning result after multiple trials is the same as after one trial
and h depends on stimulus overlaps and initial weights, with
SOP eligibility additionally ISI-dependent.
As seen, all the previously mentioned methods suffer from
some serious limitations as can be seen in Table 4-7. None of
American Journal of Psychology and Behavioral Sciences 2015; 2(2): 33-40
them can provide all features of CC. There maybe two
possible reasons for this situation. One of them is lack of
experimental data. The other one is, it may be because of
deficiencies in theory of CC. This paper may be used by
researchers to determine which model will be more suitable
for their studies.
5. Conclusion
This study set out with the aim of reviewing computational
models of CC, and its applications. The CC, which explains
human body as a living machine, is an important phenomenon
in the human learning that explains basic behaviors, such as:
fear, reflex, emotions and phobia. The CC learning stage is
analyzed in various areas because the results of experiments
provide a rich knowledge that can be used for studying basic
learning, memory, and emotion processes. Basically, it was
used for investigating consumer behavior, network security,
robotic control, response extinction, conditional fear
acquisition and extinction and obesity. As a summary, there is
not any model that capable of explaining all outcomes of CC,
which can be seen that the hardest questions in learning. As
seen, all the previously mentioned methods suffer from some
serious limitations as can be seen in Table 4-7. This paper may
be used by researchers to determine which model will be more
suitable for their studies.
References
[1]
Pavlov,
Ivan
Petrovich.
DoverPublications. com, 1927.
[2]
Martin Inderbitzin, Ivan Herreros-Alonso and Paul F.M.J.
Verschure, “An Integrated Computational Model of the Two
Phase Theory of Classical Conditioning”, IEEE, 2010
[3]
Dugatkin, LA., “Principles of animal behavior” New York,
New York, USA: WW Norton, 2004.
[4]
Raja L. Clouse, Soowon Kim, and Manjula B. Waldron, “An
Adaptive Threshold Learning Algorithm for Classical
Conditioning”, Proceedings - 19th International Conference IEEE/EMBS Oct. 30 - Nov. 2, 1997 Chicago, IL. USA, pg.
1380-1382.
[5]
[6]
Conditioned
reflexes.
Salotti, Jean Marc, and Florent Lepretre. "Classical and
Operant Conditioning as Roots of Interaction for Robots."
Proceedings of the Workshop from Motor to Interaction
Learning in Robots, Conference on Intelligent Robotics
Systems, Nice. 2008.
Malaka, Rainer, and Martin Hammer. "Real-time models of
classical conditioning." Neural Networks, 1996, IEEE
International Conference on. Vol. 2. IEEE, 1996.
[7]
Louwrence
D
Erasmus,
“Classical
Conditioning
Implementation with RealNeuron Networks”, IEEE, 2007.
[8]
Gonzalo Miguez, Mario A. Laborda and Ralph R. Miller,
“Classical conditioning and pain: Conditioned analgesia and
hyperalgesia”, Acta Psychologica 145 (2014) 10–20.
[9]
Derick H. Lindquist, Greta Sokoloff, Joseph E. Steinmetz,
“Ethanol-exposed neonatal rats are impaired as adults in
38
classical eyeblink conditioning at multiple unconditioned
stimulus intensities”, Brain Research 1150 (2007) 155 – 166.
[10] Christina Dalla, Tracey J. Shors, “Sex differences in learning
processes of classical and operant conditioning”, Physiology &
Behavior 97 (2009) 229–238.
[11] Mirte Brom, Stephanie Both, Ellen Laan,Walter Everaerd,
Philip Spinhoven, “The role of conditioning, learning and
dopamine in sexual behavior: A narrative review of animal and
human studies”, Neuroscience and Biobehavioral Reviews 38
(2014) 38–59.
[12] Ole Åsli, Magne Arve Flaten, “Conditioned facilitation of the
unconditioned reflex after classical eyeblink conditioning”,
International Journal of Psychophysiology 67 (2008) 17–22.
[13] Tomer Shechner, Melanie Hong, Jennifer C. Britton, Daniel S.
Pine, Nathan A. Fox, “Fear conditioning and extinction across
development: Evidence from human studies and animal
models”, Biological Psychology 100 (2014) 1–12.
[14] Bram Vervliet, Maarten Geens, “Fear generalization in humans:
Impact of feature learning on conditioning and extinction”,
Neurobiology of Learning and Memory 113 (2014) 143–148.
[15] Masayuki Yoshida, Izumi Okamura, Kazumasa Uematsu,
“Involvement of the cerebellum in classical fear conditioning
in goldfish”, Behavioural Brain Research, 153 (2004) 143–148.
[16] Kazuyuki Yamada, “Strain differences of selective attention in
mice: Effect of Kamin blocking on classical fear conditioning”,
Behavioural Brain Research 213 (2010) 126–129.
[17] Thomas M. Ricarta, Xilu Jiao, Kevin C.H. Pang, Kevin D.
Beck, Richard. J. Servatius, “Classical and instrumental
conditioning of eyeblink responses inWistar–Kyoto and
Sprague–Dawley rats”, Behavioural Brain Research 216 (2011)
414–418.
[18] Miriam S. Nokia, Markku Penttonen, Tapani Korhonen, Jan
Wikgren, “Hippocampal theta (3–8 Hz) activity during
classical eyeblink conditioning in rabbits”, Neurobiology of
Learning and Memory 90 (2008) 62–70.
[19] Elisabetta Baldi, Corrado Bucherelli, “Entorhinal cortex
contribution to contextual fear conditioning extinction and
reconsolidation in rats”, Neurobiology of Learning and
Memory 110 (2014) 64–71.
[20] Andrew R. Delamater, R. Frederick Westbrook, “Psychological
and neural mechanisms of experimental extinction: A selective
review”, Neurobiology of Learning and Memory 108 (2014)
38–51.
[21] Travis P. Todd, Drina Vurbic, Mark E. Bouton, “Behavioral and
neurobiological mechanisms of extinction in Pavlovian and
instrumental learning”, Neurobiology of Learning and Memory
108 (2014) 52–64.
[22] M.E. Bouton, E.W. Moody, “Memory processes in classical
conditioning”, Neuroscience and Biobehavioral Reviews 28
(2004) 663–674.
[23] Luis Aguado, “Neuroscience of Pavlovian Conditioning: A
Brief Review”, The Spanish Journal of Psychology , 2003, Vol.
6, No. 2, 155-167.
[24] Jonathan D. Raybuck, K. Matthew Lattal, “Bridging the
interval: Theory and neurobiology of trace conditioning”,
Behavioural Processes 101 (2014) 103–111.
39
Mehmet Emin Tagluk and Omer Faruk Ertugrul:
A Review of Computational Classical Conditioning Models
[25] R. F. Thompson and J. E. Steinmetz, “The Role of the
Cerebellum in Classical Conditioning of Discrete Behavioral
Responses”, Neuroscience 162 (2009) 732–755.
[40] Chris T. Allen, Thomas J. Madden, “A Closer Look at Classical
Conditioning”, The Journal of Consumer Research, Vol. 12, No:
3, December 1985, pp. 301-315.
[26] Germund Hesslow, Dan-Anders Jirenhed, Anders Rasmussen,
Fredrik Johansson, “Classical conditioning of motor
responses:What is the learning mechanism?”, Neural Networks
47 (2013) 81–87.
[41] Elnora W. Stuart, Terence A. Shimp, Randall W. Enge,
“Classical Conditioning of Consumer Attitudes: Four
Experiments in an Advertising Context”, Journal of Consumer
Research, Vol. 14, December 1987, pp. 334-349.
[27] Stephen D. Berry, Loren C. Hoffmann, “Hippocampal
theta-dependent eyeblink classical conditioning: Coordination
of a distributed learning system”, Neurobiology of Learning
and Memory 95 (2011) 185–189.
[42] Chanthika Pornpitakpan, “A critical review of classical
conditioning effects on consumer behavior”, Australasian
Marketing Journal 20 (2012) 282–296.
[28] Drew C. Thompson, Irving Z. Basanez, Douglas A. Baxter and
John H. Byrne, “A Computational Study of Operant and
Classical Conditioning in a Central Pattern Generator Neural
Circuit”.
[29] Blackburn, Michael R. "A general architecture for classical
conditioning of perceptual-motor sequences." Neural Networks,
1990., 1990 IJCNN International Joint Conference on. IEEE,
1990.
[30] Ahmed A. Moustafa, Catherine E. Myers, Mark A. Gluck, “A
neurocomputational model of classical conditioning
phenomena: A putative role for the hippocampal region in
associative learning”, Brain Research 1276 (2009) 180 – 195.
[31] Torsten Lehmann, “Classical Conditioning with Pulsed
Integrated Neural Networks: Circuits and System”, IEEE
Transactions on Circuits and Systems—II: Analog and Digital
Signal Processing, 45(6), (1998), 720-728.
[32] Lehmann, Torsten. "Eccopunch: the Edinburgh classical
conditioning pulsed neural chip." Circuits and Systems, 1997.
ISCAS'97., Proceedings of 1997 IEEE International
Symposium on, 1997.
[33] Magri, S., Pepe, I., Traverso, S., & Nicolini, C. (1993, October).
A model of associative learning and classical conditioning in
Aplysia californica: a preliminary study. In Neural Networks,
1993. IJCNN'93-Nagoya. Proceedings of 1993 International
Joint Conference on Vol. 3, pp. 2694-2697, IEEE.
[34] B. Yeşilyaprak, B. Aydın, M. Bilgin, G. Can, K. Ersanlı, M.
Kılıç, İ. Kısaç, İ. Korkmaz, H. Küçükkaragöz, Ş. Küllahoğlu, B.
Öztürk
ve
E.
Uçar,
Eğitim
Psikolojisi
Gelişim-Öğrenme-Öğretim, Ankara: Pegem Akademi, 2012.
[35] R. L. Veznedaroğlu ve A. O. Özgür, “Öğrenme Stilleri:
Tanımlamalar, Modeller ve İşlevleri”, İlköğretim Online, 4(2),
(2005), pp. 1-16.
[36] Olson, M. H., & Hergenhahn, B. R. (2009). An introduction to
theories of learning. Pearson/Prentice Hall.
[37] Garcia, J., Koelling, R. A. (1966). Relation of cue to
consequences in avoidance learning. Psychonomic Science, 4,
123-124.
[38] Kamin, L. J. (1968). Attention–like processes in classical
conditioning. In Jones, M. R. (Ed.) Miami symposium on the
prediction of behavior: aversive stimulation (pp. 9–31). Miami:
University of Miami Press.
[39] Rescorla, R. A. (1968). Probability of shock in the presence and
absence of CS in fear conditioning. Journal of comparative
physiological psychology, 66, 1–5.
[43] Yi Fen Chen ve Jian Han Jiang, “Effects Of Classical and
Operant Conditioning On Online Consumer Purchase and
Repurchase Intention”, Effects Of Classical and Operant
Conditioning On Online Consumer Purchase and Repurchase
Intention.
[44] Suping Liu, Yongsheng Ding, “A Classical Conditioning
Model for Policy-based Management”, 2009 International
Conference on Networks Security, Wireless Communications
and Trusted Computing, DOI 10.1109/NSWCTC.2009.129, pp.
249-252.
[45] Suping Liu, Yongsheng Ding, “An Adaptive Network Policy
Management Framework Based on Classical Conditioning”,
Proceedings of the 7th World Congress on Intelligent Control
and Automation, June 25 - 27, 2008, Chongqing, China, pg.
3336-3340.
[46] R. Prueckl, A.H. Taub, I. Herreros, R. Hogri, A. Magal, S.A.
Bamford, A. Giovannucci, R. Ofek, Y. Shacham-Diamand,
P.F.M.J. Verschure, M. Mintz, J. Scharinger, A. Silmon, C.
Guger, “Behavioral Rehabilitation of the Eye Closure Reflex in
Senescent Rats using a Real-Time Biosignal Acquisition
System”, 33rd Annual International Conference of the IEEE
EMBS, Boston, Massachusetts USA, August 30 - September 3,
2011, Pg. 4211-4214.
[47] Guoshi Li, Gregory J. Quirk, and Satish S. Nair, “Modeling
Acquisition and Extinction of Conditioned Fear in LA Neurons
using Learning Algorithm”, Proceedings of the 2007 American
Control Conference Marriott Marquis Hotel at Times Square
New York City, USA, July 11-13, 2007, Pg. 552-557.
[48] Susan Shi Yuan Li, Gavan P. McNally, “The conditions that
promote fear learning: Prediction error and Pavlovian fear
conditioning”, Neurobiology of Learning and Memory 108
(2014) 14–21.
[49] Sven Benson, Joswin Kattoor, Jennifer S. Kullmann, Sarah
Hofmann, Harald Engler, Michael Forsting, Elke R. Gizewski,
Sigrid Elsenbruch, “Towards understanding sex differences in
visceral pain: Enhanced reactivation of classically-conditioned
fear in healthy women”, Neurobiology of Learning and
Memory 109 (2014) 113–121.
[50] Furze T. A., “The Application of Classical Conditioning to the
Machine Learning of a Commonsense Knowledge of Visual
Events”, PhD Thesis, The University of Leeds, School of
Computing.
[51] Elliot A. Ludvig, Marc G. Bellemare and Keir G. Pearson, “A
Primer on Reinforcement Learning in the Brain: Psychological,
Computational, and Neural Perspectives” in Eduardo Alonso,
Esther Mondragón, “Computational Neuroscience for
Advancing Artificial Intelligence: Models, Methods and
Applications”, Medical Information Science Reference,
Hershey • New York, 2011.
American Journal of Psychology and Behavioral Sciences 2015; 2(2): 33-40
[52] Dawson M. R. W., “Connectionism and Classical
Conditioning”, A peer-reviewed monograph published by
Comparative Cognition and Behavior Reviews on behalf of the
Comparative
Cognition
Society.
2008
doi:
10.3819/ccbr.2008.30008.
[53]
Schmajuk N., “Mechanisms in Classical Conditioning: A
Computational Approach”, Cambridge University Press, The
Edinburgh Building, Cambridge CB2 8RU, UK, 2010.
[54] Schmajuk N., “Computational Models of Conditioning”,
Cambridge University Press, The Edinburgh Building,
Cambridge CB2 8RU, UK, 2010.
[55] Balkenius, Christian, and Jan Morén. "Computational models
of classical conditioning: a comparative study." Proceedings of
the fifth international conference on simulation of adaptive
behavior on From animals to animats. Vol. 5. 1998.
[56] Daniel L. Chester, “A Comparison of some Neural Network
Models of Classical Conditioning”, 1990 IEEE, pp.1163-1168.
[57] A. C. Courville, N. D. Daw, G. J. Gordon ve D. S. Touretzky,
“Model Uncertainty in Classical Conditioning”, Advances in
neural information processing systems. 2003.
[58] Samuel J. Gershman & Yael Niv, “Exploring a latent cause
theory of classical conditioning”, Learn Behav (2012)
40:255–268 DOI 10.3758/s13420-012-0080-8.
[59] Anja Austermann, Seiji Yamada, “Learning to Understand
Multimodal Rewards for Human-Robot-Interaction using
Hidden Markov Models and Classical Conditioning”, 2008
IEEE Congress on Evolutionary Computation (CEC 2008), Pg.
4096-4103.
[60] Christopher Johansson ve Anders Lansner, “An Associative
Neural Network Model of Classical Conditioning”, Report
from Studies of Artificial Neural Systems, (SANS), Numerical
Analysis and Computer Science (Nada), Royal Institute of
Technology (KTH) , S-100 44 STOCKHOLM, Sweden.
theory
of
adaptive
networks:
expectation
prediction." Psychological review 88.2 (1981): 135.
40
and
[64] Wagner, A.R. (1981) SOP: A model of automatic memory
processing in animal behavior. In Spear, N.E. and R.R. Miller,
editors, Informatzon Processing in Animals: Memory
Mechaieisms, volume 85, chapter 1, pages 5-44. Erlbaum, New
Jersey.
[65] Sutton, R.S. and A .G. Barto (1990) Time-derivative models of
Pavlovian reinforcement. In Gabriel, A. and J. Moore, editors,
Learning and Computational Neuroscience: Foundations and
Adaptive Networks, chapter 12, pages 497-537. MIT Press,
Cambridge, Mass and London.
[66] Klopf, A.H. (1989) Classical conditioning: Phenomena
predicted by a drive-reinforcement model of neural function. In
Byrne, J.H. and W.O. Berry, editors, Neural Models of
Placticity: Experimental and Theoretical Approaches, chapter 6,
pages 94-103. Academic Press.
[67] Christian Balkenius, Jan Mor´en, “Dynamics of a Classical
Conditioning Model”, Autonomous Robots 7, 41–56 (1999).
[68] Malaka, R., R. Lange, and M. Hammer (1995) A constant
prediction model for classical conditioning. In Elser, N. and R.
Menzel, editors, Learning and Memory: Proceedings of the
23rd Gottingen Neurobiology Conference, volume 1, page 75.
Thieme-Verlag, Stuttgart, New York.
[69] Klopf, A. H. (1988). A neuronal model of classical
conditioning. 16, 2, 85–125.
[70] John Y. Cheung, David C. Chance, and Asa Lawton, “An
Examination of Real-Time Neuronal Models in a Classical
Conditioning Framework”, 1992 IEEE, Pg. III-504 – III-509.
[71] Ertugrul, O. F., & Tagluk, M. E. (2014, April). Learning with
classical conditioning. In Signal Processing and
Communications Applications Conference (SIU), 2014 22nd
(pp. 927-930). IEEE. doi: 10.1109/SIU.2014.6830382.
[61] H. Hassan Ve M. Watan, “On Mathematical Analysis of
Pavlovian Condltioning Learning Process Using Artificial
Neural Network Model”, 10th Mediterranean Electrotechnical
Conference, MEleCon 2000, Vol. II, Pg. 578-581.
[72] Nestor A. Schmajuk and James J. DiCarlo, “Stimulus
Configuration, Classical Conditioning, and Hippocampal
Function”, Psychological Review, 1992, Vol. 99, No.
2,268-305.
[62] Hiroyuki Ishii, Masaki Nakasuji, Motonori Ogura, Hiroyasu
Miwa, Atsuo Takanishi, “Accelerating Rat’s Learning Speed
Using a Robot - The robot autonomously shows rats its
functions –”, Proceedings of the 2004 IEEE International
Workshop on Robot and Human Interactive Communication,
Kurashiki, Okayama Japan September 20-22,2004.
[73] Edgar H. Vogel, Maria E. Castro, Maria A. Saavedra,
“Quantitative models of Pavlovian conditioning”, Brain
Research Bulletin 63 (2004) 173–202.
[63] Sutton, Richard S., and Andrew G. Barto. "Toward a modern
[74] Rainer Malaka, “Models of Classical Conditioning Models of
Classical Conditioning”, Bulletin of Mathematical Biology
(1999) 61, 33–83, Article No. bulm.1998.0074.