CN510: Principles and Methods of Cognitive and Neural
Modeling
Spike-Timing Dependent Plasticity II
Lecture 14
Instructor: Anatoli Gorchetchnikov <[email protected]>
Our Motivation
The rule should only depend on the information that is
available at the synapse at the time of synaptic
modification
Biological reason: cells are unlikely to have the information
about the internal state of other cells; timing of the recent
events can only be estimated from some processes
triggered by these events
Computational reason: cell or compartment is a likely
candidate for an object in an object oriented neuronal
simulator, and passing the information between objects is
more expensive than within an object
CN 510 Lecture 14
Three Components of STDP
Presynaptic timing
Postsynaptic timing
The efficiency of learning for a certain time difference
All three components of STDP might follow from a single
mechanism (Holmes and Levy, 1990)
We designed the measures of presynaptic timing and
postsynaptic timing, the learning efficiency followed from
this design
CN 510 Lecture 14
Back to Hebb
When an axon of cell A is near enough to excite a cell B and
repeatedly or persistently takes part in firing it, some
growth process or metabolic change takes place in one or
both cells such that A's efficiency, as one of the cells firing
B, is increased
Usually written as: w   x pre x post
Hebb emphasized causality and, therefore, a temporal order
of neuronal firing
Equation is based on correlation and does not include
causality
Unless xpre and xpost are specifically designed to include this
information
CN 510 Lecture 14
Core Idea
w   x pre x post
Which xpre and xpost can include the temporal order of
neuronal firing?
EPSP at the postsynaptic cell is a result of presynaptic firing
and has temporal dynamics, thus it implicitly includes the
timing of presynaptic spike
Membrane potential at the postsynaptic cell obviously
includes the timing of postsynaptic spike
What if we simply multiply the two together?
CN 510 Lecture 14
Assumption 1
Let tpost-tpre=s
Let tpre=0 so s=tpost so at t=0 the presynaptic spike starts to
manifest itself at the synapse
Approximate the effect of presynaptic transmitter release on
synaptic conductance by an alpha function:
t
x pre  g syn  g syn e

 t
1 
 
where g syn is the maximal conductance, t is the time since the
presynaptic action potential, and τ is the time constant of
the channel
Assume that g syn decays completely at t=10τ
CN 510 Lecture 14
Assumption 2
Approximate the postsynaptic action potential with a
piecewise linear function
x post
 A(t  s )  B

 C (t  s )  D
0

B
t s
A
D
st s
C
s
A(t-s)+B models the depolarization part, where A>0 is the
slope of a spike and B>0 is a peak amplitude
C(t-s)+D models the hyperpolarization part, where C>0 is the
slope and D<0 is the trough amplitude
CN 510 Lecture 14
Resulting Rule
Substituting these signals in Hebb's rule yields
 t

t 1 
B
s

t s
A
(
t

s
)

B
e


A


w
 t
D
t 1 

s

t

s

C
 C (t  s )  D   e
Learning only happens if
D
B
 s  10 
C
A
CN 510 Lecture 14
Total Weight Change
Integrate over the length of the learning window
This integral has an analytic solution
e
 t
 1 
 
(  s)(t   )   (t   )
where μ=A, ν=B if
μ=C, ν=D if
B
t s
A
D
st s
C
s
2

  2  X
(potentiation)
(depression)
So the total weight change is w   P t2   D t3
t
1
t
2
where the limits follow from case studies as
w   P
max 0 , min( s ,10 ) 
B

max  0 , s  
A



 D

max  s , min  s  ,10  
 C


D max 0, s 
CN 510 Lecture 14
Plotting
We can plot
w   P
max 0 , min( s ,10 ) 
B

max  0 , s  
A

as a function of s
Note the shift of
zero crossing to
the right
CN 510 Lecture


 D

max  s , min  s  ,10  
 C


D max 0, s 
Limiting the Weights
Hebbian rule lets the resulting synaptic weights to grow
infinitely large
Introduce a decay term proportional to the current weight
w   x pre x post  f  x pre , x post  w
What shape f() should take?
We want to achieve lim w  q x pre x post
t 


The synaptic weight can not be negative if it is defined as a
density of the ion channels in the synapse, so q x pre x post
should be non-negative

CN 510 Lecture 14

t
x pre  g syn  g syn e
 t
1 
 
is
non-negative,
but
xpost
has
negative part 
Replace parameters in our piece-wise linear approximation:
instead of setting them directly from membrane potential
use a bounded function
Set the parameters B and D to normalize the values of xpost
over the interval [D,B] of the length 1
Then xpost changes between D and B, xpost+(–D) is between 0
and 1 and the product xprexpost thus is also in [0,1]
Since –D=1–B we can write it as
q  x pre x post   x pre x post  1  B
in [0,1]
CN 510 Lecture 14
Using
lim w  q  x pre x post   x pre x post  1  B
t 
Leads to
if x pre x post  B or maximal LTP
lim w  0 if x pre x post  B  1  D or maximal LTD
t 
lim w  1  B if x pre x post  0 or no correlation between input
t 
and output
lim w  1
t 
In addition we would like to accommodate the variable spike
duration and to correct the shift in zero crossing
CN 510 Lecture 14
Redesign xpost
x post
B
 A(t  s )  B

 
1
C  t  s  A   D

 
0
st s
1
A
1
D 1
s t  s 
A
C A
A<0, 0<B<1, C>0, D=B-1<0
CN 510 Lecture 14
New STDP Curve
2
New rule leads to w   P t  T
t
1
where T
t3
t2
t3
t
 B e
t2

 t
 1 
 
dt  e
 t
 1 
 
t3
t2
  D t4
t

3
max t * , min  s ,10 

B(t   )

 
min s , max 0 ,t *
Note more balanced
curve and correct zero
crossing
CN 510 Lecture
Extending the Weights
Using
lim w  q  x pre x post   x pre x post  1  B
t 
Leads to
if x pre x post  B or maximal LTP
lim w  0 if x pre x post  B  1  D or maximal LTD
t 
lim w  1  B if x pre x post  0 or no correlation between input
t 
and output
lim w  1
t 
We might want the weights to change between
w instead of between 0 and 1
CN 510 Lecture 14
w
and
Extending the Weights
Multiplying by the interval and adding
w
lim w   x pre x post  1  B   w  w  w
t 
Here

lim w  w if x pre x post  B or maximal LTP
t 

lim w  w if x pre x post  B  1  D or maximal LTD
t 

 
lim w  w  B(w  w)  w0 if x pre x post  0
or no correlation
t 
between input and output
Rewriting in terms of weights
lim w  x pre x post  w  w  w0
t 
gives the equation
w    x pre x post  w  w  w0  w
CN 510 Lecture 14
Final Form
The final rule is
w    x pre x post  w  w  w0  w f  x pre , x post 


The reason the gating function f x pre , x post is outside here is
that we do not want to change the limit values of weights
Note that it will not affect the learning much, because both
xpre and xpost have to be non-zero for learning to happen
and the gating function is usually less restrictive
Also note that the only requirement on f x pre , x post is nonnegativity so it does not change the sign of the weight
change
Aside from minimal, maximal, and baseline weights this rule
only has two parameters: slopes A and C, which can be
measured experimentally

CN 510 Lecture 14

Comparison with (Pfister and Gerstner, 2006)
We use both presynaptic and postsynaptic traces for both
potentiation and depression components
As a result both components are expressed within a single
formula
Our depression component has a lasting effect since we use
alpha function for the synapse rather then delta function for
a spike; it is also slightly delayed
Our potentiation component takes into account spike width in
addition to timing
Both our traces are saturating,
– postsynaptic only takes into account the last spike,
– presynaptic takes into account two spikes in our
simulations
CN 510 Lecture 14
Can We Relate It to BCM?
Our weight dependency is additive, so according to
Izhikevich we would have trouble if we used all-to-all
pairing, but we used (two pre)-to-(one post) pairing
Achieving sliding threshold effect is less straightforward,
there is nothing in the rule that directly manipulates the
postsynaptic rate, so some external mechanism is
necessary
CN 510 Lecture 14
Place Cells
Found by O’Keefe
Dostrowsky, (1971)
and
Firing of a cell is restricted to
an area of space while the
animal navigates through the
environment
Found in hippocampal areas
CA1, CA3 and dentate gyrus
(DG)
Led to cognitive map theory
CN 510 Lecture 14
Place Cells: Phase Precession
Phase precession effect was
discovered by O’Keefe
and Recce (1993), and
extensively described by
Skaggs et al. (1996)
When the animal enters the
place field of the cell, this
cell tend to fire later in the
theta cycle
As the animal moves through
the place field, place cell
fires earlier in the theta
cycle
CN 510 Lecture 14
Grid Cells
Found by Hafting et al., (2005)
Firing of a cell is restricted to
multiple areas of space while
the animal navigates through
the environment
These areas repeat in hexagonal
grid pattern with small (3080cm) periods
Found in all layers of entorhinal
cortex (EC)
CN 510 Lecture 14
23
Spatial Scales of Grid Cells
Periods of the grid (spatial
scales) gradually increase
from dorsal to ventral EC
Within scale there are cells
with various translations
(phases) of the grid
Projections from EC layer II
to DG are topographic:
cells along the dorsoventral axis of EC project
to transverse slices along
septo-temporal axis of DG
CN 510 Lecture 14
Drive for Spectral Spacing
How can small periods of grid cells
lead
to
large
spaces
representations of place cells?
Periods of the grid (spatial scales)
gradually increase from dorsal to
ventral EC
Projections from EC layer II to DG
are topographic: cells along the
dorso-ventral axis of EC project
to transverse slices along septotemporal axis of DG
Can we use this spectrum of small
spatial scales in EC to decode
large spatial scales in DG?
CN 510 Lecture 14
Core Idea
Adding several nearby scales
results in least common
multiple (LCM) rule
40 and 50cm = 2m
x4
44 and 52cm = 5.72m x10
41 and 53cm = 21.73m x40
LCM works perfectly for
points, but not for areas
CN 510 Lecture 14
Core Idea
There are complete overlaps
at LCM frequency
There are also partial
overlaps that lead to
secondary place fields or
to individual spikes
LCM allows multi-peaked
place cells
DG place cells have 1.79±1.4
place fields per cell (Jung
and McNaughton, 1993)
In the data multi-peak place
fields are usually uneven
CN 510 Lecture 14
Input 1
+
Input 2
=
Output
Secondary peak or
“spontaneous noise”?
Grid-to-Place Decoding And Learning
Five EC cells per spatial scale
DG cells combine two entorhinal spatial scales
CN 510 Lecture 14
DG cells compete
Winner triggers retrograde learning signal
CN 510 Lecture 14
Fixed Weights Results
25 place fields, 5 with two peaks
Place fields cover half the space
“Spontaneous” firing outside place fields
Proof of concept, but can this map be learned?
Phase Precession to the Rescue
Field that the rat is exiting has
spikes early in the theta cycle
Field that the rat is entering has
spikes late in the theta cycle
(Skaggs et al., 1996)
Found in both EC layer II and DG
Temporal distance 50-60ms
CN 510 Lecture 14
Spike Timing Dependent Plasticity (STDP)
Bi and Poo (1998)
Synaptic change is limited to a
narrow time window of 3050ms
Given the separation between
spikes from phase precession
we get independent learning
episodes on different phases
of theta
Simulation: Grid-to-Place Learning
Most place fields prewired in first simulation were recreated during
self-organization
Some developed extra peaks
The structure of the weights approaches preset weights used in the
first simulation
Spatial Scale
Grid cell index
Random initial weights
1
2
3
4
5
44
0.167
0.158
1.010
0.499
0.479
52
0.857
0.963
0.194
0.333
0.558
Spatial Scale
Learned weights
Grid cell index
1
2
3
4
5
44
0.050
0.050
0.950
0.111
0.050
52
0.082
0.986
0.114
0.050
0.050
Self-organizing map between EC and DG can expand small
spatial scales into a large spatial representation
Results for Curious
Nothing Is Forewer
Praveen Pilly has shown that neither phase precession nor
STDP is necessary to achieve a similar effect
He used Gaussian-like overlapping grid cell profiles in the
rate model and synaptic competition in the postsynaptic
cells (similar to Levy’s second rule)
As a result strong overlaps led to strong weights while weak
overlaps were suppressed due to competition
CN 510 Lecture 14
Next Week
Dynamic interactions between short-term memory
represented by neuronal activation and long-term memory
represented by weight changes are discussed with
reference to a classical conditioning example
The outstar learning theorem is summarized
A hypothetical modeling problem is carried out using the
outstar as a network building block
Finally, the use of outstars in a model of temporal sequence
learning, the associative avalanche, is described
Homework due: Communication in spiking network
CN 510 Lecture 14