LETTER
Communicated by William Bialek
Enhancement of Information Transmission Efficiency by
Synaptic Failures
Mark S. Goldman∗
[email protected]
Brain and Cognitive Sciences Department, MIT, Cambridge, MA 02139, U.S.A.
Many synapses have a high percentage of synaptic transmission failures. I
consider the hypothesis that synaptic failures can increase the efficiency
of information transmission across the synapse. I use the information
transmitted per vesicle release about the presynaptic spike train as a
measure of synaptic transmission efficiency and show that this measure
can increase with the synaptic failure probability. I analytically calculate
the Shannon mutual information transmitted across two model synapses
with probabilistic transmission: one with a constant probability of vesicle release and one with vesicle release probabilities governed by the
dynamics of synaptic depression. For inputs generated by a non-Poisson
process with positive autocorrelations, both synapses can transmit more
information per vesicle release than a synapse with perfect transmission, although the information increases are greater for the depressing
synapse than for a constant-probability synapse with the same average
transmission probability. The enhanced performance of the depressing
synapse over the constant-release-probability synapse primarily reflects a
decrease in noise entropy rather than an increase in the total transmission
entropy. This indicates a limitation of analysis methods, such as decorrelation, that consider only the total response entropy. My results suggest that
synaptic transmission failures governed by appropriately tuned synaptic
dynamics can increase the information-carrying efficiency of a synapse.
1 Introduction
Synaptic transmission failures are commonly seen in recordings of single
central synapses, with the fraction of synaptic failures at a given synapse
often exceeding the fraction of successful transmissions (Stevens & Wang,
1995; Murthy, Sejnowski, & Stevens, 1997). When a synapse fails to transmit, information about the presynaptic spike train is lost. Nevertheless, the
prevalence of synaptic failures suggests that they may have a functional role,
and a variety of such roles have been proposed. Synaptic failures could re∗ Present address: Department of Physics and Program in Neuroscience, Wellesley
College, Wellesley, MA 02481, U.S.A.
c 2004 Massachusetts Institute of Technology
Neural Computation 16, 1137–1162 (2004) 1138
M. Goldman
flect the conservation of limited synaptic resources for a synapse with a
limited pool of primed, readily releasable vesicles (Murthy et al., 1997). Alternatively, synaptic failures could provide an energy-saving mechanism
that reduces the large metabolic costs of synaptic transmission (Balasubramanian, Kimber, & Berry, 2001; Laughlin, 2001; Levy & Baxter, 2002;
Schreiber, Machens, Herz, & Laughlin, 2002). Congruent with these proposals is the suggestion that synapses could serve as dynamic filters of the
presynaptic spike train, with less important spikes being filtered out while
more important spikes are transmitted (Maass & Zador, 1999; Natschlager
& Maass, 2001; Goldman, Maldonado, & Abbott, 2002). Short-term synaptic
dynamics such as synaptic depression or facilitation have been suggested as
the mechanism underlying this role. Similarly, synaptic failures associated
with synaptic depression have been suggested as a means of gain control
by which a postsynaptic neuron can respond to important information arriving along particular synapses while filtering out less interesting inputs
arriving at other synapses (Abbott, Varela, Sen, & Nelson, 1997).
Common to the above proposals is the suggestion that synaptic failures
can increase the efficiency of information transmission by a synapse. One
possible source of inefficiency in natural spike trains is temporal autocorrelations. Spike train autocorrelations have been widely observed in the
sensory areas of animals viewing natural stimuli (Dan, Atick, & Reid, 1996)
and of freely viewing animals (Baddeley et al., 1997; Goldman et al., 2002)
and indicate that information is encoded in a redundant manner (Barlow,
1961). Reduction of this redundancy has been suggested as a computational principle underlying early sensory processing (see, e.g., Attneave,
1954; Barlow, 1961; Srinivasan, Laughlin, & Dubs, 1982; Barlow & Foldiak
1989). It has been suggested that the removal of spike train autocorrelations by the synapse could produce a more efficient representation of the
encoded information (Goldman et al., 2002). However, the mechanism by
which this decorrelation was proposed, synaptic depression, is inherently
stochastic. This means that the same spike train, if presented multiple times,
will produce different sets of transmissions across a depressing synapse. In
information-theoretic terms, for a fixed number of transmissions, decorrelation leads to maximization of the total response entropy of the transmissions
but does not generally lead to a reduction in the noise entropy. In this article,
I re-explore the hypothesis that synaptic failures can increase the efficiency
of information transmission across a synapse. As a measure of information
transmission efficiency, I use the Shannon information transmitted per vesicle release by a sequence of postsynaptic vesicle releases about a sequence
of presynaptic action potentials. As a measure of information transmission
efficiency, I use the Shannon information transmitted per vesicle release by
a sequence of postsynaptic vesicle releases about a sequence of presynaptic
action potentials (see section 2). For the cases considered here, it will be seen
that the noise entropy rather than the total response entropy dominates the
behavior of this quantity.
Both the noise and the total entropies have been calculated in previous
Enhancement of Information Transmission Efficiency
1139
studies that use direct numerical calculations (de Ruyter van Steveninck,
Lewen, Strong, Koberle, & Bialek, 1997; Strong, Koberle, de Ruyter van
Steveninck, & Bialek, 1998) of information. However, these studies are typically constrained to timescales on the order of tens of milliseconds. This limitation is due to a combinatoric explosion in the number of spike train probabilities that need to be computed numerically. Spike train autocorrelations
and short-term synaptic dynamics often extend for many hundreds of milliseconds, making numerical calculations intractable for typical spike rates.
Previous studies have approximated the information transmitted across
a depressing synapse under various simplifying assumptions (Goldman,
2000; de la Rocha, Nevado, & Parga, 2002; Fuhrmann, Segev, Markram, &
Tsodyks, 2002; Goldman et al., 2002). Here, I perform an exact analytic calculation of the information transmitted across a model depressing synapse.
To make this calculation tractable, I use simplified models of a stochastic
depressing synapse and of positively autocorrelated spike trains.
Using several examples, I analyze the effects of synaptic failures, spike
train autocorrelations, and synaptic dynamics on the transmission of information across a synapse. In section 3, the information transmitted per vesicle
release by a synapse with constant probability of transmission is compared
to that transmitted by a perfectly transmitting synapse with no synaptic failures. In section 4, the information per vesicle release for a model depressing
synapse is compared to that for the perfectly transmitting synapse. In section 5.1, the depressing and constant-probability synapses are compared
directly. This last comparison gives insight into how synaptic dynamics can
increase the efficiency of information transmission across a synapse. The
results of the analytic calculations are constrained to a particular, analytically tractable synaptic model and form of spike train autocorrelations. In
section 5.2, a qualitative framework is presented for understanding more
generally how synaptic dynamics can enhance the efficiency of information
transmission across a synapse when spike trains are autocorrelated.
2 Information Calculations with Point Process Spike Trains and
Transmissions
The calculations that follow compute the Shannon mutual information
I(S; V) conveyed by a sequence of N synaptic transmissions (vesicle releases) occurring at times v = (v1 , v2 , . . . , vN ) about a presynaptic spike
train with n spikes occurring at times s ≡ (s1 , s2 , . . . , sn ):
I(S; V) = H(V) − H(V|S)
=−
P(v) log2 (P(v))
v
+
s
P(s)
v
(2.1)
(2.2)
P(v|s) log2 (P(v|s)),
where P(·) denotes probability, the random variables S and V denote the
ensembles of spike trains and vesicle releases, respectively, and the sums are
1140
M. Goldman
over all possible realizations s and v, respectively, of the random variables
S and V. Written in this form, the information is defined as the difference
between the transmission entropy H(V) and the noise entropy H(V|S) of
the train of synaptic vesicle releases. H(V) quantifies the variability in the
sequence of synaptic vesicle releases and gives the information-carrying
capacity of the vesicle releases. Subtracting H(V|S) from the transmission
entropy removes the portion of the vesicle release train variability that arises
when the presynaptic spike train is held fixed.
The presynaptic spike and postsynaptic vesicle release sequences are
modeled by point processes, with spike and release times specified with a
constant precision δT. The probabilities P(v) and P(s) are then expressed in
terms of probability densities P (v) and P (s) as P(v) = P (v)(δT)N and P(s) =
P (s)(δT)n , respectively. A consequence of this model is that it gives rise to
a term −N log2 (δT) in H(V). Correspondingly, the information per vesicle
release I(S; V)/N contains a constant term − log2 (δT) that approaches ∞ as
δT → 0. This reflects that, in this limit, each of the N vesicle releases is specified with infinitesimal resolution and has a correspondingly large capacity
to carry information. In practice, any numerical calculation of the information requires the choice of a finite time bin that defines the meaningful
precision for a given system. Since the biologically relevant precision δT is
not necessarily known, it is useful to have a measure for comparing synaptic
transmission efficiency that is independent of δT. Because measures such as
the information transmitted per unit time or energy are dependent on δT, I
choose not to use these even though they could provide interesting results
that are calculable for specific choices of δT. I instead use the difference
between the information per vesicle release transmitted by two synapses,
which is independent of δT. In section 6, I point out some qualitative distinctions between these different measures.
3 Information Transmission for a Synapse with Constant Vesicle
Release Probability
3.1 Poisson Input with Time-varying Rate. I first calculate the information I(S; V) for a synapse with constant vesicle release probability p receiving input described by a Poisson process of arbitrary rate r(t). This example
illustrates the effect of synaptic unreliability in the case that: (1) spikes are
generated independently, and (2) transmission failures are governed by a
process that has no dynamics. After the calculation, I consider the effects of
relaxing these two assumptions.
Because the transmission probability p is time independent and the input
is Poisson, H(V) and H(V|S) can be found from the infinitesimal entropies
δH(V) and δH(V|S) corresponding to a single time bin (t, t + δT), where δT
is assumed to be small. Inserting the probabilities of vesicle release, p r(t)δT,
and failure, 1 − p r(t)δT, into equation 2.2 and ignoring terms of order (δT)2
Enhancement of Information Transmission Efficiency
1141
gives
δH(V) = −pr(t)δT log2 (pr(t)δT)−(1−pr(t)δT) log2 (1−pr(t)δT)
1
≈ pr(t)δT
− log2 (pr(t)δT)
ln 2
δH(V|S) = −r(t)δT[p log2 (p) + (1 − p) log2 (1 − p)]
(3.1)
(3.2)
(3.3)
Summing the above equations over time and dividing by the average
T
number of transmissions in a time Ttotal , N = 0 total pr(t) dt, gives the entropies and information per vesicle release (here and throughout, I assume
δT is small and denote sums over time intervals of size δT by integrals with
respect to the dummy integration variable t):
H(V)
1
=
−
N
ln 2
Ttotal
0
r(t) log2 (pr(t)δT) dt
.
Ttotal
r(t) dt
0
(3.4)
H(V|S)
1−p
= − log2 (p) +
log2 (1 − p) ,
N
p
I(S; V)
1
1−p
=
+
log2 (1 − p) −
N
ln 2
p
Ttotal
0
r(t) log2 (r(t)δT) dt
.
Ttotal
r(t) dt
0
(3.5)
(3.6)
It should be noted that, although different in detail, the above derivation is
similar in form and spirit to that for the information carried by the arrival
time of a single spike (Brenner, Strong, et al., 2000).
For a synapse with perfect transmission (p = 1), the corresponding information per vesicle release equals the entropy per spike present in the
presynaptic spike train,
Ip=1 (S; V)
H(S)
1
=
=
−
N
n
ln 2
Ttotal
0
r(t) log2 (r(t)δT) dt
,
Ttotal
r(t) dt
0
(3.7)
where n is the number of spikes in the presynaptic train.
The difference Iconst /N between the information transmitted per vesicle
release by the constant probability and perfectly transmitting synapses is
independent of the rate r(t),
Iconst /N =
I(S; V) Ip=1 (S; V)
1−p
−
=
log2 (1 − p),
N
N
p
and is shown in Figure 1.
(3.8)
1142
M. Goldman
'Iconst/N (bits)
0.0
-0.5
-1.0
-1.5
0.0
0.2
0.4
0.6
0.8
1.0
Synaptic failure probability
Figure 1: Synaptic unreliability decreases information per vesicle release for a
constant-probability synapse receiving time-varying Poisson input. The information difference Iconst /N = Iconst /N − Ip=1 /N decreases monotonically with
the synaptic failure probability 1 − p. This result is independent of the Poisson
rate r(t).
One might have naively expected the information per transmission to
be independent of p for a constant-probability synapse. This would be the
case if the information transmitted were proportional to the number of
transmissions. Instead, the information per transmission decreases monotonically with the failure probability 1 − p (see Figure 1). For example, a
constant-probability synapse with p = 0.5 transmits 1 bit per transmission
less than a perfectly transmitting synapse. Mathematically, the loss of information per vesicle release derives from the contribution of the failures to the
noise entropy, expressed by the second term of equation 3.5. Qualitatively,
the information per transmission is less for the constant-probability synapse
because, in addition to reducing the number of transmissions, there is added
uncertainty associated with not knowing how many presynaptic action potentials failed to transmit and which of the many possible arrangements
of these untransmitted action potentials was actually present in the presynaptic spike train (a similar situation is described by Shannon & Weaver,
1949).
3.2 Input Generated by a Stationary Renewal Process. For Poisson input, the probability of producing a spike in any given time bin depends on
only the rate r(t) and is independent of the spiking activity in any other
time bin. I next show that when the locations of spikes depend on the locations of previous spikes, the constant-probability synapse can transmit more
information per vesicle release than a perfectly transmitting synapse. To introduce spike-spike dependencies, I model the presynaptic spike train by a
Enhancement of Information Transmission Efficiency
1143
renewal process, which generates successive interspike intervals (ISIs) independently from one another. Furthermore, for tractability of calculation,
the analyses presented here are restricted to stationary renewal processes,
which are characterized by a time-independent probability density PISI (S )
of generating an ISI of duration S .
The stationary renewal process is a generalization of the stationary (or homogeneous) Poisson process. Unlike the homogeneous Poisson process, the
stationary renewal process generally has nonzero autocorrelations. Spike
train autocorrelations are an indication of redundancy in the neural code
(Barlow, 1961). This suggests that synaptic transmission failures might increase the efficiency of information transmission by removing redundancy
from the presynaptic spike trains.
Below, I derive general expressions for H(V)/N and H(V|S)/N for a
constant-probability synapse receiving stationary renewal spike train input. I then apply these results to a model renewal spike train constructed
to have nonnegative autocorrelations. The autocorrelation function of the
stationary renewal process spike trains is derived in appendix A.
3.2.1 General Results. The probability P(s) of generating a particular nspike train s from a stationary renewal process is given by
P(s) =
n
[PISI (S,i )δT],
(3.9)
i=1
where δT gives the precision with which all spike and transmission times
are defined, and S,i denotes the duration of the ith ISI.
For a synapse with constant vesicle release probability p receiving a spike
train generated by a stationary renewal process, the intervesicle release interval (IVI) distribution PIVI (V ) is also a stationary renewal process. This
is because the transmission probabilities across the constant-probability
synapse do not depend on the locations of spikes previous to the one transmitted. Thus, the probability of generating a particular N-vesicle-release
train v is given by
P(v) =
N
[PIVI (V,i )δT],
(3.10)
i=1
where V,i denotes the duration of the ith IVI. The IVI distribution PIVI (V )
for the constant probability synapse can be derived from the ISI distribution
PISI (S ) by using the method of Laplace transforms (Cox, 1962; Cox &
Lewis, 1966; de la Rocha et al., 2002). The resulting expression, derived in
appendix B, is
ISI (u)
pP
−1
PIVI (V ) = L
,
(3.11)
ISI (u)
1 − (1 − p)P
1144
M. Goldman
denotes the Laplace transform of the distribution P , u denotes the
where P
argument of the Laplace transform, and L−1 {·} indicates the operation of
taking the inverse Laplace transform.
Inserting equation 3.10 into the definition of the transmission entropy
(equation 2.2) and noting that the integral separates into N identical terms
gives the transmission entropy per vesicle release H(V)/N as
∞
H(V)
dV PIVI (V ) log2 (PIVI (V )δT).
(3.12)
=−
N
0
Using equations 3.11 and 3.12, H(V)/N can be calculated for any spike train
generated by a stationary renewal process.
I next find a corresponding expression for the noise entropy per release
H(V|S)/N. To calculate this quantity, note that the vesicle releases across the
constant probability synapse have no history dependence. This implies that
the conditional probability P(v|s) can be written as a product of probabilities
Q(bi |S,i ) that the ith spike triggers a vesicle release, bi = 1, or does not,
bi = 0:
P(v|s) =
n
Q(bi |S,i ),
(3.13)
i=1
where the probability Q(b|S ) is given by,
1 − p, b = 0
Q(b|S ) =
p, b = 1.
(3.14)
Substituting equation 3.13 into the definition of the noise entropy (equation 2.2) gives the noise entropy per spike H(V|S)/n,
∞
H(V|S)
=−
dS PISI (S )
Q(b|S ) log2 (Q(b|S )).
(3.15)
n
0
b=0,1
Converting this to a noise entropy per vesicle release H(V|S)/N by dividing by the probability of vesicle transmission p gives, after substitution for
Q(b|S ) from equation 3.14,
(1 − p)
H(V|S)
(3.16)
= − log2 p +
log2 (1 − p) .
N
p
This expression is independent of the ISI distribution PISI (S ) and is identical to that derived previously for the Poisson spike train, equation 3.5.
From the general expressions given above for H(V)/N (equation 3.12
using PIVI (V ) from equation 3.11) and H(V|S)/N (equation 3.16), the information per vesicle release I(S; V)/N can be obtained for any spike train
defined by a stationary renewal process.
Enhancement of Information Transmission Efficiency
1145
3.2.2 Input with Nonnegative Autocorrelations That Decay Exponentially.
In this section, the general expressions derived above are evaluated for
a stationary renewal spike train of average rate r and with positive autocorrelations of the form C(τ ) = Ae−|τ |/τcorr . This example allows independent
study of the effects of changing the average rate r, the magnitude of autocorrelations A (normalized to be independent of r), and the exponential
decay time of the autocorrelations τcorr .
In appendix A, I show that autocorrelations of this form are generated
when the ISI distribution is given by (Cox, 1962; Cox & Lewis, 1966; de la
Rocha et al., 2002)
PISI (S ) = αλ− e−λ− S + (1 − α)λ+ e−λ+ S
(3.17)
with
λ± = K ±
α=
−1
K2 − rτcorr
,
−1 − λ
τcorr
−
,
λ+ − λ−
(3.18)
(3.19)
−1 ]/2. P ( ) above corresponds to choosing
and where K ≡ [r(1 + A) + τcorr
ISI
S
each ISI randomly from a homogeneous Poisson distribution of either rate
λ− (with probability α) or rate λ+ (with probability 1 − α). The inverse relations for r, A, and τcorr as functions of α, λ− , and λ+ are given in appendix A.
To determine I(S; V)/N from equations 3.12 and 3.16, one need only calculate PIVI (V ). For the constant probability synapse, PIVI (V ) can be obtained by the following trick. Because the constant-probability synapse has
no history dependence, vesicle release failures at the constant-probability
synapse are equivalent to failures to receive a spike. Therefore, the vesicle release train produced by a constant-probability synapse with release
probability p receiving input of average rate r is statistically identical to
that produced by a reliable synapse receiving input of average rate pr. This
implies that PIVI (V ) can be determined from PISI (V ) by replacing r in
equations 3.17 through 3.19 by pr.
The difference Iconst /N between I(S; V)/N for the unreliable (p < 1)
and perfectly transmitting (p = 1) synapses is shown in Figure 2. When the
input spike train has no autocorrelations (A = 0; see Figure 2, dashed lines),
the input is a homogeneous Poisson process, and Iconst /N is identical to
that found in section 3.1 (see Figure 1).
When the spike trains have sufficiently strong autocorrelations (see Figure 2, A = 5 and rτcorr ≥ 3), the constant-probability synapse can transmit
more information per vesicle release than a reliable synapse. This reflects
that when inputs are generated by a non-Poisson process, the location of
1146
M. Goldman
'Iconst/N (bits)
1.0
r Wcorr = 0.5
r Wcorr = 3
r Wcorr = 15
0.5
0.0
-0.5
-1.0
-1.5
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Failure probability
Failure probability
Failure probability
A=0
A=1
A=5
Figure 2: Information per vesicle release difference Iconst /N = I(S; V)/N −
Ip=1 (S; V)/N for a constant-probability synapse receiving input generated by
the stationary renewal process of equation 3.17. Columns correspond to inputs
with different decay times of positive autocorrelations (measured relative to the
spike rate r). Individual traces give Iconst /N for the specified autocorrelation
magnitudes A (see the legend). Iconst /N tends to be negative except when the
input is strongly correlated and the transmission probability is very low.
one spike is correlated with the location of other spikes. Observation of a
vesicle release therefore not only identifies the location of the spike that
caused the transmission but also partially identifies the locations of correlated spikes. When the spikes are sufficiently strongly autocorrelated, the
extra information gained about the locations of nontransmitted spikes can
outweigh the noisiness of the constant-probability synapse. In this case,
Iconst /N can be greater than zero. However, the positive information differences tend to be small and occur at only very low transmission probabilities (see Figure 2, <7% transmitted for A = 5, rτcorr = 3; <3% transmitted
for A = 5, rτcorr = 15). Thus, the information per unit time transmitted
by such a synapse would be quite low in this regime (see section 6). The
constant-probability synapse has the same autocorrelation function C(τ ) as
the presynaptic spike train and has maximal noise entropy for a given number of releases. Therefore, positive information differences are attributable
to increases in transmission entropy as the transmission rate becomes lower
rather than to changes in the statistical structure of the releases. For the same
statistical structure, lower transmission rates tend to increase the entropy
per transmission because transmissions are less probable, and thus more
surprising, at lower rates (for a concrete example, consider the low rate
limit of equation 3.4 for the Poisson process). As I will show, positive infor-
Enhancement of Information Transmission Efficiency
1147
mation differences for positively autocorrelated spike trains can be much
more dramatic when the synapse has the dynamics of synaptic depression.
4 Information Transmission for a Stochastic Depressing Synapse
4.1 Model of Synaptic Depression. I next calculate the information
transmitted across a stochastic model synapse with failures of transmission
resulting from post–vesicle release refractoriness. The model is intended to
capture, in an analytically tractable way, both the stochasticity of synaptic
transmission and the essential dynamics of depressing synapses, in which
the average transmission amplitude falls dramatically following a presynaptic spike before recovering over a characteristic timescale (Abbott et al.,
1997; Tsodyks & Markram, 1997; Varela et al., 1997).
I assume a model synapse with a single active zone containing a single vesicle release site. The synapse transmits perfectly when a vesicle is
present at the release site and does not transmit when the vesicle is absent (which I will refer to as being in the depleted state). Following vesicle
release, the synapse becomes refractory until the vesicle is replenished. Vesicle replenishment is governed by a homogeneous Poisson process of rate
1/τD (for similar models, see Vere-Jones, 1966; Goldman, 2000; Matveev &
Wang, 2000). The simplifying assumption that the synapse either transmits
perfectly (when the vesicle is available) or is refractory (when the vesicle
is unavailable) makes the calculation of information transmission for this
model analytically tractable.
The ensemble-averaged behavior of this synapse is that the transmission
probability drops to zero following release and then recovers exponentially
back to one with a time constant τD . Because the synaptic transmission
probability drops all the way to zero following release, this model represents
a very strongly depressing synapse.
Failures of transmission by the model depressing synapse result exclusively from vesicle depletion and are governed by the dynamics of vesicle replenishment. This contrasts with the failures of transmission by the
constant-probability synapse that result from a purely static probability of
vesicle release failure. Studying these two types of synapses allows a comparison between the effects of stochasticity in the presence or absence of
synaptic dynamics.
4.2 Input Generated by a Stationary Renewal Process. In this section,
the calculations of section 3.2 are repeated for the model depressing synapse.
As in that section, general results are first given for an arbitrary stationary
renewal process and then applied to the autocorrelated renewal process
described in section 3.2.2.
4.2.1 General Results. The calculations of H(V)/N and H(V|S)/N for
the depressing synapse follow closely those for the constant-probability
1148
M. Goldman
synapse (section 3.2). This is a consequence of the simplified model of synaptic depression, as described below.
I consider a stationary renewal process with P(s) defined as in equation
3.9. Because the ISIs are independent and the depressing synapse is in an
identical (depleted) state after each transmission, the IVIs are independent
as well. This implies that P(v) takes the form of a product, as in equation 3.10,
and that H(V)/N assumes the form given in equation 3.12. The only difference between H(V)/N for the constant-probability and model-depressing
synapses is in the form of the IVI distribution function PIVI (V ). This can
be found for the depressing synapse by the method of Laplace transforms
(Cox, 1962; Cox & Lewis, 1966; de la Rocha et al., 2002). The calculation is
outlined in appendix B. The resulting distribution is
−1
PIVI (V ) = L
ISI (u + τ −1 )
ISI (u) − P
P
D
.
ISI (u + τ −1 )
1−P
(4.1)
D
The noise entropy per release H(V|S)/n is also of the same form as that
for the constant-probability synapse (see equation 3.15). This is because the
depressing synapse is in an identical (depleted) state not only following
each transmission, but also following each spike. If the synapse is depleted
just before the spike, it remains depleted; if the synapse is not depleted
before the spike, it releases and becomes depleted. As a consequence, the
conditional probabilities P(v|s) factor into a product as in equation 3.13,
leading to H(V|S)/n as in equation 3.15. The difference from the constantprobability synapse is in the expression for the probability Q(b|S ). For the
depressing synapse, Q(b|S ) is given by the exponential probability that
the synapse recovers during the ISI S :
e−S /τD , b = 0
1 − e−S /τD , b = 1.
Q(b|S ) =
(4.2)
H(V|S)/n can be converted to a noise entropy per transmission H(V|S)/N
by dividing by the average probability of vesicle transmission p,
p=
N
=
n
∞
0
dS PISI (S )Q(1|S ),
(4.3)
giving
H(V|S)
=−
N
∞
0
dS PISI (S ) b=0,1 Q(b|S ) log2 (Q(b|S ))
∞
.
0 dS PISI (S )Q(1|S )
(4.4)
The information per vesicle release I(S; V)/N is the difference between
H(V)/N (equation 3.12, using PIVI (V ) from equation 4.1) and H(V|S)/N
(equation 4.4).
Enhancement of Information Transmission Efficiency
'Idep/N (bits)
5
r Wcorr = 0.5
r Wcorr = 3
1149
r Wcorr = 15
4
3
2
1
0
-1
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Failure probability
Failure probability
Failure probability
A=0
A=1
A=5
Figure 3: Information per vesicle release difference Idep /N = I(S; V)/N −
Ip=1 (S; V)/N for the model-depressing synapse receiving input generated by
the stationary renewal process of equation 3.17. Graphs are organized as in Figure 2. Idep /N is positive for a wide range of parameters when the input has
nonzero autocorrelations.
4.2.2 Input with Nonnegative Autocorrelations That Decay Exponentially.
For positively correlated inputs, I(S; V)/N can be much larger for the depressing synapse than for a reliable (p = 1) synapse. I demonstrate this by
evaluating the general results derived above for the exponentially correlated
renewal process (see section 3.2.2, equations 3.17 through 3.19). PIVI (V ) and
p for this process are calculated in appendix B. Here, I give only the resulting
information differences Idep /N = I(S; V)/N − Ip=1 (S; V)/N between the
depressing and reliable synapses.
Figure 3 shows Idep /N as a function of the input parameters and average synaptic failure probability 1 − p. Idep /N > 0 over a wide range
of parameter values, indicating that the depressing synapse can transmit
more information per vesicle release than a reliable synapse. This contrasts
with the corresponding result for the constant-probability synapse (see Figure 2), in which positive information differences occur only for extremely
low transmission probabilities.
For uncorrelated input, the depressing synapse always transmits less information per vesicle release than the perfectly transmitting synapse (Idep <
0; see Figure 3, dashed lines). This is to be expected because for uncorrelated
input, the perfectly transmitting synapse has both maximal total transmission entropy (for a given number of transmissions) and no noise entropy.
However, for correlated input, the perfectly transmitting synapse has nonmaximal total transmission entropy and may transmit less information per
1150
M. Goldman
vesicle release than the depressing synapse (Idep > 0). The positive information differences tend to increase with the magnitude of autocorrelations
A (see Figure 3, thin and thick solid lines). Large increases in Idep occur
around values of p where the correlation and depression timescales match,
τD = τcorr (these values are indicated by the arrows on Figure 4B), and tend
to increase more rapidly for higher values of τcorr (compare the three graphs
in Figure 3). Because higher values of τD correspond to higher failure probabilities (see equation B.13), the rapid increases in Idep shift to higher failure
probabilities when τcorr is increased.
5 Comparison of the Information Transmitted Across the
Constant-Probability and Depressing Synapses
The information differences between the depressing and reliable synapses
reflect differences in both average transmission probability and synaptic
dynamics. To isolate the effects of synaptic dynamics, I next compare the
information transmitted by the depressing synapse to that transmitted by
a constant-probability synapse with the same average transmission probability. The nonnegatively autocorrelated renewal spike train is considered
below. This is followed by a qualitative analysis that enables the information transmission process to be understood in a framework that is more
generally applicable to non-analytically tractable studies.
5.1 Input with Nonnegative Autocorrelations That Decay Exponentially. A comparison of Figures 2 and 3 reveals that the information transmitted by the depressing synapse, Idep (S; V), exceeds that transmitted by the
constant-probability synapse, Iconst (S; V), when the depressing and constantprobability synapses have the same average transmission probability (i.e.,
when p = p so that the average number of transmissions N is the same
for both synapses). Figure 4A shows this explicitly, plotting Idep−const /N =
Idep (S; V)/N − Iconst (S; V)/N as a function of the input parameters and average failure probability. For all parameter values tested, Idep−const /N ≥ 0.
This demonstrates that the depressing synapse transmits more information
about these trains than does the constant probability synapse.
The larger transmission of information per vesicle release by the depressing synapse, compared to the constant probability synapse, primarily
reflects a decrease in the noise entropy H(V|S)/N (see Figure 4C) rather than
an increase in the total entropy H(V)/N (see Figure 4B). This is surprising
in the light of previous information studies (Srinivasan, Laughlin, & Dubs,
1982; Barlow & Foldiak, 1989; Dan et al., 1996; de la Rocha et al., 2002; Goldman et al., 2002; Wang, Liu, Sanchez-Vives, & McCormick, 2003) that have
primarily used decorrelation, or maximization of H(V) for a fixed number
of transmissions, as a measure of information transmission efficiency. For
uncorrelated input, the constant-probability synapse has maximal total entropy per release H(V)/N, reflecting that the transmissions are completely
Enhancement of Information Transmission Efficiency
'Idep-const/N (bits)
A
r Wcorr = 3
r Wcorr = 15
5
4
3
2
1
0
-1
B
'Hdep-const(V)/N
r Wcorr = 0.5
1151
5
4
3
2
1
0
-1
'Hdep-const(V|S)/N
C
1
0
-1
-2
-3
-4
-5
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Failure probability
Correlation
D1
Failure probability
Failure probability
A=0
A=1
A=5
1
0
0
-1
0
W (ms)
-1
500
0
W (ms)
500
Figure 4: Synaptic depression increases the information per vesicle release relative to a constant-probability synapse with the same average transmission probability. (A) Information per vesicle release difference Idep−const /N = Idep /N −
Iconst /N between the depressing and constant-probability synapses. Graphs are
organized as in Figures 2 and 3. (B) The corresponding response entropy difference Hdep−const (V)/N = Hdep (V)/N − Hconst (V)/N is negative for uncorrelated
input (dashed line) but can be positive for correlated input (solid lines). Peak positive values are located near where τD = τcorr (indicated by arrows). These results
reflect the autocorrelation structure of the transmissions (panel D). (C) The noise
entropy difference Hdep−const (V|S)/N = Hdep (V|S)/N − Hconst (V|S)/N is negative for all parameter values and is larger in magnitude than Hdep−const (V)/N.
(D) Autocorrelations C(τ ) of the vesicle releases across a constant-probability
(left) and depressing (right) synapse receiving uncorrelated input. Each synapse
transmitted 50% of the presynaptic spikes. The depressing synapse transmits
more information per vesicle release than the constant-probability synapse even
though the transmissions across the constant-probability synapse are uncorrelated.
1152
M. Goldman
uncorrelated (see Figure 4D, left). By contrast, the depressing synapse has
nonzero autocorrelations (see Figure 4D, right). Thus, for a given failure
probability, the constant-probability synapse has larger total entropy per
release than the depressing synapse in this case (Hdep−const (V)/N < 0; see
Figure 4B, dashed lines). Nevertheless, the depressing synapse transmits
more information per release (Idep−const /N > 0; see Figure 4A, dashed
lines), reflecting that the relative decrease in total entropy for the depressing synapse is more than compensated for by the relative decrease in noise
entropy (H(V|S)/N < 0; see Figure 4C, dashed lines).
The noise entropy per release is always larger for the constant-probability
synapse because a synapse with constant probability of transmission is maximally noisy in the sense that all spikes are transmitted with equal probability. The depressing synapse is less noisy because it has a nonuniform
probability of transmission: spikes that follow long ISIs are transmitted
more reliably, and spikes that follow short ISIs are transmitted less reliably.
For correlated input, the noise entropy differences Hdep−const (V|S)/N are
even larger (see Figure 4C, solid lines). This is because the positively correlated input tends to have clusters of spikes that enhance the nonuniformity
of transmission probabilities across the depressing synapse. The first spike
in a cluster is transmitted with high probability, whereas the latter spikes
are not. Moreover, Hdep−const (V)/N can be greater than zero when the input has positive autocorrelations (see Figure 4B, solid lines). This reflects
the decorrelation of the positively correlated spike trains by the depressing
synapse and, as noted in previous work (Goldman et al., 2002), tends to be
maximal when τD = τcorr (indicated by the arrows in Figure 4B).
5.2 Qualitative Analysis. The previous sections give insight into how
the total entropies compare for the depressing and constant-probability
synapses and how the noise entropies compare. In some cases, such as for
uncorrelated input (see Figure 4, dashed lines), both the total entropy and the
noise entropy are smaller for the depressing synapse than for the constantprobability synapse. To understand why the depressing synapse transmits
more information per vesicle release (Idep−const /N > 0) in these cases
therefore requires the relatively nonintuitive assessment of whether the difference in total entropy, Hdep−const (V)/N, or difference in noise entropy,
Hdep−const (V|S)/N, is larger in magnitude. However, as demonstrated below, it becomes obvious that Idep−const /N > 0 for uncorrelated input if the
information is expressed instead as
I(S; V) = H(S) − H(S|V)
=−
P(s) log2 (P(s))
s
+
v
P(v)
s
P(s|v) log2 (P(s|v)).
(5.1)
(5.2)
Enhancement of Information Transmission Efficiency
1153
When expressed in this manner, I(S; V) gives the amount by which the
uncertainty in the presynaptic spike train ensemble S can be reduced by
observation of the transmissions. The spike train entropy H(S) gives the
uncertainty in the spike train when the transmissions are not known. The
conditional spike train entropy H(S|V) gives the uncertainty remaining after
the transmissions have been observed.
The advantage of writing I(S; V) in this form is that H(S) is independent
of the synaptic properties. Thus, the difference in the information transmitted by two synapses about a given spike train ensemble S is determined
solely by differences in H(S|V), that is, Idep−const (S; V) = −Hdep−const (S|V).
H(S|V) is given
by an average over transmission trains v of the entropy term
h(S|v) = − s P(s|v) log2 (P(s|v)). To understand the qualitative behavior
of H(S|V), I approximate the average over transmission trains by considering a typical set of transmissions (see Figure 5, v). I then compare h(S|v) for
the constant-probability and depressing synapses by estimating the probabilities P(s|v) of various presynaptic spike trains (see Figure 5, s1 , s2 , and
s3 ). More nonuniform distributions of P(s|v) correspond to lower h(S|v) and
therefore to larger I(S; V).
Figure 5A illustrates the information transmission process when the
presynaptic spike train is generated by a homogeneous Poisson process.
Consider a set of transmissions that could be produced by either the constantprobability or depressing synapse (see Figure 5, v). P(s|v) is nonzero only
for presynaptic spike trains with spikes occurring at the times of transmission. I consider three such trains (see Figure 5, s1 , s2 , and s3 ). Because these
trains have equal numbers of spikes, they have equal prior probabilities P(s)
of being generated by the homogeneous Poisson process. For the constantprobability synapse, the posterior probabilities P(s|v) are also equal (see
Figure 5A, top right) because each spike in the presynaptic train has an
equal probability of transmission.
By contrast, these trains have nonuniform probabilities of being transmitted by the depressing synapse (see Figure 5A, bottom right). Spikes
located far away from a previous transmission are more likely to be transmitted by the depressing synapse. Therefore, the spike trains most likely to
be consistent with the shown transmissions are those that do not have any
nontransmitted spikes located far away from the first transmission. This
implies that P(s|v) is lowest for train s3 and greatest for train s1 .
This pictorial argument suggests that P(s|v) is more nonuniform for the
depressing synapse than for the constant-probability synapse, and therefore I(S; V) is greater for the depressing synapse. The nonuniformity of
P(s|v) is not particular to synaptic depression. It would be expected for a
synapse with any form of dynamics. Thus, this result is not particular to
depressing synapses but more generally applies to synapses with dynamic
vesicle release probabilities. Qualitatively, synaptic dynamics allow information to be obtained not only about presynaptic spikes occurring at the
1154
M. Goldman
P(s), or P(s|v) for
const.-prob. synapse
A
s1
s2
WD
B
s1
s2
s3
v
s2
s3
s1
s2
s3
Spike Train
WD
time
P(s|v) with
P(s), or P(s|v) for
depression const.-prob. synapse
v
s1
P(s|v) with
depression
s3
s1
s2
s3
s1
s2
s3
Spike Train
Figure 5: Qualitative comparison of the information transmitted across a depressing and constant-probability synapse with the same average transmission
probabilities. The information transmitted corresponds to the ability to identify which of the many possible presynaptic trains s gives rise to an observed
set of transmissions v. (A) The depressing synapse transmits more information
about a Poisson spike train than does the constant-probability synapse. P(s|v)
is uniform for the constant-probability synapse (top right) but nonuniform for
the depressing synapse (bottom right). This reflects that spikes located far from
the previous transmission are more likely to be transmitted by the depressing synapse. (B) The depressing synapse is particularly well tuned for carrying
information about positively correlated trains. Correlated spike trains tend to
have clusters of spikes (top left) that lead to a strongly nonuniform P(s|v) for
the depressing synapse. In both A and B, we assume that the shown trains are
typical and have equal prior probabilities P(s). In the top right panels, P(s) has
been scaled to be the same height as P(s|v) for the constant-probability synapse.
The arrow labeled τD indicates the time constant of depression. See the text for
details.
Enhancement of Information Transmission Efficiency
1155
times of transmission but also about spikes occurring when there are no
transmissions.
For correlated spike trains, the form of the dynamics is important. Spike
trains generated by a positively correlated process tend to have clustered
spikes. Consider a typical train of transmissions v and three typical presynaptic trains with clustered spikes (see Figure 5B, left). For simplicity, I
assume that these spike trains have equal prior probabilities P(s) and do
not consider less clustered spike trains that would have lower prior probabilities. By the same arguments that were made for uncorrelated input,
the constant-probability synapse has a uniform distribution P(s|v) for these
trains. The depressing synapse, however, has a highly nonuniform distribution because the clustering of spikes makes it relatively likely that trains
will have spikes located either very far away from previous transmissions
(like s1 and s2 ) or very close to the previous transmission (like s3 ). This gives
rise to a much more nonuniform distribution P(s|v) than was found for uncorrelated input (see Figure 5B, left). This suggests why Idep−const /N tends
to be larger for correlated input than for uncorrelated input (see Figure 4A).
The result for correlated input does depend on the form of the dynamics. For example, a facilitating synapse would differ from the depressing
synapse in that it would be more likely, for a fixed number of transmissions,
to transmit multiple spikes from the same cluster and no spikes from other
clusters. Information about the nontransmitted clusters would be lost, and
the representation of the transmitted clusters would be redundant. This
suggests that a facilitating synapse would be less efficient at transmitting
these trains than a depressing synapse.
6 Discussion
I have studied how the information transmission across a synapse depends
on the correlation structure of the presynaptic spike trains, the stochastic nature of synaptic transmission, and the dynamics of synaptic depression. In
summary, when spikes are generated by an uncorrelated process, synaptic
transmission failures reduce the information per vesicle release transmitted
across a synapse. Intuitively, this is because the input has no redundancy that
can be reduced by synaptic failures. For correlated input, synaptic failures
can increase the information transmitted per vesicle release. Information efficiency increases are limited for the constant-probability synapse but can be
large for a depressing synapse. For the nonnegatively autocorrelated inputs
considered here, the information transmitted by the depressing synapse exceeds that transmitted across a constant probability synapse with the same
average transmission probability.
I have used information per vesicle release as the measure of synaptic
transmission efficiency. This measure is easily interpretable and allows a
comparison between synapses that is independent of the precision δT with
which spike times are specified (see section 2). An artifact of the use of this
1156
M. Goldman
measure of efficiency is that in some cases, its maximum value occurs when
the transmission probability equals zero (see Figures 2 and 3). This suggests
that other information-maximization measures might be more suitable in
the low-transmission-probability regime. For example, maximizing information per unit time favors perfect transmission. Maximizing information
per energy usage (Laughlin, de Ruyter van Steveninck, & Anderson, 1998;
Balasubramanian et al., 2001; Levy & Baxter, 2002; Schreiber et al., 2002)
similarly tends to avoid very low transmission probabilities because of the
costs of transmission-independent energy expenditures.
Many studies of information transmission in sensory areas focus on the
role of decorrelation in maximizing information transmission (Srinivasan
et al., 1982; Barlow & Foldiak, 1989; Dan et al., 1996; Goldman et al., 2002;
Wang et al., 2003). Decorrelation corresponds to a maximization of the response entropy that, when the noise entropy is small, also corresponds to an
approximate maximization of the information transmitted. However, the results here show that the noise entropy cannot be neglected. Rather, changes
in the noise entropy are the dominant contributor to the enhanced performance of the depressing synapse over the constant-probability synapse.
The constant-probability synapse is maximally noisy because it transmits
all spikes with equal probability. By contrast, the synaptic failures across the
depressing synapse are much more predictable, with higher probabilities of
failure following short ISIs and lower probabilities following long ISIs.
The noise entropy is likely to be smaller for many synaptic transmission
situations whose behavior is more complex than those modeled here. For
example, noise in the transmission process can be reduced if multiple independently releasing active zones, in either the same or different synapses,
receive the same input (de Ruyter van Steveninck & Laughlin, 1996; Manwani & Koch, 2000). In such cases, maximization of the total transmission
entropy (see Figure 4B) would assume a relatively larger role in maximizing
the information transmission. In the limit that the total transmission entropy
dominates, the result of maximizing the information per transmission for a
given fraction of transmissions would be expected to approach that found
in previous decorrelation studies (Goldman et al., 2002). However, the dominance of the noise entropy in the examples studied here suggests that for
many situations, the noise entropy is a significant factor that cannot be neglected. This is consistent with studies of other adaptation (Wainwright,
1999; Brenner, Bialek, & de Ruyter van Stevenick, 2000; Fairhall, Lewen,
Bialek, & de Ruyter van Steveninck, 2001) or refractory (Berry & Meister,
1998) processes that implicate a significant role for noise reduction in the
information transmission process.
Direct numerical calculations of information (de Ruyter van Steveninck
et al., 1997; Strong et al., 1998) explicitly account for noise entropies but are
limited in practice by the need to use either large time bins or brief duration word sizes. These constraints can be problematic when considering
spike trains that may have high temporal precision but long timescales of
Enhancement of Information Transmission Efficiency
1157
autocorrelations. I handle this problem by analytically solving the synaptic information transmission problem for simplified models of positively
correlated spike trains and depressing synapses. Attempts to include more
realistic features of spike trains and synapses in calculation of the Shannon
information have worked in limits that consider only total transmission
entropies (Goldman et al., 2002; de la Rocha et al., 2002), or used approximate but more easily calculated measures of the information transmission
(Fuhrmann et al., 2002).
The ideal measure of information transmission efficacy must consider the
synapse in the context of the whole neuron or network. Calculations of the
information that a postsynaptic neuron’s spike train conveys about a particular presynaptic input have thus far been limited to constant-transmissionprobability models of stochastic synapses (Zador, 1998; Manwani & Koch,
2001) or have ignored synaptic stochasticity altogether (London, Schreibman, Hausser, Larkum, & Segev, 2002). An interesting open question is
how synaptic dynamics and stochasticity may be incorporated into such
models of information transmission by the postsynaptic neuron.
Because the calculations here and elsewhere are applicable only to limited
situations, qualitative analysis provides a valuable tool for gaining insight
into the synaptic transmission process. The framework presented here formulates the information transmission process as a stimulus reconstruction
problem in which the likelihoods of different presynaptic spike trains are
estimated based on a given set of transmissions. This analysis, for example,
shows clearly how synaptic dynamics can increase the information transmitted across a synapse by providing information about the locations of
spikes that are not themselves transmitted (see Figure 5). For uncorrelated
input (see Figure 5A), the presence of synaptic dynamics, rather than the
exact form of the dynamics, was seen to be the essential feature that leads to
increases in information transmission. For correlated input, the dynamics
of synaptic depression were seen to be particularly well adapted to transmitting information about positively correlated inputs. In both examples,
general insights were drawn without requiring a detailed mathematical
model of the synapse. I expect that this framework will provide similar insights for systems with different spike train autocorrelations and different
synaptic dynamics.
Appendix A: Autocorrelation Function of a Stationary Renewal Process
Spike Train
Here, the normalized autocorrelation function C(τ ) of a stationary renewal
process spike train is related to the ISI distribution PISI (S ) from which the
spike train was generated. The normalized autocorrelation function C(τ )
1158
M. Goldman
for a point-process spike train s(t) is defined by
C(τ ) ≡
s(t)s(t + τ ) − r2
,
r2
(A.1)
where (·) denotes ensemble averaging. Following the notation of Cox and
Lewis (1966), C(τ ) can be expressed as
C(τ ) =
m f (τ ) − r
,
r
(A.2)
where m f (τ ) gives the probability density that some pair of events is separated by a time interval τ . m f (τ ) is expressed as a sum of probabilities fi (τ )
that the ith ISI is in the interval (τ, τ + dτ ),
m f (τ ) =
∞
fi (τ ).
(A.3)
i=1
For a renewal process, fi (τ ) is given by convolving the ISI distribution
PISI (S ) with itself i times. Laplace transforming the above equation thus
gives
f (u) =
m
∞
ISI (u)]i =
[P
i=1
ISI (u)
P
.
ISI (u)
1−P
(A.4)
Inverse Laplace-transforming and substituting into equation A.2 gives C(τ ).
For the ISI distribution given in equation 3.17 with Laplace transform
given by equation B.4, substitution into equation A.4 gives exponentially
decaying autocorrelations of the form C(τ ) = Ae−τ/τcorr , with A, τcorr , and
the average rate r given by
A=
α(1 − α)(λ− − λ+ )2
λ− λ+
(A.5)
1
(1 − α)λ− + αλ+
(A.6)
τcorr =
r=
λ− λ+
.
(1 − α)λ− + αλ+
(A.7)
It can be shown that this ISI distribution can assume any arbitrary coefficient of variation between one and ∞, depending on the choice of the
distribution parameters (Cox, 1962).
Enhancement of Information Transmission Efficiency
1159
Appendix B: Calculation of PIVI (V ) and p for the Stationary Renewal
Spike Train
Below, the IVI distributions PIVI (V ) are derived for the constant-probability
or model depressing synapse receiving input generated by a stationary renewal process.
For the constant-probability synapse, PIVI (V ) is obtained by noting that
the probability of having an IVI of duration V can be broken into two terms
as
PIVI (V ) = pPISI (V )
+ (1 − p)
0
(B.1)
V
dS PISI (S )PIVI (V − S ).
(B.2)
The first term gives the probability that the first ISI is of duration V and
the vesicle releases. The second term gives the sum over all possible ways
that an IVI of duration V could occur when the first ISI does not cause a
vesicle release.
PIVI (V ) is obtained from the Laplace transform of the above equation:
IVI (u) =
P
ISI (u)
pP
.
ISI (u)
1 − (1 − p)P
(B.3)
For the exponentially correlated spike train model described by equation
3.17, the Laplace transform of PISI (S ) is
ISI (u) = αλ− /(u + λ− ) + (1 − α)λ+ /(u + λ+ ).
P
(B.4)
Inserting the above into equation B.3 and inverse Laplace transforming
gives that PIVI (V ) is identical in form to the ISI distribution PISI (V ) (see
equation 3.17) except for the replacement of the average rate r by pr. This
result reflects that failures to transmit and failures to receive a presynaptic
spike are equivalent for the constant-probability synapse.
For the model-depressing synapse, PIVI (V ) can similarly be decomposed into two terms corresponding to an IVI of duration V resulting
from the first ISI or multiple ISIs:
PIVI (V ) = PISI (V )(1 − e−V /τD )
V
+
dS PISI (S )e−S /τD PIVI (V − S ).
(B.5)
(B.6)
0
Solving this equation by using Laplace transforms gives
−1
IVI (u) = PISI (u) − PISI (u + τD ) .
P
ISI (u + τ −1 )
1−P
D
(B.7)
1160
M. Goldman
For the exponentially correlated spike train, PIVI (V ) is obtained by substituting equation B.4 into equation B.7 and inverse Laplace transforming.
The resulting expression gives PIVI (V ) as a linear combination of four
exponential distributions with inverse time constants λ1 = λ− , λ2 = λ+ ,
−1 + τ −1 and corresponding weights γ :
λ3 = τD−1 , and λ4 = τcorr
i
D
PIVI (V ) =
4
γi λi e−λi V ,
(B.8)
i=1
where
γ1 =
αλ3 (λ2 − λ1 + λ3 )
(λ4 − λ1 )(λ3 − λ1 )
γ2 =
(1 − α)λ3 (λ1 − λ2 + λ3 )
(λ4 − λ2 )(λ3 − λ2 )
(B.10)
γ3 =
−1 − λ )
r(τcorr
3
(λ3 − λ2 )(λ3 − λ1 )
(B.11)
γ4 =
−Arλ3
.
(λ4 − λ2 )(λ4 − λ1 )
(B.12)
(B.9)
The weights γi can be negative so, unlike α in the ISI distribution (see equation 3.17), they cannot be interpreted as probabilities.
The average transmission probabilities p = N/n are obtained by substituting PISI (S ) (see equation 3.17) into equation 4.3. The result is:
p=1−
αλ1
(1 − α)λ2
−
.
λ1 + λ3
λ2 + λ3
(B.13)
For uncorrelated input (α = 0 or 1), p depends on only the product rτD . For
correlated input, p depends more generally on the form of the autocorrelations. This implies that for a given value of p, rτD differs across the graphs
shown in Figures 3 and 4. The arrows labeling τD = τcorr in Figure 4B give
an example of this dependence.
Acknowledgments
This work was supported by NIH grants MH58754 and MH60651, NSF grant
IBN-9817194, the Sloan Center for Theoretical Neurobiology at Brandeis
University, the W.M. Keck Foundation, and the Howard Hughes Medical
Institute. I thank Larry Abbott and Dan Butts for helpful discussions and
comments on the manuscript and Xiaohui Xie for helpful comments on the
manuscript.
Enhancement of Information Transmission Efficiency
1161
References
Abbott, L. F., Varela, J. A., Sen, K., Nelson, S. B. (1997). Synaptic depression and
cortical gain control. Science, 275, 220–223.
Attneave, F. (1954). Some informational aspects of visual perception. Psych. Rev.,
61, 183–193.
Baddeley, R., Abbott, L. F., Booth, M. C., Sengpiel, F., Freeman, T., Wakeman,
E. A., & Rolls, E. T. (1997). Responses of neurons in primary and inferior
temporal visual cortices to natural scenes. Proc. R. Soc. Lond. B Biol. Sci., 264,
1775–1783.
Balasubramanian, V., Kimber, D., Berry, M. J. II. (2001). Metabolically efficient
information processing. Neural Comput., 13, 799–815.
Barlow, H. B. (1961). Possible principles underlying the transformation of sensory messages. In W. A. Rosenblith (Ed.), Sensory communication (pp. 217–234).
Cambridge, MA: MIT Press.
Barlow, H. B., & Foldiak, P. (1989). Adaptation and decorrelation in the cortex. In
R. Durbin, C. Miall, & G. Mitchinson (Eds.), The computing neuron. Reading,
MA: Addison-Wesley.
Berry, M. J., & Meister, M. (1998). Refractoriness and neural precision. J. Neurosci.,
18, 2200–2211.
Brenner, N., Bialek, W., & de Ruyter van Steveninck, R. (2000). Adaptive rescaling
maximizes information transmission. Neuron, 26, 695–702.
Brenner, N., Strong, S. P., Koberle, R., Bialek, W., & de Ruyter van Steveninck,
R. R. (2000). Synergy in a neural code. Neural Comput., 12, 1531–1552.
Cox, D. R. (1962). Renewal theory. London: Methuen.
Cox, D. R., & Lewis, P. A. W. (1966). The statistical analysis of series of events.
London: Methuen.
Dan, Y., Atick, J. J., & Reid, R. C. (1996). Efficient coding of natural scenes in
the lateral geniculate nucleus: Experimental test of a computational theory.
J. Neurosci., 16, 3351–3362.
de Ruyter van Steveninck, R. R., & Laughlin, S. B. (1996). The rate of information
transfer at graded-potential synapses. Nature, 379, 642–645.
de Ruyter van Steveninck, R. R., Lewen, G. D., Strong, S. P., Koberle, R., & Bialek,
W. (1997). Reproducibility and variability in neural spike trains. Science, 275,
1805–1808.
de la Rocha, J., Nevado, A., & Parga, N. (2002). Information transmission by
stochastic synapses with short-term depression: Neural coding and optimization. Neurocomputing, 44–46, 85–90.
Fairhall, A. L., Lewen, G. D., Bialek, W., & de Ruyter van Steveninck, R. R. (2001).
Efficiency and ambiguity in an adaptive neural code. Nature, 412, 787–792.
Fuhrmann, G., Segev, I., Markram, H., & Tsodyks, M. (2002). Coding of temporal
information by activity-dependent synapses. J. Neurophysiol., 87, 140–148.
Goldman, M. S. (2000). Computational implications of activity-dependent neuronal
processes. Unpublished doctoral dissertation, Harvard University.
Goldman, M. S., Maldonado, P., & Abbott, L. F. (2002). Redundancy reduction
and sustained firing with stochastic depressing synapses. J. Neurosci., 22,
584–591.
1162
M. Goldman
Laughlin, S. B. (2001). Energy as a constraint on the coding and processing of
sensory information. Curr. Opin. Neurobiol., 11, 475–480.
Laughlin, S. B., de Ruyter van Steveninck, R. R., & Anderson, J. C. (1998). The
metabolic cost of neural information. Nat. Neurosci., 1, 36–41.
Levy, W. B., & Baxter, R. A. (2002). Energy-efficient neuronal computation via
quantal synaptic failures. J. Neurosci., 22, 4746–4755.
London, M., Schreibman, A., Hausser, M., Larkum, M. E., & Segev, I. (2002). The
information efficacy of a synapse. Nat. Neurosci., 5, 332–340.
Maass, W., & Zador, A. M. (1999). Dynamic stochastic synapses as computational
units. Neural Comput., 11, 903–917.
Manwani, A., & Koch, C. (2001). Detecting and estimating signals over noisy
and unreliable synapses: Information theoretic analysis. Neural Comput., 13,
1–33.
Matveev, V., & Wang, X.-J. (2000). Differential short-term synaptic plasticity and
transmission of complex spike trains: To depress or to facilitate? Cereb. Cortex,
10, 1143–1153.
Murthy, V. N., Sejnowski, T. J., & Stevens, C. F. (1997). Heterogeneous release
properties of visualized individual hippocampal synapses. Neuron, 18, 599–
612.
Natschlager, T., & Maass, W. (2001). Computing the optimally fitted spike train
for a synapse. Neural Comput., 13, 2477–2494.
Schreiber, S., Machens, C. K., Herz, A. V. M., & Laughlin, S. B. (2002). Energyefficient coding with discrete stochastic events. Neural Comput., 14, 1323–1346.
Shannon, C., & Weaver, W. (1949). The mathematical theory of communication. Urbana: University of Illinois Press.
Srinivasan, M. V., Laughlin, S. B., & Dubs, A. (1982). Predictive coding: A fresh
view of inhibition in the retina. Proc. R. Soc. Lond. B. Biol. Sci., 216, 427–459.
Stevens, C. F., & Wang, Y. (1995). Facilitation and depression at single central
synapses. Neuron, 14, 795–802.
Strong, S. P., Koberle, R., de Ruyter van Steveninck, R. R., & Bialek, W. (1998).
Entropy and information in neural spike trains. Phys. Rev. Lett., 80, 197–200.
Tsodyks, M. V., & Markram, H. (1997). The neural code between neocortical
pyramidal neurons depends on neurotransmitter release probability. Proc.
Natl. Acad. Sci. U.S.A., 94, 719–723.
Varela, J., Sen, K., Gibson, J., Fost, J., Abbott, L. F., & Nelson, S. B. (1997). A
quantitative description of short-term plasticity at excitatory synapses in
layer 2/3 of rat primary visual cortex. J. Neurosci., 17, 7926–7940.
Vere-Jones, D. (1966). Simple stochastic models for the release of quanta of transmitter from a nerve terminal. Aust. J. Stat., 8, 53–63.
Wainwright, M. J. (1999). Visual adaptation as optimal information transmission.
Vision Res., 39, 3960–3974.
Wang, X. J., Liu, Y., Sanchez-Vives, M. V., & McCormick, D. A. (2003). Adaptation
and temporal decorrelation by single neurons in the primary visual cortex.
J. Neurophysiol., 89, 3279–3293.
Zador, A. (1998). Impact of synaptic unreliability on the information transmitted
by spiking neurons. J. Neurophysiol., 79, 1219–1229.
Received May 15, 2003; accepted November 6, 2003.