Neuron
1
The detector model.
2
Biological properties of the neuron.
3
The computational unit.
1 / 29
Detector Model
Each neuron is detecting some set of conditions
(e.g., smoke detector). Representation is what is detected.
Neurons feed on each other’s outputs — layers of ever
more complicated detectors.
(Things can get very complex in terms of content, but each
neuron is still carrying out basic detector function).
output
integration
inputs
Detector
2 / 29
Detector vs. Computer
Memory &
Processing
Computer
Separate,
general-purpose
Detector
Integrated,
specialized
Operations
Logic, arithmetic
Detection
(weighing & accumulating
evidence, evaluating,
communicating)
Complex
Processing
Arbitrary sequences
of operations chained
together in a program
Highly tuned sequences
of detectors stacked
upon each other in layers
3 / 29
Understanding Neural Components in Detector Model
output
axon
cell body,
membrane
potential
integration
inputs
dendrites
synapses
Detector
Neuron
4 / 29
A Real Neuron
5 / 29
Basic Properties of a Neuron
It’s a cell: body, membrane, nucleus, DNA, RNA, proteins,
etc.
Membrane has channels, passing ions (salt water).
Cell has electrical potential (voltage), integrated in cell
body, activates action potential output in axon, releases
neurotransmitter.
Neurotransmitter activates potential via dendritic synaptic
input channels.
Excitation and inhibition are transmitted by different
neurons!
6 / 29
The Synapse
Dendrite
Spine
Receptors
mGlu AMPA
NMDA
Postsynaptic
Neuro−
transmitter
(glutamate)
Na+
Ca++
Ca++
Cleft
Presynaptic
Vesicles
Terminal
Button
Axon
Microtubule
7 / 29
Weight = Synaptic Efficacy
Synaptic efficacy = activity of presynaptic (sending) neuron
communicated to postsynaptic (receiving) neuron:
Presynaptic: # of vesicles released, NT per vesicle,
efficacy of reuptake mechanism.
Postsynaptic: # of receptors, alignment & proximity of
release site & receptors, efficacy of channels, geometry of
dendrite/spine.
Drugs: Prozac (serotonin reuptake), L-Dopa (NT in vesicles)
8 / 29
Neurophysiology
The neuron is a miniature electro-chemical system:
1
Balance of electric and diffusion forces.
2
Equilibrium potential.
3
Principal ions.
4
Integration equations.
5
The overall equilibrium potential.
9 / 29
Balance of Electric and Diffusion Forces
Ions flow into and out of the neuron under the forces of
electricity and concentration gradients (diffusion).
The net result is a electric potential difference between the
inside and outside of the cell — the membrane potential V m .
This value represents an integration of the different forces, and
an integration of the inputs impinging on the neuron.
We will use the equations describing this integration in our
models.
10 / 29
Electricity
Positive and negative charge (opposites attract, like repels).
Ions have net charge: Sodium (Na+ ), Chloride (Cl − ),
Potassium (K + ), and Calcium (Ca++ ) (brain = mini ocean).
Current flows to even out distribution of positive and negative
ions.
Disparity in charges produces potential (the potential to
generate current..)
11 / 29
Resistance
Ions encounter resistance when they move.
Neurons have channels that limit flow of ions in/out of cell.
+
V
−
+
−
G
I
The smaller the channel, the higher the resistance, the greater
the potential needed to generate given amount of current
(Ohm’s law):
V
(1)
I=
R
Conductance G = 1/R, so I = VG
12 / 29
Diffusion
Constant motion causes mixing – evens out distribution.
Unlike electricity, diffusion acts on each ion separately — can’t
compensate one + ion for another..
(same deal with potentials, conductance, etc)
I = −DC
(2)
(Fick’s First law)
13 / 29
Equilibrium
Balance between electricity and diffusion:
E = Equilibrium potential = amount of electrical potential
needed to counteract diffusion:
I = G(V − E)
(3)
Also:
Reversal potential (because current reverses on either side of
E)
Driving potential (flow of ions drives potential toward this value)
14 / 29
The Neuron and its Ions
Inhibitory
Synaptic
Input
Cl−
−70
Leak
Excitatory Vm Cl−
Synaptic
Input
K+
Na+
Na/K
Pump
+55
Vm
Vm
Na+
Vm
K+
−70
−70mV
0mV
Everything follows from the sodium pump, which creates the
“dynamic tension” (compressing the spring, winding the clock)
for subsequent neural action.
15 / 29
Drugs and Ions
Alcohol: closes Na
General anesthesia: opens K
Scorpion: opens Na and closes K
Some kind of venom: closes all muscle firing
(acetylcholine)
16 / 29
Putting it Together
Ic = gc (t)g¯c (Vm (t) − Ec )
(4)
e = excitation (Na+ )
i = inhibition (Cl − )
l = leak (K + ).
Inet
= ge (t)g¯e (Vm (t) − Ee ) +
gi (t)ḡi (Vm (t) − Ei ) +
gl (t)ḡl (Vm (t) − El )
(5)
Vm (t + 1) = Vm (t) − dtvm Inet
(6)
Vm (t + 1) = Vm (t) + dtvm Inet−
(7)
or
17 / 29
In Action
40− −30−
30− −35−
20− −40−
10− −45−
0− −50−
−10− −55−
−20−
−30−
−40−
I_net
g_e = .4
−60−
−65−
g_e = .2
V_m
−70−
0
5
10
15
20
cycles
25
30
35
40
(Two excitatory inputs at time 10, of conductances .4 and .2)
18 / 29
Overall Equilibrium Potential
If you run Vm update equations with steady inputs, neuron
settles to new equilibrium potential.
To find, set Inet = 0, solve for Vm :
Vm =
ge g¯e Ee + gi ḡi Ei + gl ḡl El
ge g¯e + gi ḡi + gl ḡl
(8)
Can now solve for the equilibrium potential as a function of
inputs.
Simplify: ignore leak for moment, set E e = 1 and Ei = 0:
Vm =
ge g¯e
ge g¯e + gi ḡi
(9)
Membrane potential computes a balance (weighted average) of
excitatory and inhibitory inputs.
19 / 29
Equilibrium Potential Illustrated
Equilibrium V_m by g_e (g_l = .1)
1.0
V_m
0.8
0.6
0.4
0.2
0.0
0.0
0.2
0.4
0.6
0.8
g_e (excitatory net input)
1.0
20 / 29
Computational Neurons (Units)
1
Really abstract: The standard sigmoidal function.
2
More neuro: The point neuron function.
3
Two kinds of outputs: discrete spiking, rate coded.
21 / 29
Computational Neurons (Units) Overview
yj ≈
Vm =
g g E +gg E +ggE
e e e
i i i
l l l
g g +gg +gg
e e i i
l l
γ [Vm− Θ]
+
γ [Vm− Θ] + 1
+
β
net = g ≈ <x i w ij > +
e
N
w ij
xi
1
Weights = synaptic efficacy; weighted input = x i wij .
2
Net conductances (average across all inputs)
excitatory (net = ge (t)), inhibitory gi (t).
3
Integrate conductances using Vm update equation.
4
Compute output yj as spikes or rate code.
22 / 29
Standard Sigmoidal Function
First compute weighted, summed net input: η j =
P
Then pass this through a sigmoidal function: y j =
i
xi wij
1
−η
1+e j
Sigmoidal Activation Function
1.0
Activation
0.8
0.6
0.4
0.2
0.0
−4
−2
0
Net Input
2
4
23 / 29
Computing Excitatory Input Conductances
g ≈Σ
e
1
β
N
b 1<
xw >
s
i ij
a+b α
xw
i ij
1<
s a
xw >
i ij
a+b α
xw
i ij
B
A
Projections
One projection per group (layer) of sending units.
Average weighted inputs hxi wij i =
1
n
P
i
xi wij .
Bias weight β: constant input.
24 / 29
Computing Vm
Use Vm (t + 1) = Vm (t) + dtvm Inet− with
biological or normalized (0-1) parameters:
Parameter
Vrest
El (K + )
Ei (Cl − )
Θ
Ee (Na+ )
mV
-70
-70
-70
-55
+55
(0-1)
0.15
0.15
0.15
0.25
1.00
Normalized used by default.
25 / 29
Thresholded Spike Outputs
Voltage gated Na+ channels open if Vm > Θ, sharp rise in Vm .
Voltage Gated K + channels open to reset spike.
−30−
1−
Rate Code
Spike
−40− 0.5−
−50−
0−
act
−60−
−70− −0.5−
−80−
−1−
0
V_m
5
10
15
20
25
30
35
40
In model: yj = 1 if Vm > Θ, then reset (also keep track of rate).
26 / 29
Rate Coded Output
Output is average firing rate value.
One unit = % spikes in population of neurons?
x
Rate approximated by X-over-X-plus-1 ( x+1
):
yj =
γ[Vm (t) − Θ]+
γ[Vm (t) − Θ]+ + 1
(10)
which is like a sigmoidal function:
yj =
1
1 + (γ[Vm (t) − Θ]+ )−1
compare to sigmoid: yj =
(11)
1
−η
1+e j
γ is the gain: makes things sharper or duller.
27 / 29
Convolution with Noise
X-over-X-plus-1 has a very sharp threshold
activity
Smooth by convolve with noise (just like “blurring” or
“smoothing” in an image manip program):
Θ
Vm
28 / 29
Fit of Rate Code to Spikes
50− 1−
40− 0.8−
30− 0.6−
spike rate
noisy x/x+1
20− 0.4−
10− 0.2−
0− 0−
−0.005
0
0.005
V_m − Q
0.01
0.015
29 / 29