CSE/NB 528
Final Lecture: All Good Things Must…
R. Rao, 528: Final Lecture
Lecture figures are from Dayan & Abbott’s book
http://people.brandeis.edu/~abbott/book/index.html
1
Course Summary (All good things must…come to an end)
✦
Where have we been?
¼ Course Highlights
✦
Where do we go from here?
¼ Challenges and Open Problems
✦
Further Reading
R. Rao, 528: Final Lecture
2
What is the neural code?
Cell A
Cell B
Cell C
Cell D
Cell E
Cell F
Cell G
Cell H
Cell I
Cell J
6
8
10
Time (s)
12
14
What is the nature of the code?
How should we represent the spiking output?
single cells vs populations
rates vs spike times vs intervals
What features of the stimulus does the neural system represent?
R. Rao, 528: Final Lecture
3
Encoding and decoding neural information
Encoding: building functional models of neurons/neural systems
and predicting the spiking output given the stimulus
Decoding: what can we say about the stimulus given what we
observe from the neuron or neural population?
R. Rao, 528: Final Lecture
4
Highlights: Neural Encoding
spike-triggering stimulus features
f1
multidimensional
decision function
x1
stimulus X(t)
spiking output ρ(t)
f2
x2
f3
x3
R. Rao, 528: Final Lecture
5
Highlights: Finding the feature space of a
neural system
Gaussian prior
stimulus distribution
covariance
STA
R. Rao, 528: Final Lecture
Spike-conditional
distribution
6
Decoding: Signal detection theory
z
p(r|-)
p(r|+)
<r>-
<r>+
Decoding corresponds to comparing test to threshold.
α(z) = P[ r ≥ z|-]
false alarm rate, “size”
β(z) = P[ r ≥ z|+]
hit rate, “power”
R. Rao, 528: Final Lecture
7
Close correspondence between neural performance
and behaviour..
R. Rao, 528: Final Lecture
8
Decoding from a population
e.g. cosine tuning curves
RMS error in estimate
R. Rao, 528: Final Lecture
9
Theunissen & Miller, 1991
More general approaches: MAP and ML
MAP: s* which maximizes p[s|r]
ML:
s* which maximizes p[r|s]
Difference is the role of the prior: differ by factor p[s]/p[r]
For cercal data:
R. Rao, 528: Final Lecture
10
Highlights:
Information
maximization
as a design
principle of the
nervous system
R. Rao, 528: Final Lecture
11
Biophysical Models
R. Rao, 528: Final Lecture
12
Excitability is due to the properties of ion channels
• Voltage dependent
• transmitter dependent (synaptic)
• Ca dependent
R. Rao, 528: Final Lecture
13
Highlights: The neural equivalent circuit
Ohm’s law:
and Kirchhoff’s law
Capacitive
current
R. Rao, 528: Final Lecture
Ionic currents
Externally
applied current
14
Highlights: Compartmental Models
Neuronal structure
can be modeled
using electrically
coupled
compartments
R. Rao, 528: Final Lecture
Coupling
conductances
15
Simplified neural models
A sequence of neural models of increasing complexity
that approach the behavior of real neurons
Integrate and fire neuron:
subthreshold, like a passive membrane
spiking is due to an imposed threshold at VT
Spike response model:
subthreshold, arbitrary kernel
spiking is due to an imposed threshold at VT
postspike, incorporates afterhyperpolarization
Simple model:
complete 2D dynamical system
spiking threshold is intrinsic
R. Rao, 528: have
Final Lecture
to include a reset potential
16
Simplified Models: Integrate-and-Fire
V
Integrate-andFire Model
τm
dV
= −(V − EL ) + I e Rm
dt
If V > Vthreshold Æ Spike
Then reset: V = Vreset
R. Rao, 528: Final Lecture
17
Simplified Models: Spike response model
Keat, Reinagel and Meister
R. Rao, 528: Final Lecture
18
Connecting neurons: Synapses
Presynaptic voltage spikes cause
neurotransmitter to cross the cleft,
triggering postsynaptic receptors
allowing ions to flow in,
changing postsynaptic potential
Glutamate: excitatory
GABAA: inhibitory
R. Rao, 528: Final Lecture
19
Synaptic voltage changes
Size of the PSP is a measure of synaptic strength.
Can vary on the short term due to input history
on the long term due to synaptic plasticity
.. one way to build circuits that learn
R. Rao, 528: Final Lecture
20
Modeling Networks of Neurons
dv
τ
= − v + F ( Wu )+ Mv)
dt
Output
Decay
Input
Feedback
R. Rao, 528: Final Lecture
21
Highlights: Unsupervised Learning
✦
v = w T u = uT w
dw
Basic Hebb Rule:
τw
= uv
dt
✦
Average effect over many inputs:
✦
For linear neuron:
τw
✦
dw
= uv = Qw
dt
Hebb rule performs principal
component analysis (PCA)
Q is the input correlation matrix:
Q = uuT
R. Rao, 528: Final Lecture
w
22
Highlights: The Connection to Statistics
Unsupervised learning = learning the hidden causes of input data
Causes v
p[ v | u; G ]
G = (µv, σv)
(posterior)
Causes of
clustered
data
Recognition
Use EM
model
Generative
model
algorithm for
learning
p[u | v; G ]
Data u
R. Rao, 528: Final Lecture
“Causes”
of natural
images
(data likelihood)
23
Highlights: Generative Models
Droning lecture
R. Rao, 528: Final Lecture
Lack of sleep
Mathematical
derivations
24
Highlights: Supervised Learning
Backpropagation for Multilayered Networks
vim = g (∑ Wij g (∑ w jk ukm ))
j
k
Goal: Find W and w that minimize errors:
x mj E (Wij , w jk ) =
1
( d im − vim ) 2
∑
2 m ,i
Desired output
u
m
k
Gradient descent learning rules:
Wij → Wij − ε
∂E
∂Wij
w jk → w jk − ε
R. Rao, 528: Final Lecture
(Delta rule)
m
∂E
∂E ∂x j (Chain rule)
= w jk − ε m ⋅
∂w jk
∂x j ∂w jk
25
Highlights: Reinforcement Learning
✦
Learning to predict rewards:
w → w + ε (r − v )u
✦
Learning to predict delayed
rewards (TD learning):
(http://employees.csbsju.edu/tcreed/pb/pdoganim.html)
w(τ ) → w(τ ) + ε [r (t ) + v (t + 1) − v(t )] u (t − τ )
✦
Actor-Critic Learning:
¼ Critic learns value of each state
using TD learning
¼ Actor learns best actions based
on value of next state (using
the TD error)
R. Rao, 528: Final Lecture
2.5
1
26
The Future: Challenges and Open Problems
✦
How do neurons encode information? Rate-Based versus
Temporal Coding
¼ Topics: Synchrony, Spike-timing based learning, Dynamic synapses
✦
Does a neuron’s structure confer computational advantages?
¼ Topics: Role of dendrites, plasticity in channels and their density
✦
How do networks of neurons implement computational
principles such as efficient coding and Bayesian inference?
✦
How do networks of neurons learn “optimal” representations of
their environment and engage in purposeful behavior?
¼ Topics: Unsupervised/reinforcement/imitation learning in the brain
✦
Applications: Machine learning, robotics, brain-machine interfaces
R. Rao, 528: Final Lecture
27
Further Reading (for the summer and beyond)
✦
Spikes: Exploring the Neural Code, F. Rieke et
al., MIT Press, 1997
✦
The Biophysics of Computation, C. Koch, Oxford
University Press, 1999
✦
Large-Scale Neuronal Theories of the Brain, C.
Koch and J. L. Davis, MIT Press, 1994
✦
Probabilistic Models of the Brain, R. Rao et al.,
MIT Press, 2002
✦
Bayesian Brain, K. Doya et al., MIT Press, 2007
✦
Reinforcement Learning: An Introduction, R.
Sutton and A. Barto, MIT Press, 1998
R. Rao, 528: Final Lecture
28
Next meeting: Project presentations!
✦
Project presentations will be on Monday, June 4 (10:30am12:20pm) in the same classroom
✦
Keep your presentation short: 8-10 slides, 15 mins/group
✦
Slides:
¼ Email Raj your powerpoint (or other) slides before 8am
on Monday, June 4 to use the class laptop
OR
¼ Bring your own laptop if you want to show videos etc.
✦
Projects reports (10-15 pages) also due same day (by email
to both Raj and Adrienne before end of the day)
✦
No classes next week – work on your projects!
R. Rao, 528: Final Lecture
Have a
great
summer!
R. Rao, 528: Final Lecture
29
Ne
Net ural
wor
for ks
Dum
mie
s
Au revoir!
30