Yet another method for calculating information from neural data. P.E.

Yet another method for
calculating information
from neural data.
P.E. Latham, S.M. Carcieri,
A.L. Jacobs and S. Nirenberg
University of California at
Los Angeles
52.7
SFN 2000
Mutual information, I, gives us a powerful
tool answering the question:
what aspects of a spike train are important?
I = -∑ P(R) log P(R) + ∑ P(S) ∑ P(R|S) log P(R|S),
R
S
R
= H(R) - H(R|S)
S
R
H(R)
H(R|S)
= stimulus,
= response (typically spike trains),
= entropy of responses,
= conditional entropy.
For example:
S = pictures shown to a monkey
R = neuronal response in V1, consisting of:
A. R = spike count in
300 ms window
I (bits) = 1.2 ± 1.0
B. R = precise spike
times in 300 ms
window
I (bits) = 3.4 ± 2.6
Spike timing
is important
Well, maybe not ...
The moral: to make a meaningful
comparison, one must
A. Accurately estimate entropy,
B. Provide error bars.
The holy grail:
•
Given data, D, find an unbiased
^
estimator, H(D), such that
〈H(D)〉 = Htrue
^
where 〈...〉 indicates an average over
experiments.
•
Compute the variance of the
estimator.
Why is this a hard problem?
The short answer:
when you collect data from a distribution
with long tails, you don’t sample the tails.
nj
true log pj
0
0
5000
10000
j
^
Hnaive = -∑ (nj /N) log(nj /N) = 5.8 bits (N=1000)
H true = -∑ pj log pj
= 8.6 bits
An example
letter
word
{
2 0 1 0 2 0 1 1 2 0 0 0 1 0 0 0 0 0 0 0 1 2 0 0 1
construct histogram:
word
frequency
00000
47
00001
18
00010
29
...
23114
1
word frequency
200
^
Hnaive = 1.9 bits
(N=300)
Htrue = 2.4 bits
true log pj
0
0
100
j
200
Bias in naive estimate of entropy
50 ms words, 10 ms letters, inhomogeneous
Poisson process, mean rate = 10 Hz
^
Bias in Hnaive
(percent)
20
10
0
0
2000
4000
N
word
1
2
3
...
~1-500
For conditional
entropy, you need to
present the same
stimulus many times.
"Many" corresponds
to several hundred.
That’s because each
stimulus is several
seconds long, and
recordings last ~hours.
Observations:
1.
Small errors in entropy can cause large errors
in mutual information, since mutual information
is the difference between two entropies.
2.
There is no unbiased estimator of entropy, since
a finite amount of data does not tell you about
the tails of the distribution.
3.
There is no estimator with a consistent upward
bias, since the entropy can be infinite (e.g.,
pj ~ 1/jlog 2 j ).
What one can do:
1.
Know your data -- if you know something about
the tails of the distribution, i.e., how pj behaves
as j → ∞, there is hope of coming up with a
decent estimator.
2.
Strategy: find an estimator that provide, on
average, lower and upper bounds on the entropy.
rigorous
model dependent
Our estimators
Data:
nj = number of occurrences of element j (e.g., word j).
Definitions:
N = ∑j nj = total number of elements sampled.
M = number of non-zero n j .
M1 = number of n j = 1.
Example:
nj = (4, 5, 0, 1, 4, 1, 0, 2)
N = 17
M= 6
M1 = 2
Estimators (LB = lower bound; UB = upper bound):
^
naive
^
^
}
HUB = HLB + M1 logN/N
pulled more or less
out of thin air.
}
}
HLB = -∑ (nj /N) log (nj /N) + (M-1)/2N ln 2
Treves and Panzeri
correction (NC 7, 1995)
^
bias = 〈 H 〉 - H true
= ∑ H(n) P(n|p) - (-∑ pj log p)
j
{n}
j
P(n|p) = multinomial = N! ∏ pjnj /nj !
j
A little algebra:
^
-1
〈 HLB 〉 - Htrue = N ∑ f(pj )
j
^
-1
〈 HUB 〉 - Htrue = N ∑ g(pj )
j
f(p) = ∑ n log (n/N) P(n|p) + [1-p-exp(Nlog(1-p))]/2 ln 2
n
g(p) = f(p) + (log N) Np exp((N-1)log(1-p))
binomial = p n(1-p) N-n N!/n!(N-n)!
Upper and lower bounds
N=1000
0.0035
g(p)
0
-6
-9
10
10
0.000
f(p)
-0.001
-9
10
-6
-3
10
10
0
10
p
Observations:
1. g(p) and f(p) have a min and max at p ≈ 1/N.
2. f(p) is never positive.
3. g(p) goes negative at p ≈ 1/N 2.
0
-9
10
Recall:
-6
10
Does this little negative
portion on g(p) (a supposed
upper bound) matter?
-1
^
〈 HUB〉 - Htrue = N ∑ g(pj )
j
∑ g(pj ) → ∫ dj g(pj ) = ∫ d log p g(p) d log p/dj
-1
j
Everything depends on how pj depends on j.
A reasonable model for the words/letters approach
to computing entropy: if n is the number of letters
(word length), and the spike train has no temporal
correlations, then (see last page of poster):
pj ≈ p exp[-(p n!j) 1/n]
0
0
g(p) d log p/dj
-1
N=100
10
n=10
n=5
0
n=1
-10
-12
10
-6
10
0
10
∫ d log p g(p) d log p/dj
-1
p
2
0
N=100
N=1000
-14
0
8
n (word length)
16
50 ms words, 10 ms letters, inhomogeneous
Poisson process with correlations,
mean rate = 10 Hz
lower bound
upper bound
N=100
Relative frequency
20
10
10
0
0
0
2
4
Htrue
N=1000
20
0
10
0
0
2
4
Htrue
2
4
Htrue
N=10000
20
10
0
N=500
20
0
Entropy
2
4
Htrue
100 ms words, 10 ms letters, inhomogeneous
Poisson process with correlations,
mean rate = 10 Hz
lower bound
upper bound
N=100
Relative frequency
20
10
10
0
0
0
4
8
Htrue
N=1000
20
0
10
0
0
4
8
Htrue
4
8
Htrue
N=10000
20
10
0
N=500
20
0
Entropy
4
8
Htrue
Open questions:
1. How well does this method work with
real data? Answering this will require
extensive simulations with surrogate
data designed to match the properties
of real data.
2. How well can we estimate the variance?
Our current estimates for the variance
tend to be too small by 10-30%. Better
estimates are needed.
3. Are there better estimators for the upper
and lower bounds than the ones presented
here?
"!#$&%')(+*-,/.01235476189$:;"-=<
>1?@BACDFE7GHC@+@JIKCLM?ONQPRSL?E2G5TIDUC@WVGHCXG5A@YCBZHDUC[D\A^]YLM_]YD\GÒC?aQI?HbcCMd@+NL?Ne@BLCMDU@0A-?RCMd@"_IV@JLM`FPKDFIfNL?gGgDU`FDhCP
VD\ACMLDFg_CMDU?IiRSLM?OEjb dDF]XdCd@kVGHCMGlbWGOAmAMGEkN`F@BVon7pQN)@B]YDUTe]BG`FÙPZoRS?OLqGrNLM?OgGOgDU`FDUCPsVDFACLMDFg_CDF?OI1ZtuZ1Cd@k@JIKCML?NKP
]Y?EkN_C@0VrRSL?EvGwTIDUC@xVGHCXGwA@YCV@JN)@JIVAACML?IfO`FP&?IrCMd@;g)@JdGzyQDF?OL ?R=tu;b d@BI:{7D\A `FGOLf@On5|~}5@JLM@qGIeVrDFIb dGC
RS?O`F`F?Hb5Ab@xLXGIa?OLXV@BL5Cd@tuOZ1A?&CdeGHC tu[&/tunx>WdQ_ABZ-`\GLMfO@m{]Y?OLMLM@BANe?IVAC?rCd@wCMGODU`\A?R+Cd@7VD\ACMLDFg_CMDU?I
tunwAWb@Ad?Hb@BV7DFI:CMd@Ne?KACM@JL0ZCMd@m@BACDFE7GHC@RS?LGOI:_NN)@JL g)?O_IV:?OR-Cd@@BIKCLM?ONQPkge@0]Y?OEk@0A+b?OLXA@GOA"Cd@CXGDF`FA
?R[Cd@qVDFACLMDUg_CDF?OIg)@B]Y?Ek@)GHCCM@JL0n?OLM@J?Hy@JL0ZORS?LWCMGDF`\ACdGCGLM@CM?K?k)GHCBZQCMd@;_NN)@JLge?_IVl]JGI@ByO@JI:C_LMIDUIKCM?
GwÙ?Hb@JLWg)?O_IeVoZLM@JIV@BLDFIfxCd@;@0ACMDUE7GC@;_A@J`F@BAMAJn
}5@BL@b@l@0ACMDUE7GC@lCd@sV@JN)@JIV@BI]Y@?OR+t u ?OI{'b d@BIct u GOL@lb+?LMVNLM?OgGOgDU`FDUCDF@BABZ+GOAk]BG`\]Y_`\GHCM@BV_ADFIfCd@
VDULM@B]YCM7Ek@YCMd?VUY |A@J@xGO`FA?&N)?AC@BLNGOfO@xnc@waKI?Hb2?ORI?&LMDUf?OLM?O_A5bGzPrC?r@BACDFEkGC@t u DUIfO@BI@JLXG`ZeA?&b@
NLM?HyQDFV@rGsLM@BG`F`FPcLM?O_fOd@BACDFE7GHCM@On'@V?iCdD\AwgKPGAA_EkDUIfCdGCxCMd@JLM@GLM@&I?sCM@JEkN)?OLXG`W]Y?OLMLM@J`\GHCDF?OIeAqDFICd@
ANDUa@5CMLMGODUI-QCMdGHCWD\AJZb@GONNLM?zDUE7GHCM@ Cd@mANDFaO@]Y?O_ICWNLM?OgGOgDFÙDUCP7DUIr@BG]Xd:gDFIrGA+DFIV@BNe@BIV@BIC ?OR[G`F`=?Cd@JLWgDFIAJn
-@JCCMDUIfmt=|ge@mCMd@;NLM?OgGOgDU`FDUCP7?R?OgA@JLMyQDUIfkANDUa@BAWDFIlgDUIs |S?O_C5?R[¡cgDFIAXZCMd@;DUIeV@JN)@JIV@JI]J@qGOAMA_EkNCMDU?I
G`F`F?Hb5A+_eAWC?kb LMDhCM@
¤
t[|~ z¢B£U£F£U¢ ¥¤¦ § t=Q|S
|z
Y¨
b d@BL@5t^| H¢J£F£U£F¢ ¥¤DFACd@xNLM?OgeGgDFÙDUCPl?R?OgeA@BLyQDFIf:SkANDUa@BADUIigDUI' =Z- l¦© ¢B£U£F£U¢ ¡Z)GOIV@0GO]Xds?ORCd@t[| ¢J£F£F£U¢ ¥¤Q
]Y?LLM@BAN)?OIVACM?&G&VD\ACMDUIe]C"t u nW>[?kTIVs?O_Cd?Hbt u V@JN)@JIVA?OI&{eZeb@mTLXACA?Ùy@CMd@ªDFIQyO@BLMA@mNL?g`F@JE©«lb+@mTIeV
{)|¬t=ZCMd@;IQ_Eªg)@JL ?OR^b?OLXVAb DhCMdNL?gGgDU`FDhCP:fOLM@BGHCM@JL+CdGOIktl«&GOIV:Cd@BIlDFIQyO@BLC"{)|¬t=+C?wTIVwt u n
>Wd@ªK_GOICMDhCP7{)|¬t=WDFA5Cd@ªIK_Eªg)@JL5?OR]J?OITefO_LXGHCMDU?IAW?RCd@ªSkA_e]XdCdGC"t[|~ z¢J£F£F£U¢ ¥¤ ®t1n >^GaQDFIfkCd@ªÙ?f7?R
g)?CdADFV@BA GIeV:_ADFIf&¯")n-|zZCMdD\AW@YNLM@BAMADF?OIlge@0]Y?Ew@0A
° `F?Of-t | ² Ù?f-t £
|~³
©±
±
>[?'V@YC@BLEkDFI@Cd@sIQ_Eªg)@JL7?ORm]J?OITf_LXGHCDF?OIeAx?ORCMd@i CdGCrAMGHCD\AT@BA7¯+en5|³OYZ{)|ht)ZWDUC&DFA&_A@JRS_`5CM?cCdDFIa´?R
| `F?Of-toH|~Y ¢ Ù?foteµK|¥µB ¢J£F£U£F¢ `F?Of-t ¤ | ¤ :GA&N)?ODFICXA:DFI¶GOI·¡=VDUEk@BIADF?OIeGÀNG]Y@On¸>Wd@JI1Z+{)|¬t):D\ArADUEkN`FPĆd@
IQ± _Eªg)@JLW?OR[N)?O± DFICXAWge@BÙ?Hb·Cd@mN± `\GI@ª¹ ^º ¦ `F?Ofot[Zb d@JLM@CMd@ º GLM@]J?Q?OLXVDUIeGHC@0ADUIrCd@;AMGEk@¡=VDFEk@JIADF?OIGO`
±
ANGO]J@mGACd@ `F?Ofot | Yn
@xI?Hb2Ek± GOaO@ªG7A@B]J?OIVGONNLM?zDUE7GHCMDU?I-ZCMdGHC5CMd@ªN)?ODFIKCMANe?DUIKCMAx| `F?Of-toH|J ¢ `F?Ofot)µ|~¥µB ¢1£F£U£F¢ `F?Of-t ¤ | ¤
GLM@sGNNL?zDFE7GHC@BÙP@Jy@JI`FPÁNG]Y@BV-CMdGHClDFABZ Ù?f1t |~ `F?Of1t |»¸zs± ¦½¼ ZGONN± LM?zDUE7GHCM@J`FPDFIV± @JN)@JIeV@JIKC?R
n'¾ICdGCk]BGOA@OZ1{)|¬t=qD\AªNL?Ne?LCMDU?IG`"C?Cd@lyO?Ù_Ek@r± ge?_IV@0VgQP'Cd@l¡i»¿ldQPQNe@BLN`\GI@0A ¹ 1º c¦ `F?Of1t
GIeV º ¦ `F?Ofot=|ÀKZ+ Á¦Â ¢J£F£F£U¢ ¡Zb DhCMdCMd@]Y?OIeACXGIKCx?ORNLM?ON)?OLCDF?OIGOÙDUCP'@BK_G`+C?Á ¡ÃXÄ ¼wzÅ nÆ@Y± TIDUIf
Ä ¼ ;Ç tÈ± Zb+@mCMdK_eAWdGzyO@
¤
{)|¬t="¦ ± Ù?f-tx» ¡¹ Ã te È Ù?f-t |À £
|ÉK
¾IQyO@BLCMDUIfxCdDFA @JNL@0AADF?OIrGOIVl@JIRS?LM]JDUIfªCMd@;I?OLME7G`FDUÊ0GHCMDU?IlËÈj
Ì Í {-t u5Î OZb@;GLMLDFyO@mGHC
tu Î teÈ-@YN1 |¬teÈJ¡Ã {K Ï ¤ £
|SÐQ
±
>WdD\A5@YNLM@BAMADU?I-Zb dD\]XdDFA D\V@BICMDF]BG`1C?kCd@q?OI@ª?OINGf@70³w?OR[CMd@qN)?AC@JL0ZeAd?_`FVlNLM?HyQDFV@qGÑÒHÓOÔzÕw@BACDFE7GHCM@
?R)tuOn
Ö -$X-1!×=1
n pQCLM?OIfeZopon ØnFZoÙm?Og)@JLMÙ@Z=ÚqnUZ-V@wÚ _PKC@BLmyzGOIcpKCM@Jy@JIDFI]Xa=ZoÚ;n Úqn=Û©ÜD\G`F@Ja=Z=¿no¯IKCLM?ONQPGIViDUIRS?LE7GCDF?OIsDFI
I@J_LXG`-ANDFaO@CLXGDFIABnÝ"ÕQÞzßJà1áâYãHàoä1âJå~åà"æ)ç1ZoBèKéz³OÀOÀr|BèèOêKn

Download Report

Yet another method for calculating information from neural data. P.E.

Paperzz.com

Your Paperzz