Eleanor Fraser
NS248 Problem Set 7
February 23, 2010
1. a)
[
]
H ( X ) = # " p( x ) * log 2 ( p( x )) =
x
["1/8 * log2 (1/8)] + ["1/2 * log 2 (1/2)] + ["1/4 * log2 (1/4)] + ["1/8 * log 2 (1/8)] =
["1/8 * "3] + ["1/2 * "1] + ["1/4 * "2] + ["1/8 * "3] =
!
!
!
!
!
1
1
1
1
b) 1.75 " L = * l1 + * l2 + * l3 + * l4
8
2
4
8
l1 + 4 * l2 + 2 * l3 + l4 " 14
X=2 occurs most often, so make it shortest: 0
l1 + 2 * l3 + l4 " 10
X=3: 10
l1 + l4 " 6
Now any string starting with 0 begins with X=3.
If it starts with 10, that’s X=2.
Starting with 11 is undefined.
X=1: 110
X=3: 111
Final code:
X Code l p
weighted
1 110
3 .125 0.375
2
0 1 .5 0.5
3
10 2 .25 0.5
4 111
3 .125 0.375
L= 1.75
c) I’m not sure what this is asking.
d)
H ( X ) = # " p( x ) * log 2 ( p( x )) = 3* ["1/3* log 2 (1/3)] = log 2 ( 3) = 1.585
x
!
!
!
3 4 4 3
+ + + = 1.75
8 8 8 8
[
]
1
1
1
1.585 " L = * l1 + * l2 + * l3
3
3
3
l1 + l2 + l3 " 4.8 (since all lengths are integers, real minimum L is 5/3)
Y Code l
1
0 1
2
10 2
3
11 2
p
.333
.333
.333
L=
weighted
0.333
0.667
0.667
1.67 > H(Y), but minimum possible
2. a) Pr(strong flash) = 0.5
Pr(spike|strong flash)= 0.5+Δp
E(spiketime|strong flash & spike) = 1+Δτ
Pr(no spike|strong flash)=0.5-Δp
Pr(strong flash|spike)= 0.5+Δp
Pr(spike)=0.5
Pr(weak flash) = 0.5
Pr(spike|weak flash)= 0.5-Δp
E(spiketime|strong flash & spike) = 1-Δτ
Pr(no spike|weak flash)= 0.5+Δp
Pr(weak flash|spike)= 0.5-Δp
Eleanor Fraser
NS248 Problem Set 7
February 23, 2010
!
# 1 &
1" ln%
(
# #
)
#
strong &
strong &&
$1± +, '
" * Pr% spiketime = t | spike &
( * log 2 % Pr% spiketime = t | spike &
(( dt =
0
$
weak '
weak ''
ln(2)
$ $
H(total)=H(spike)+H(spiketime|spike)
H(spike)= 1
% 1 (
% 1 (
1" ln'
1" ln'
*
*
&1+ #$ )
&1" #$ )
* (0.5 + #p) +
* (0.5 " #p)
H(spiketime|spike)=
ln(2)
ln(2)
% 1 (
% 1 (
1" ln'
1" ln'
*
*
&1+ #$ )
&1" #$ )
* (0.5 + #p) +
* (0.5 " #p)
H(total)= 1+
ln(2)
ln(2)
!
function h=entropycalc(dp,dt)
h_spiking=1;
! h_timing=(1-log(1./(1+dt)))/log(2).*(0.5+dp)+...
(1-log(1./(1-dt)))/log(2).*(0.5-dp);
h=h_spiking+h_timing;
end
h=h_spiking+h_timing;
clear
dp=0:0.005:0.5;
dt=0:0.005:1;
hdpdt=nan(length(dp),length(dt));
for dpi=1:length(dp)
hdpdt(dpi,:)=entropycalc(dp(dpi),dt);
end
surf(dp,dt,hdpdt')
xlabel('delta-p'); ylabel('delta-tau'); zlabel('H (bits)');
print -dpng Problem2a.png
This can’t be right. H(spiking)>0, but 1+H(spiking) < 1 for high Δτ. Since this is wrong, all the rest
will be too.
b) I(output,input)=H(output)-H(output|input)=H(input)-H(input|output)
H(output|input)=
Eleanor Fraser
NS248 Problem Set 7
February 23, 2010
*
$
',
&
Pr(
flash)
*
"
Pr
spiking
|
flash
*
log
Pr
spiking
|
flash
)
(
))))/
#
2(
& # (
flash,
%
spiking
(/.
+
$
*Pr ( spike _ t | strong) * log 2 (Pr( spike _ t | strong))-'
&
/)
= 0.5 * "#,
&
)
,
+Pr ( ~ spike | strong) * log 2 (Pr( ~ spike | strong)) /.(
%
$
*Pr( spike _ t | weak ) * log 2 (Pr( spike _ t | weak ))-'
&
/)
+0.5 * "#,
&
)
,
+Pr( ~ spike | weak ) * log 2 (Pr( ~ spike | weak )) /.(
%
$ 1 '
1" ln&
)
%1+ 01 (
= "0.5 * (0.5 + 0p) *
" 0.5 * (0.5 " 0p)
ln(2)
$ 1 '
1" ln&
)
%1" 01 (
"0.5 * (0.5 " 0p) *
" 0.5 * (0.5 + 0p)
ln(2)
*
$ 1 '
$ 1 ') " ln&
)/
,2 " ln&
% 1+ 01 (
%1+ 01 ( /
,
= "0.5 * (0.5 + 0p) *
" 0.25
ln(2)
,
/
,+
/.
!
Again, can’t be right because of values < 0.