16.2
a
η i = β0 + β1 x i
β0
β=
β1


1 x1
1 x2 


X = .
.. 
 ..
. 
1 xn
η = Xβ
θ = g −1 (η) = logit−1 (η) = logit−1 (Xβ)
yi ∼ Binom(ni , θi )
Posterior distribution is proportional to :
Y
p(α, β|y) ∝
logit−1 (α + βxi )yi (1 − logit−1 (α + βxi ))ni −yi
i
Y eα+βxi
1
)yi (
)ni −yi
∝
(
α+βxi
α+βxi
1
+
e
1
+
e
i
Y
∝
eyi (α+βxi ) (1 + eα+βxi )−ni
i
log(p(α, β|y) =
X
yi (α + βxi ) − ni log(1 + eα+βxi ) + C
i
xs <− c ( −0.86 , −0.30 , −0.05 , 0 . 7 3 )
ys <− c ( 0 , 1 , 3 , 5 )
ns <− c ( 5 , 5 , 5 , 5 )
L o g C o n d i t i o n a l P r o b <− function ( a , b ) {
e t a <− a+b∗ xs
l l i k <− ys ∗ ( e t a )−ns∗log (1+exp ( e t a ) )
return (sum( l l i k ) )
}
ProposalAlpha <− function ( x ) rnorm ( 1 , x , 1 . 8 )
P r o p o s a l B e t a <− function ( x ) rnorm ( 1 , x , 1 0 )
G i b b s M e t r o p o l i s <− function ( ProposalAlpha , ProposalBeta , LogProb , N, s t a r t . v a l u e s=c ( 0 . 5 , 7 )
{
a l p h a <− s t a r t . v a l u e s [ 1 ]
beta <− s t a r t . v a l u e s [ 2 ]
a l p h a s <− numeric ( 2 ∗N)
b e t a s <− numeric ( 2 ∗N)
a c c . a l p h a <− numeric ( 2 ∗N)
1
a c c . beta <− numeric ( 2 ∗N)
f o r ( i i n 1 : ( 2 ∗N) )
{
a l p h a . s t a r <− ProposalAlpha ( a l p h a )
beta . s t a r <− P r o p o s a l B e t a ( beta )
pr . a l p h a <− min( 1 , exp ( LogProb ( a l p h a . s t a r , beta)−LogProb ( alpha , beta ) ) )
a l p h a <− sample ( c ( a l p h a . s t a r , a l p h a ) , s i z e =1, prob=c ( pr . alpha , 1−pr . a l p h a ) )
pr . beta <− min( 1 , exp ( LogProb ( alpha , beta . s t a r )−LogProb ( alpha , beta ) ) )
beta <− sample ( c ( beta . s t a r , beta ) , s i z e =1, prob=c ( pr . beta , 1−pr . beta ) )
a l p h a s [ i ] <− a l p h a
b e t a s [ i ] <− beta
a c c . a l p h a [ i ] <− pr . a l p h a
a c c . beta [ i ] <− pr . beta
}
print ( " Alpha ␣Acc : " )
print (mean( a c c . a l p h a ) )
print ( " Beta ␣Acc : " )
print (mean( a c c . beta ) )
m <− matrix ( c ( a l p h a s [ ( N+ 1 ) : ( 2 ∗N) ] , b e t a s [ ( N+ 1 ) : ( 2 ∗N ) ] ) , 2 , N, byrow=T)
return (m)
}
s a m p l e s 1 <− G i b b s M e t r o p o l i s ( ProposalAlpha , ProposalBeta , LogConditionalProb , 1 0 0 0 0 , c ( 0 , 1 )
s a m p l e s 2 <− G i b b s M e t r o p o l i s ( ProposalAlpha , ProposalBeta , LogConditionalProb , 1 0 0 0 0 , c ( 2 , 1 5
alphas = c ( samples1 [ 1 , ] , samples2 [ 1 , ] )
b e t a s = c ( s am p l e s 1 [ 2 , ] , s a m p l e s 2 [ 2 , ] )
SummaryStats <− function ( name , s a m p l e s ) {
print ( name )
print (mean( s am p l e s ) )
print ( quantile ( samples , c ( 0 . 0 2 5 , 0 . 9 7 5 ) ) )
}
SummaryStats ( " a l p h a " , a l p h a s )
SummaryStats ( " b e t a " , b e t a s )
LD50 <− −a l p h a s / b e t a s
SummaryStats ( "LD50" , LD50 )
#
#
#
#
#
#
#
> s o u r c e (" e x e r c i s e 1 6 2 . r ")
[ 1 ] " Alpha Acc : "
[ 1 ] 0.4066657
[ 1 ] " Beta Acc : "
[ 1 ] 0.2535952
[ 1 ] " Alpha Acc : "
[ 1 ] 0.4101632
2
#
#
#
#
#
#
#
#
#
#
#
#
#
#
[1]
[1]
[1]
[1]
" Beta Acc : "
0.2534394
" alpha "
0.7755103
2.5%
97.5%
−0.7453023 2 . 5 5 4 7 3 9 6
[ 1 ] " beta "
[ 1 ] 8.444438
2.5%
97.5%
2.865092 17.223470
[ 1 ] "LD50"
[ 1 ] −0.08526614
2.5%
97.5%
−0.2765251 0 . 1 5 4 0 0 2 5
17
a
Draw xa and xb from f (x) and label such that
f (xa )q(xa |xb ) ≤ f (xb )q(xb |xa )
p(xt−1 = xa , xt = xb ) = f (xa )q(xb |xa )
p(xt−1 = xb , xt = xa ) = f (xb )q(xa |xb )
q(xb |xa ) f (xa )
q(xa |xb ) f (xb )
p(xt−1 = xb , xt = xa ) = f (xa )q(xb |xa )
b
In order to be practically effective:
• The ’steps’ in the proposal distribution must not be too small compared to the parameter space
• The acceptance ratio must not be too small
c
Where q(x|y) = q(y|x) the proposal distribution is symmetric and we can use Metropolis
Where q(x|y) = q0 (x), the proposals are sampled from the same distribution. When q0 is very different than
f , the acceptance rate is low and convergence slow.
d
RandomU <− function ( x ) runif ( 1 , x/ 3 , x∗ 3 )
LogDistU <− function ( y , x ) log ( dunif ( y , x/ 3 , x∗ 3 ) )
LogDistG <− function ( x ) log (dgamma( x , shape=3/ 2 , r a t e =1))
M e t r o p o l i s H a s t i n g s <− function (RandomQ , LogDistQ , LogF , N, x =1.5){
xs <− numeric ( 2 ∗N)
a c c <− numeric ( 2 ∗N)
3
f o r ( i i n 1 : ( 2 ∗N) ) {
y <− RandomQ( x )
pr <− min( 1 , exp ( LogDistQ ( x , y)+LogF ( y)−LogDistQ ( y , x)−LogF ( x ) ) )
x <− sample ( c ( y , x ) , s i z e =1, prob = c ( pr , 1−pr ) )
xs [ i ] = x
a c c [ i ] = pr
}
print (mean( a c c ) )
return ( xs [ ( N+ 1 ) : ( 2 ∗N ) ] )
}
xs = M e t r o p o l i s H a s t i n g s (RandomU , LogDistU , LogDistG , 1 0 0 0 0 )
print ( "Mean" )
print (mean( xs ) )
print ( "Var" )
print ( var ( xs ) )
#
#
#
#
#
s o u r c e (" n i l s 1 7 . r ")
[ 1 ] "Mean"
[ 1 ] 1.511603
[ 1 ] "Var"
[ 1 ] 1.517227
4
18
a
Chi−s q u a r e d t e s t f o r g i v e n p r o b a b i l i t i e s
data : ns
X−s q u a r e d = 1 4 . 3 0 7 6 , df = 1 9 , p−v a l u e = 0 . 7 6 5 5
5
b
c
Fermi/Teller
6
d
[ 1 ] " Metropolis "
Chi−s q u a r e d t e s t f o r g i v e n p r o b a b i l i t i e s
data : ns
X−s q u a r e d = 6 1 . 6 5 3 8 , df = 1 9 , p−v a l u e = 2 . 1 1 3 e −06
7
e
[ 1 ] " SparseMetropolis "
Chi−s q u a r e d t e s t f o r g i v e n p r o b a b i l i t i e s
data : ns
X−s q u a r e d = 2 7 . 9 1 5 5 , df = 1 9 , p−v a l u e = 0 . 0 8 5 0 7
f
When M is large, the chain moves too slowly. Could use a larger uniform proposal function.
8
9
Using a larger window
10
11
Using asymmetric proposal function
12
library ( plyr )
t e s t f i t <− function ( data , M, name=" t e s t " ) {
j s = 1 :M
T = M∗ (M+1)/2
ps = j s /T
c o u n t s = count ( data )
ns = numeric (M)
ns [ c o u n t s $x ] = c o u n t s $ f r e q
# png ( p a s t e ( name , " h i s t . png " , s e p = " " ) )
h i s t ( data , 0 . 5 : (M+ 0 . 5 ) , main=name )
# dev . o f f ( )
13
print ( name )
print ( c h i s q . t e s t ( ns , p=ps ) )
return ( ns )
}
suba <− function ( ) {
M = 20
j s = 1 :M
T = M∗ (M+1)/2
ps = j s /T
s = sample ( j s , 1 0 0 0 0 , replace=T, prob=ps )
t e s t f i t ( s , M, " sample " )
}
M e t r o p o l i s <− function ( randomQ , f , N, M=20) {
x <− M/2
xs <− numeric ( 2 ∗N)
f o r ( i i n 1 : ( 2 ∗N) ) {
y <− randomQ ( x , M)
pr <− min( 1 , f ( y ) / f ( x ) )
x <− sample ( c ( y , x ) , s i z e =1, prob = c ( pr , 1−pr ) )
xs [ i ] <− x
}
return ( xs [ ( N+ 1 ) : ( 2 ∗N ) ] )
}
rqD <− function ( x , M=20) sample ( c ( ( x+1) %% M, ( x−1) %% M) , s i z e =1, prob=c ( 0 . 5 , 0 . 5 ) )
fD <− function ( x ) x+1
subd <− function ( )
{
N = 50000
xs = M e t r o p o l i s ( rqD , fD , N)+1
t e s t f i t ( xs , 2 0 , " M e t r o p o l i s " )
}
sube <− function ( )
{
N = 5000
xs = M e t r o p o l i s ( rqD , fD , N)+1
pruned = xs [ ( 1 : ( N/ 1 0 0 ) ) ∗ 1 0 0 ]
t e s t f i t ( pruned , 2 0 , " S p a r s e M e t r o p o l i s " )
}
s u b f <− function ( ) {
N = 50000
M = 5000
xs = M e t r o p o l i s ( rqD , fD , N, M)+1
png ( " 5000 p l o t . png" )
plot ( xs , type=" o " )
dev . o f f ( )
t e s t f i t ( xs , M, " 5000 a" )
14
new . p r o p o s a l <− function ( x , M) {
v = sample((−M/ 2 0 ) : (M/ 2 0 ) , 1 )
return ( ( x+v ) %% M)
}
xs = M e t r o p o l i s (new . p r o p o s a l , fD , N, M)+1
#png ("5000 p l o t 2 . png ")
plot ( xs , type=" o " )
#dev . o f f ( )
t e s t f i t ( xs , M, " 5000 b" )
maxLimit = 5000
asymmetric . p r o p o s a l <− function ( x ) {
min . l e v e l <− f l o o r ( x/ 2 )
max. l e v e l <− min( maxLimit −1, x∗ 2 )
v = sample (min . l e v e l :max. l e v e l , 1 )
return ( v )
}
asymmetric . density <− function ( y , x ) {
min . l e v e l <− f l o o r ( x/ 2 )
max. l e v e l <− min( maxLimit −1, x∗ 2 )
return ( log ( 1 / (max. l e v e l −min . l e v e l +1)))
}
prob . f u n c <− function ( x ) log ( x+1)
source ( " n i l s 1 7 . r " )
xs <− M e t r o p o l i s H a s t i n g s (
asymmetric . p r o p o s a l ,
asymmetric . density ,
prob . func ,
N,
x=f l o o r (M/ 2 ) )
t e s t f i t ( xs , 5 0 0 0 , " H a s t i n g s " )
}
#suba ( )
# subd ( )
#s u b e ( )
subf ()
15