ST4004 Management Science Methods
Laboratory on Bayesian Inference
Open Bayesian-inference-lab.xlsx. It shows the socks example for the
case where only 9 values of p are possible (0.1, 0.2, ..., 0.9), rather than all values in
the range (0,1) as considered in classes.
Note that N (total no. of socks) and k (no. of black socks) are defined to be 30 and 9
respectively. Then in a table are (in columns) the values of p, the prior probability P(p)
(here each value of p is given equal probability of P(p) = 1/9), the binomial likelihood
N
P(k | N , p)    p k (1  p) N k ,
k
then P(p) P(k | N,p) and finally the posterior, which by Bayes law is the previous
column values normalised so that they sum to 1:
P( p | N , p) 
P( p) P(k | N , p)
.
 P( p) P(k | N , p)
all p
Then there are plots of the prior, the likelihood, the posterior and finally both the prior
and posterior on one plot for comparison.
1.
Go through the formulae in the table and relate them to the above use of Bayes
law.
2.
The proportion of observed black socks in the file is 9 from 30 (or 0.3). Recompute the posterior distribution for the following differing amounts of data
where the same proportion of black socks are observed: 3 from 10, 18 from 60,
90 from 300, 900 from 3000; that is, the evidence that p=0.3 is increasing. Write
down P(p=0.3 | k, N) for each of these cases. Comment on the values of this
probability as the amount of data increases.
3.
Return to k=9, N=30. We'll now explore the effect of the prior on the posterior
distribution. We assign the following prior:

P(p) = p
P(p)p
P(0.5) = 
where we are free to choose  and  as long as  ≥ 0 and 4 + 4 ≤ (so
that we are defining a probability distribution). This allows us to model beliefs
where we think that p is more likely to be larger than 0.5 (), smaller than
0.5 () or all prior probabilities equal as in part 2. ().
Set up the worksheet so that the prior probabilities are defined in terms of  and
. Initially set so that you have the prior of part 2.
Compute the posterior for  = 0.01 and  = 0.02, 0.04, 0.08, 0.12 and 0.2
(increasing prior weight on smaller values of p). Look at P(p=0.3 | k,N) for each
case and comment on its value as  changes.
4.
Let  = 0.2 and  = 0.01 (strong prior belief that p is bigger than 0.5). Look at
the posterior as the evidence for the data that p=0.3 gets larger, as in part 2. That
is, look at P(p=0.3 | k,N) for the cases in part 2 for k and N. Comment on its
value and compare to the values you obtained for this probability in part 2.
5.
The predictive distribution tells you how many socks you expect to see based on
a distribution for p. If all we know about p is the prior P(p) then by the Partition
Law:
0.9
N
P(k from N )   P(k | N , p) P( p)     p k (1  p) N k P( p) .
all p
p  0.1  k 
Now suppose that we have already observed k from N. Now we have a new
experiment where there are M socks. How many socks m out of the M are black?
Now we use the posterior distribution of p:
P(m from M | k from N )

0.9
M 
 P(m | M , p) P( p | k , N )    m  p
all p
p  0.1


m
(1  p) M m P( p | k , N ), m  0,1,..., M
For the case of a new wash of M = 20 socks, compute on the worksheet the
predictive distribution for m based on:
(a)
(b)
the uniform prior P(p) = 1/9, p=0.1,...,0.9;
the data k=9, N=30.
Comment on the distributions that you find. Repeat (b) for k=18, N=60 and
k=90,N=300 and comment on what happens to the predictive as the amount of
evidence that p=0.3 increases.