Introduction to Hypothesis Testing:
The Binomial Test
9/29
ESP
• Your friend claims he can predict the future
• You flip a coin 5 times, and he’s right on 4
• Is your friend psychic?
Two Hypotheses
• Hypothesis
– A theory about how the world works
– Proposed as an explanation for data
– Posed as statement about population parameters
• Psychic
– Some ability to predict future
– Not perfect, but better than chance
• Luck
– Random chance
– Right half the time, wrong half the time
• Hypothesis testing
– A method that uses inferential statistics to decide which of two
hypotheses the data support
Likelihood
• Likelihood
– Probability distribution of a statistic, according to each hypothesis
– If result is likely according to a hypothesis, we say data “support” or “are
consistent with” the hypothesis
• Determining likelihood
•
Inference
– Psychic: hard to say; how psychic?
– Luck: can work out exactly; 50/50 chance each time
Sample
Strategy: Population
– Work out probabilities according to luck hypothesis
– Calculate how likely outcome
would be
Probability
– Likely: luck can explain it
– Unlikely: luck can't explain; must be ESP
Support
• “Innocent until proven guilty”
– Assume luck unless compelling evidence to rule it out
Hypotheses
Statistics
Likelihood
Likelihood According to Luck
6/32 = 18.75%
Flip
Flip
Flip
Flip
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
Y Y Y Y Y
Y N Y Y Y
N Y Y Y Y
N N Y Y Y
Y Y Y Y N
Y N Y Y N
N Y Y Y N
N N Y Y N
Y Y Y N Y
Y N Y N Y
N Y Y N Y
N N Y N Y
Y Y Y N N
Y N Y N N
N Y Y N N
N N Y N N
Y Y N Y Y
Y N N Y Y
N Y N Y Y
N N N Y Y
Y Y N Y N
Y N N Y N
N Y N Y N
N N N Y N
Y Y N N Y
Y N N N Y
N Y N N Y
N N N N Y
Y Y N N N
Y N N N N
N Y N N N
N N N N N
Binomial Distribution
• Binary data
– A set of two-choice outcomes, e.g. yes/no, right/wrong
• Binomial variable
– A statistic for binary samples
– Counts how many are “yes” / “right” / etc.
• Binomial distribution
– Probability distribution for a binomial variable
– Gives probability for each possible value, from 0 to n
• A family of distributions
– Like Normal (need to specify mean and SD)
– n: number of observations (sample size)
– q: probability correct each time
Binomial Distribution
Formula (optional):
p(count  c)  q c (1  q) nc
0.00
0.05
0.05
Probability
0.10
0.20
0.05
0.10
0.00
Probability
0.10
0.10 0.150.15 0.20
nn==20,
20,qq==.25
.5
0.150.30
nn==20,
5, qq == .5
.5
n!
c!( n c )!
0 02
41 6
3 14 416 18 5 20
82 10 12
Count
0
2
4
6
8
10 12 14 16 18 20
Count
Binomial Test
•
•
Hypothesis testing for binomial statistics
Null hypothesis
– q = .5
– Nothing interesting going on; blind chance
•
Alternative hypothesis
– q equals something else
•
Find likelihood according to null hypothesis
•
If p > .05
– p(count  c)
– Special number, called p-value or p
– Tells how good an explanation for the data the null hypothesis is
– Null hypothesis can explain data
•
If p < .05
– Null hypothesis cannot explain data
– Rule out null and believe alternative hypothesis
•
.05 is arbitrary cut-off; more on this next time
Applications
• Any question of rate or proportion
– Rates of outcomes: coins, other probabilities
– Answers relative to chance: learning, memory, etc.
– Composition of group: sex, politics, attitudes
• Population
– All people, events, etc.
– Often infinite (probabilities)
• Parameter
– Proportion or probability in population
• Sample
– Set of binary outcomes
– Answers for one student; category for each subject
• Statistic
– Proportion or count in sample
Testing for ESP
• Null hypothesis: Luck, q = .5
• Alternative hypothesis: q > .5
• Likelihood according to null: Binomial(5, .5)
p(count  4)  .15625  .03125  .1875
0.20
0.10
.15625
.03125
0.00
Probability
Retain null hypothesis
Luck can explain the data
0.30
• p > .05
0
1
2
3
Count
4
5
Testing for ESP – Stronger Evidence
•
•
•
•
Nine correct out of ten
Null hypothesis: Luck, q = .5
Alternative hypothesis: q > .5
Likelihood according to null: Binomial(10, .5)
0.15
.010
0.05
0.10
.001
0.00
Luck can’t explain the data
Reject null hypothesis
Probability
• p < .05
0.20
p(count  9)  .010  .001  .011
0
1
2
3
4
5
Count
6
7
8
9
10
Other values of q
• Multiple-choice tests; guessing card suits
– q = .25
• New null hypothesis, likelihood function
0.10
0.05
p = .014
0.00
Probability
0.15
0.20
n = 20, q = .25
0
2
4
6
8
10 12 14 16 18 20
Count
Normal Approximation
• For large n, Binomial is nearly Normal
– Consequence of Central Limit Theorem: count = mean  n
– Mean = nq, Standard Deviation = nq(1  q)
nq(1  q )
– Use Standard Normal to find p
Probability
Probability
– .z 
(count  12 )  nq
count
n
q Mean
SD
cutoff
z
4
5
.5
2.5
1.1
3.5
.89
9
10 .5
5.0
1.6
8.5
2.21
0.00
0.05 0.10
0.10 0.15
0.15 0.20
0.20 0.25
0.25
0.00 0.05
• Convert cutoff to z-score
Cutoff = 8.5
p
Approx
Exact
.186
.188
.013
00
11
22
33
44
.011
55
Count
Count
66
7
8
9 10