Chapter 9: Inferences for Proportions and Counts
Methods for Discrete Data: Categorical, Nominal, Ordinal data
One Sample - Section 9.1
• Large Sample methods : confidence intervals, sample sizing,
hypothesis tests, power calculations
• Small Sample methods: confidence intervals, hypothesis tests
Two Samples (comparisons) - Section 9.2
• Independent Samples design: Large & Small sample methods
• Matched Pairs design
One-way Counts - Section 9.3
• χ2 Multinomial distribution test
• χ2 Goodness-of-Fit test
Two-way Counts - Section 9.4
• Sampling methods
• Hypothesis tests
• Odds ratio
Class #7
ACM 625.403 - Telford
1
Large Sample Confidence Interval for Proportions
Recall that
pˆ − p
≈ N(0,1) if n is large enough [ np≥10 & n(1-p) ≥10 ]
pq / n
Approximate Confidence Interval for p:
pˆ − zα / 2
pˆ qˆ
pˆ qˆ
≤ p ≤ pˆ + zα / 2
n
n
This formula is VERY commonly
used for “back-of-the-envelope”
calculations.
More accurate Confidence Interval for p:
Equation 9.3 - If small sample, should use method in
Section 9.2. However, Equation 9.3 can be hand-calculated.
• See Example 9.1, proportion of homes with guns, p. 301
One-sided confidence limits are also available (p. 301)
Class #7
ACM 625.403 - Telford
2
1
Sample Size for Confidence Interval for a Proportion
Want (1-α)-level two-sided C.I.:
pˆ ± E where E is the margin of error. Then E = za / 2
2
z

Solving for n gives n =  a / 2  pˆ qˆ .
E


pˆ qˆ
n
.
 1  1  1
Largest value of p q =    = so conservative sample size is :
 2  2  4
2
z
 1
n =  a/2  × .
 E  4


There are one-sided sample size formulas also.
These two formulas are often used to sample size
for the number of Monte Carlo runs needed.
Class #7
ACM 625.403 - Telford
3
Example 9.2: Presidential approval, p.302
Want to estimate the proportion
with a favorable opinion of the
President with a margin of error
of 3 percentage points, using
95% confidence.
2
1
1.960 
n = 
 × 4 = 1067.11
0
.
03


→ a maximum of 1068 people need to be polled
If a prior estimate of 65% is available,
2
1.960 
n = 
 × (0.65)( 0.35) = 971.07
 0.03 
→ 972 people should be polled
Class #7
ACM 625.403 - Telford
4
2
Large Sample Hypothesis Tests on a Proportion
H0 : p = p0 vs. H1 : p ≠ p0
Test statistic:
z=
pˆ − p0
x − n p0
or
p0 q0 / n
n p0 q0
These formulas are VERY commonly
used for “back-of-the-envelope”
calculations.
Z ≈ N(0,1) if n is large enough [ np ≥ 10 (or 5 or 1) & n(1-p) ≥10 ]
The usual dual relationship between the C.I. and the hypothesis
test does NOT hold:
p̂ vs. p0 used in the estimate of the standard deviation of p̂ .
General rule: Z is always computed under H0.
• See Example 9.3, Free throws, pp. 303-304
Class #7
5
ACM 625.403 - Telford
Large Sample Hypothesis Tests on a Proportion
There are two-sided and one-sided hypothesis tests:
see formulas in Table 9.1, p. 304.
Remember: State the conclusion in terms of the original question.
It is appropriate in the case of a two-sided hypothesis to further
state the direction that the data indicates.
H0 : p = 0.50
vs.
H1 : p ≠ 0.50
Class #7
Reject H0 .
(King thinks
H1 : p < 0.50)
ACM 625.403 - Telford
Conclusion: The Queen is
different from being half
bad and is MORE than
half bad, at some α level.
6
3
Power Calculation and Sample Size for
Large Sample Test on a Proportion
H0 : p = p0 vs. H1 : p > p0 (one-sided test)
Suppose that the power must be 1 - β when the true proportion
is p = p1 > p0 .
 z p q + zβ p1q1 

n= α 0 0


δ


2
If a two-sided test, replace zα by zα/2 .
• See Example 9.4, Pizza Tasting, pp. 305-306
Class #7
7
ACM 625.403 - Telford
Small Sample Hypothesis Test on a Proportion
What if not all n1p1, n1(1 - p1), n2p2, n2(1 - p2) ≥ 10 (or 5) ?
Large sample asymptotic normality (CLT) does NOT hold for
pˆ =
X
n
or equivalently X = npˆ ~ Bin ( n, p ).
Upper one-sided test:
H0 : p ≤ p0 vs. H1 : p > p0
n
n
P − value = P ( X ≥ x | p = p0 ) = ∑   p0i (1 − p0 ) n −i
i
i=x  
Lower one-sided test:
H0 : p ≥ p0 vs. H1 : p < p0
x
n
P − value = P ( X ≤ x | p = p0 ) = ∑   p0i (1 − p0 ) n −i
i
i =0  
Two-sided test:
H0 : p = p0 vs. H1 : p ≠ p0
P-value = 2 min of (upper one-sided, lower one-side P-values)
Class #7
ACM 625.403 - Telford
8
4
Small Sample Confidence Interval on a Proportion
Not in book ! (Clopper-E.S.Pearson exact confidence intervals, 1933)
What if not all n1p1, n1(1 - p1), n2p2, n2(1 -p2) ≥ 10 (or 5) ?
Use the cumulative binomial distribution function or equivalently
the cumulative beta function.
Two-sided (1-α) confidence interval:
[ 1 - betainv(1-(α/2), n-x+1, x),
1 - betainv(α/2, n-x, x+1) ]
Upper one-sided (1-α) confidence limit:
[ 0, 1 - betainv(α, n-x+1, x) ]
Lower one-sided (1-α) confidence limit:
[ 1 - betainv(1-α, n-x , x+1), 1 ]
Class #7
ACM 625.403 - Telford
9
Comparing Two Proportions for Large Samples:
Confidence Interval for Independent Samples Design
If all n1p1, n1(1 - p1), n2p2, n2(1 - p2) ≥ 10 , then
Z=
( pˆ1 − pˆ 2 ) − ( p1 − p2 )
p1 (1− p1 ) p2 (1− p2 )
+
n1
n2
≈ N (0, 1)
Two-sided (1-α) confidence interval for p1 - p2:
pˆ1 − pˆ 2 ± zα / 2
p1 (1− p1 ) p2 (1− p2 )
+
n1
n2
One-sided confidence intervals are also available.
Class #7
ACM 625.403 - Telford
10
5
Comparing Two Proportions for Large Samples:
Hypothesis Tests for Independent Samples Design
Test of H0 : p1 - p2 = δ0
( pˆ1 − pˆ 2 ) − δ 0
Z=
p1 (1− p1 ) p2 (1− p2 )
+
n1
n2
Test of equality of proportions, H0 : p1 = p2
Z=
( pˆ 1 − pˆ 2 ) − ( p1 − p2 )
p p (1 − p p ) n1 + n1 
 1 2
where p p =
n1 pˆ1 + n2 pˆ 2 X 1 + X 2
=
n1 + n2
n1 + n2
One-sided hypothesis tests are also available.
• See Example 9.5, polio vaccine, p. 309
• See Example 9.6, leukemia therapies, pp. 309-310
Class #7
11
ACM 625.403 - Telford
Comparing Two Proportions for Small Samples:
Hypothesis Tests for Independent Samples Design
When the large sample, asymptotic assumptions do NOT hold,
Fisher’s (exact) Test can be used for testing the equality of
proportions, H0 : p1 = p2 .
See Table 9.3, 2 x 2 table, p. 310
Success Failure
Sample 1
x
n1 - x
Row
Total
n1
Sample 2
y
n2 - y
n2
Column
Total
m
n-m
n
P-values are calculated using the
hypergeometric distribution:
( )( )
P( X = i | X + Y = m) =
()
n1
i
n2
m −i
n
m
Two-sided and one-sided hypothesis tests are available.
• See Example 9.7, age discrimination, pp. 311-312
• See Example 9.8, leukemia therapies, p. 312 (low power/pessimistic/
conservative test - other small sample tests are available in the literature)
Class #7
ACM 625.403 - Telford
12
6
Comparing Two Proportions
Hypothesis Tests for Independent Samples Design
S uccess F ailure
S et 1
36
4
40
S et 2
16
4
20
52
8
60
Note: Fisher’s test has
the largest p-value; is the
least sensitive test for
detecting changes.
Small sample/exact methods
Large sample/approx. methods
Class #7
13
ACM 625.403 - Telford
Comparing Two Proportions for Large Samples:
Hypothesis Tests for Matched Pairs Design
Condition 2
When the samples are matched pairs, McNemar’s Test can be
used for testing the equality of response proportions, H0: p1 = p2 .
See Table 9.5, 2x2 table, p. 312
p1 = pA + pC and p2 = pA + pB
Condition 1
Yes
No
Yes
a
b
No
c
d
where a + b + c + d = n
pA + pB + pC + pD = 1
Test H0: p1=p2 or equivalently pB=pC
B ~ Bin( m, p =
pB
p B + pC
) = Bin( m, 12 )
where m = b + c .
Test H0 : p = 0.5
Small sample and large sample tests, as well as two-sided and
one-sided hypothesis tests, are available (p. 313).
• See Example 9.9, change in voting preference, pp. 313-314
Class #7
ACM 625.403 - Telford
14
7
Inference for One-way Count Data
Test for Multinomial distribution χ2 (Karl Pearson):
C a te g o r y O b s e r v e d
( n i)
1
n1
2
P r o b a b ility
( p io )
p 1o
E x p e c te d
( e i = n × p io )
e1
p 2o
e2
n2
…
…
…
…
c
nc
p co
ec
c
c
where ∑ ni = ∑ ei = n
i =1
i =1
c
and
∑p
i =1
io
= 1.
H0 : p1=p1o, p2=p2o, …, pc=pco vs. H1 : at least one pi ≠ pio
(ni − ei ) 2
~ χ2 with c-1 d.f. since the
The test statistic χ = ∑
e
i =1
i
pio are known values with
2
c
one constraint (sum = 1).
• See Example 9.10, uniformity of random digits, p. 316
• See Example 9.11, Mendelian genetics, pp. 316-317
Class #7
ACM 625.403 - Telford
15
Inference for One-way Count Data
Known Probabilities
Example 9.11, Mendelian genetics, pp. 316-317
Enter all
hypothesized
probabilities
Conclusion: Data fits
hypothesized distribution,
almost too well.
Class #7
ACM 625.403 - Telford
16
8
Inference for One-way Count Data
Goodness-of-Fit Test χ2:
Category Observed Est. Prob. Expected
∧
∧
∧
(pio)
(ei=n×pio)
(ni)
∧
∧
1
p1o
e1
n1
∧
∧
2
n2
p2o
e2
…
…
…
…
c
nc
pco
∧
c
c
where ∑ ni = ∑ eˆi = n
i =1
and
i =1
c
∑ pˆ
i =1
io
= 1.
∧
ec
H0 : p1=p1o, p2=p2o, …, pc=pco vs. H1 : at least one pi ≠ pio
(ni − eˆi ) 2
~ χ2 with c-1-p d.f. since
The test statistic χ = ∑
eˆi
i =1
p parameters are being
2
c
estimated from the data.
• See Example 9.12, radioactive decay counts, pp. 319-321
Class #7
17
ACM 625.403 - Telford
Inference for One-way Count Data
Estimated Probabilities
Example 9.11, Mendelian genetics, pp. 316-317
Pretend that the probabilities are estimated.
Same data
entry steps
Leave out one of the
hypothesized
probabilities and choose
“Fix hypothesized
values, rescale omitted”
reduces by
one d.f.
Class #7
ACM 625.403 - Telford
18
9
Inference for Two-way Count Data
Sampling Model 1 (Total Sample Size Fixed):
Sample of 901 from a single population that is then cross-classified.
The null hypothesis is that row and column classifications are independent:
H0 : pij = pi• × p•j vs. H1 : pij ≠ pi• × p•j (independence hypothesis)
Class #7
ACM 625.403 - Telford
19
Inference for Two-way Count Data
Sampling Model 1 (Total Sample Size Fixed):
Estimated expected frequencies
eˆij = n pˆ ij = n pˆ i• pˆ • j = n
ni• n• j ni•n• j
=
n n
n
Estimated expected frequency for Cell (1,1)
eˆ11 = n
206 62 206 × 62
n1• n•1
= 901 ×
×
=
= 14.18
901 901
901
n n
c
r
χ = ∑∑
2
i =1 j =1
( nij − eˆi j ) 2
eˆij
~ χ2 with (r-1)(c-1) d.f. under H0 .
• See Example 9.13 and Table 9.10, pp. 324-325, for calculations
Class #7
ACM 625.403 - Telford
20
10
Inference for Two-way Count Data
Sampling Model 1 (Total Sample Size Fixed):
Chi-square test critical value for Example 9.13:
The d.f. for this χ2 test statistic is (4 - 1)(4 - 1) = 3 × 3 = 9.
Since χ2(9 d.f.,α=0.05) = 16.919, the calculated χ2 of 11.989 is not
sufficiently large to reject the hypothesis of independence at
the α = 0.05 level.
Conclusion: There is not a statistically significant association
between income level and job satisfaction across the four
levels of each, with 5% false alarm rate level.
Class #7
ACM 625.403 - Telford
21
Inference for Two-way Count Data
Class #7
ACM 625.403 - Telford
22
11
Inference for Two-way Count Data
Class #7
ACM 625.403 - Telford
23
Inference for Two-way Count Data
Note that most of the
contribution to the Chi-square
statistics comes from the
corner cells.
Lack of significance in the
Chi-square statistics is the
result of the low contribution
to the Chi-square statistic
coming from the center cells.
Class #7
ACM 625.403 - Telford
24
12
Inference for Two-way Count Data
This is now a (2-1)(2-1) = 1×1 d.f. test,
so χ2(1 d.f.,α=0.05) = 3.843
Conclusion: There is a highly statistically significant
association between high and low income level and high
and low job satisfaction, with a p-value of less than 0.01%.
Class #7
ACM 625.403 - Telford
25
Inference for Two-way Count Data
Sampling Model 2 (Row Totals Fixed):
Total number of patients in each drug group is fixed.
The null hypothesis is that the probability of response j is the
same, regardless of the row classification:
H0 : pij = p•j for all j vs. H1 : pij ≠ p•j for some j (homogeneity
hypothesis)
c
r
χ = ∑∑
2
i =1 j =1
( nij − eˆi j ) 2
eˆij
~ χ2 with (r-1)(c-1) d.f.
under H0 .
Class #7
ACM 625.403 - Telford
26
13
Inference for Two-way Count Data
Class #7
27
ACM 625.403 - Telford
Inference for Two-way Count Data
Why is this a BAD
data display?
1.
2.
Conclusion: A statistically significant
difference between the successes
rates for the two drug groups, with the
higher successes in the
Prednisone+VCR group, with α=0.05.
Class #7
ACM 625.403 - Telford
Fisher’s
Test is less
likely to reject
since it has
the higher
P-value.
28
14
Remarks about Chi-square Test
The distribution of the Chi-square statistic under the null hypothesis
is approximately Chi-square only when the sample sizes are large.
Rules of Thumb:
• All expected cell counts should be greater than 1.
• No more than 1/5th of the expected cell counts be less than 5.
Combine sparse cell (having small expected cell counts) with
adjacent cells. Unfortunately, this has the drawback of losing some
information.
Odds Ratio as a Measure of Association for Discrete Data
Section 9.4.3, pp. 327-329, formulas for Sampling Models 1 & 2
Test for significant difference from 1.
• See Example 9.15, leukemia therapies, pp. 328-329
Class #7
ACM 625.403 - Telford
29
Homework #7 - due March 26
Read Chapter 9
Exercises: (answers to odd problems in the back of the book)
(show work – no credit if calculations or printout not shown)
9.1 (c) 929
9.3 (a) use Clopper-Pearson formula also [0.0352,0.1516]
9.5, 9.9, 9.11, 9.13
9.16 (b) 0.0012
9.17
9.20 (b) 7.815
9.27, 9.29, 9.33
Read or skim Chapter 10
Next class (March 12) - MIDTERM on Chapters 1 through 8
Class #7
ACM 625.403 - Telford
30
15