McNemar’s Test
Example (crossover study: Drug vs Placebo)
86 subjects. Randomly assign each to “drug then placebo” or
“placebo then drug”. Binary response for (S, F) for each.
Treatment
Drug
Placebo
S
61
22
F Total
25
86
64
86
Dependant samples: e.g., same subject in each sample, which yield
matched pairs.
Display data as 86 obs rather than 2*86 obs.
Drug
S
F
Total
Placebo
S
F
12
49 |
10
15 |
---------22
64 |
total
61
25
--86
population probabilities:
drug S
drug F
total
placebo S
π11
π21
π+1
placebo F
π12
π22
π+2
total
π1+
π2+
1
Compare dependent samples by making inference about π1+ − π+1 .
There is marginal homogeneity if π1+ = π+1 .
McNemar’s test
Under H0 : π1+ = π+1 or π12 = π21
π12
1
π12 +π21 = 2 .
under H0 , n∗ = n12 + n21 ∼ Bin (n∗ , 21 ).
q
n∗
with mean = 2 , sd = n∗ 12 12 .
for large n∗ ,
∗
z=
n −n
q12 2
n∗ 12 12
=
√n12 −n21
n12 +n21
or equivalently
2
21 )
2
z 2 = (nn1212−n
+n21 ∼ χ1
called McNemar’s test.
∼ N(0, 1)
Drug
S
F
Total
Placebo
S
F
12
49 |
10
15 |
---------22
64 |
(26%)
total
61
25
--86
(71%)
= 5.1 (or z 2 = χ21 = 25.8, d.f . = 1)
z = √49−10
49+10
p-value= 2P(z > 5.1) < 0.0001. Strong evidence the probability
of success is higher for drug than for placebo.
CI for π1+ − π+1
21
Point estimate: p1+ − p+1 = n12 −n
.
n
q
2
21 )
SE = n1 n12 + n21 − (n12 −n
.
n
Example:
p1+ − p+1 = 49−10
86 = 0.453.
q
2
1
SE = 86
49 + 10 − (49−10)
= 0.075.
86
95% CI: 0.453 ± 1.96 ∗ 0.075 = 0.453 ± 0.146 = (0.31, 0.60).
> crossover=matrix(c(12,10,49,15),2,2)
> mcnemar.test(crossover,correct=F)
McNemar’s Chi-squared test
data: crossover
McNemar’s chi-squared = 25.7797, df = 1,
p-value = 3.827e-07
A recent GSS asked subjects whether they believed in heaven or
hell. The table below showed the results.
Believe in
Believe in Heaven
Yes
No
Yes
833
2
Hell
No
125
160
a. Test the hypothesis that the population proportion answering
“yes” were identical for heaven and hell. Use a two-sided
alternative.
b. Find a 90% CI for the difference between the population
proportions.
crossover experiment
A crossover experiment with 100 subjects compares two treatments
for migraine headache. The response scale is success (+) and
failure (-). Half the subjects, randomly selected, used drug A first
and drug B next. For them, six had response (A+, B+), 25 had
response (A+, B-), 10 had (A-, B+) and nine had (A-,B-). Among
the other 50 subjects, 10 were (A+, B+), 20 were (A+, B-), 12
were (A-, B+), and eight were (A-, B-).
Use the McNemar test to compare the success probabilities for the
two drugs.
drug A
+
-
drug B
+
16
45
22
17
H0 : π1+ = π+1 or π12 = π21
H1 : π1+ 6= π+1
z = √45−22
= 2.8099 or χ21 = 7.90.
45+22
p-value = 2P(z > 2.8099) = 0.005.
Reject H0 .
CI for π1+
q− π+1 :
2
1
45 + 22 − (45−22)
= 0.079.
SE = 100
100
CI: 0.23 ± 1.96 ∗ 0.079 = (0.075, 0.385).
Measure Rater Agreement
Observed
agreement
percentage:
P
P
nii
p = n++ = π̂ii
Cohen’s
P Kappa:
P
πiiP
− πi+ π+i
κ = 1−
πi+ π+i .
It depends strongly on marginal distributions.
Method A
Yes
No
Method B
Yes
No
61
2
6
25
observed agreement percentage:
κ̂ =
P
π̂ii =
0.915−[(63/94)∗(67/94)+(31/94)∗(27/94)]
1−[(63/94)∗(67/94)+(31/94)∗(27/94)]
=
61+25
61+2+6+25
0.915−0.572
1−0.572
= 0.915
= 0.801.