Megan Templin, MPH, M.S.
H. James Norton, PhD
Dickson Advanced Analytics
Research Division
Carolinas Healthcare System
1
Chi-square Test
2
Jeopardy: Statistics for 100
The test that should be
performed to answer: …“The general
problem may be stated as follows:
having given the number of instances in
which things are both thus and so, in
which they are thus but not so, in which
they are so but not thus, and in which
they are neither thus nor so, it is
required …to determine the quantitative
relativity between the thusness and the
soness of the things.”
3
Jeopardy: Statistics for 100
What is the chi-square test?
Thus
Not Thus
So
Not So
Bulletin of the Philosophical Society of
Washington, (1888).
4
Jeopardy: Statistics for 800
The book this is a quote
from:…“Grown-ups love figures. When
you tell them you have made a new
friend, they never ask you about
essential matters. They never ask you,
“What games does he love best?
…Instead they demand, “How old is
he? How much money does his father
make?” Only from these figures do they
think they have learned anything about
him.”
5
Jeopardy: Statistics for 800
What is “The Little Prince?”
6
What is the chi-square test?
• A statistical test used to compare groups
on the percentage with a condition or
outcome
• Also known as contingency table
7
Assumptions of 
2
1. Frequency data
2. Random sample
3. Adequate sample size
4. Adequate expected cell counts
5. Independent observations
8
Chi-square hypothesis
• Null hypothesis:
– There is no difference in the proportions of the two treatment
groups
– There is not an association between the two variables
– The outcome (survival) is INDEPENDENT of the treatment group
– H0: P1=P2
• Alternative hypothesis:
– There is a difference in the proportions of the two treatment
groups
– There is an association between the two variables
– The outcome (survival) is DEPENDENT on the treatment group
(the 2 variables are not independent and therefore survival
depends on which group you are in)
– HA:P1≠P2
9
Chi-square Example
• Suppose we are interested in studying a
drug that we hope will increase the 2-year
survival of patients following an M.I.
• We have 46 patients in each of 2 groups
(treatment vs. placebo).
• Furthermore, suppose we lose to follow-up
one patient in the experimental drug
group.
10
Example (continued)
• Null hypothesis:
– H0: P1=P2
– There is no difference in the proportions of the two treatment
groups.
– The outcome (survival) is INDEPENDENT of treatment group
• Alternative hypothesis:
– HA:P1≠P2
– There is a difference in the proportions of the two treatment
groups;
– The outcome (survival) is DEPENDENT of the treatment group
(survival and treatment are not independent of each other and
therefore survival depends on which treatment group you are in)
11
Example (continued)
Drug
Outcome
Experimental
Drug
Control
Survived
38
29
67
Died
7
17
24
45
46
91
Marginal totals
Marginal
totals
Grand
Total
12
Example (continued)
• In the literature we could report this data as:
 % surviving at 2 years with experimental drug
38
=
= 84%
45

29
% surviving at 2 years with control =
= 63%
46
13
Example (continued)
Drug
Exp. Drug
Outcome
Survived
Died
Control
38
29
67
7
17
24
46
91
45
The 4 cells of this table are OBSERVED values
We must calculate what is known as EXPECTED values one would expect
if the 2 variables were INDEPENDENT
14
Example (continued)
Drug
Exp. Drug
Outcome
Survived
Died
Control
38
29
67
7
17
24
46
91
45
To calculate the critical value you need to first calculate the degrees of freedom
and the expected value for each cell.
Expected values:
Chi-square:
2
2
=
# in row x # in column
total number
1df

(O  E )

E
15
Example (continued)
df  (# rows  1)  (# columns  1)
df  (2  1)  (2  1)
df  1



2
1df
2
1df
2
1df
(O  E ) 2

E
(38  33.1) 2 (29  33.9) 2 (7  11.9) 2 (17  12.1) 2




33.1
33.9
11.9
12.1
 5.37, p  0.02
16
Chi-Square Table
17
Chi-Square Table
5.024
6.635
So, 0.025 > p > 0.01
Computers give exact calculation, p=0.02
Conclude: The percentage of people who survived at 2 years was significantly
higher in the experimental drug group (84%) than in the control group (63%).
18
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone, R. A., MD, MPH; Hanseman, D. J., PhD; Robinson, B.
R. H., MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult
versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109-116.
Injury Type
Adult Trauma
Pediatric
Center
Trauma Center
Total
N
N
N
3,175
1,676
4,851
Penetrating
618
321
939
Burn
82
66
148
Asphyxia
12
26
38
3,887
2,089
5,976
Blunt
Total
The null hypothesis says that
there is not an association
between injury type and trauma
center (they are independent)
The alternative hypothesis says
there is an association between
injury type and trauma center
(they are dependent)
19
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone, R. A., MD, MPH; Hanseman, D. J., PhD; Robinson, B.
R. H., MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult
versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109-116.
Injury Type
Adult Trauma
Pediatric
Center
Trauma Center
Total
Injury Type
Column %
Column %
N
Blunt
81.7%
80.2%
4,851
Penetrating
15.9%
15.3%
Burn
2.1%
Asphyxia
Total
Adult Trauma
Pediatric
Center
Trauma Center
Total
Row %
Row %
N
Blunt
65.5%
34.5%
4,851
939
Penetrating
65.8%
34.2%
939
3.1%
148
Burn
55.4%
44.6%
148
0.3%
1.2%
38
Asphyxia
31.6%
68.4%
38
3,887
2,089
5,976
Total
3,887
2,089
5,976
20
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone, R. A., MD, MPH; Hanseman, D. J., PhD; Robinson, B.
R. H., MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult
versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109-116.
Injury Type
Adult Trauma
Pediatric
Center
Trauma Center
Total
N
N
N
3,175
1,676
4,851
Penetrating
618
321
939
Burn
82
66
148
Asphyxia
12
26
38
3,887
2,089
5,976
Blunt
Total
Degree of Freedom:
• df = (# rows - 1) x (# columns - 1)
• Injury Type:
4 - 1=3
• Type of Center:
2 - 1=1
• df = 3 x 1 = 3
21
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone, R. A., MD, MPH; Hanseman, D. J., PhD; Robinson, B.
R. H., MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult
versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109-116.
Injury Type
Adult Trauma
Pediatric
Center
Trauma Center
Total
N
N
N
3,175
1,676
4,851
Penetrating
618
321
939
Burn
82
66
148
Asphyxia
12
26
38
3,887
2,089
5,976
Blunt
Total
Expected values:
=
# in row x # in column
total number
22
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone, R. A., MD, MPH; Hanseman, D. J., PhD; Robinson, B.
R. H., MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult
versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109-116.
Injury Type
Adult Trauma
Pediatric
Center
Trauma Center
Total
Expected values:
=
# in row x # in column
total number
Blunt injury – Adult Trauma Center:
= 4,851 x 3,887 = 3,155.3
5,976
N
N
N
3,175
1,676
4,851
Penetrating
618
321
939
Observed n = 3,175
Burn
82
66
148
Expected n = 3,155.3
Asphyxia
12
26
38
3,887
2,089
5,976
Blunt
Total
23
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone, R. A., MD, MPH; Hanseman, D. J., PhD; Robinson, B.
R. H., MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult
versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109-116.
Injury Type
Adult Trauma
Pediatric
Center
Trauma Center
Total
N
N
N
3,175
1,676
4,851
Penetrating
618
321
939
Burn
82
66
148
Asphyxia
12
26
38
3,887
2,089
5,976
Blunt
Total
Expected values:
=
# in row x # in column
total number
Burn injury – Peds. Trauma Center:
= 148 x 2,089 = 51.7
5,976
Observed n = 66
Expected n = 51.7
24
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone, R. A., MD, MPH; Hanseman, D. J., PhD; Robinson, B.
R. H., MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult
versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109-116.
Observed and Expected Values
Adult Trauma Center
Pediatric Trauma Center
Total
N
N
N
3,175 (3155.3)
1,676 (1695.7)
4,851
618 (610.8)
321 (328.2)
939
Burn
82 (96.3)
66 (51.7)
148
Asphyxia
12 (24.7)
26 (13.3)
38
3,887
2,089
5,976
Injury Type
Blunt
Penetrating
Total
25
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone, R. A., MD, MPH; Hanseman, D. J., PhD; Robinson, B.
R. H., MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult
versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109-116.
Chi-Square Calculation
26
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone, R. A., MD, MPH; Hanseman, D. J., PhD; Robinson, B.
R. H., MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult
versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109-116.
Degrees of freedom = 3
Critical value = 25.33
27
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone, R. A., MD, MPH; Hanseman, D. J., PhD; Robinson, B.
R. H., MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult
versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109-116.
Degrees of freedom = 3
Critical value = 25.33
28
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone, R. A., MD, MPH; Hanseman, D. J., PhD; Robinson, B.
R. H., MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult
versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109-116.
Degrees of freedom = 3
Critical value = 25.33
Chi-square value of 25.33 and 3 degrees of freedom is greater than
12.838. Therefore, the p-value is less than 0.005. For adolescent patients,
there is a statistically significant difference in the distribution of injury
types between adult and pediatric trauma centers
29
Walther, A. E., MD; Pritts, T. A., MD, PhD; Falcone,
R. A., MD, MPH; Hanseman, D. J., PhD; Robinson,
B. R. H., MD. (2014). Teen Trauma without the
Drama: Outcomes of Adolescents Treated at
Ohio Adult versus Pediatric Trauma Centers.
Journal of Trauma Acute Care Surgery, vol. 77 (n.
1), pp. 109-116.
Our calculated significance:
• P-value:
< 0.005
Therefore, reject the null hypothesis and
state there is an association between
type of trauma center and injury type.
30
Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric
Trauma Centers.
SAS Output:
Our calculations:
• Df:
3
• Critical value: 25.33
• P-value:
< 0.005
31
Key points
 The further apart observed values are from expected values then the larger the chisquare value. This goes back to our null hypothesis that the two things we are
studying are independent…in other words, the less independent two things are, the
further apart the observed and expected values will be.
 If the chi-square equals 0, then proportions perfectly match the hypothesis.
 When the chi-square value is large, it leads to a smaller p-value and rejection of the
null hypothesis.
 You need a larger sample size for categorical comparisons compared to the sample
size need for numeric comparisons (example, age in categories vs. numeric age)
 Tests ASSOCIATION, NOT CAUSE & EFFECT!!
 If the two variables being compared are dependent (paired) samples, McNemar’s
test should be used.
 If the grand total is <40 (some say<20) or if the expected value of any cell is <5
(some say <1), then use Fisher’s Exact test.
32
Sample Size Calculations for Chi-Square
To calculate the number of
patients needed in an
experimental and a control
group for a given probability
of obtaining a significant
result (two-sided test), you
need to know (or educated
guess):
1.Smaller proportion
2.Difference in
proportions
3.Alpha
4.Power
From: Career Medicine – Holland, JF,& Frei E.I. Chapter 8
33
Sample Size Example 1
Suppose the failure rate of a drug is 40%. You would like to decrease that
rate to 20% with a new treatment and design a study comparing the two
drugs. Using the table, how many subjects are needed in each group for an
alpha level of 0.05 and power of 0.80?
34
Sample Size Example 1
Suppose the failure rate of a drug is 40%. You would like to decrease that
rate to 20% with a new treatment and design a study comparing the two
drugs. Using the table, how many subjects are needed in each group for an
alpha level of 0.05 and power of 0.80?
1. Smaller proportion:
20%
2. Difference in proportions:
20%
3. Alpha:
.05
4. Power:
.80
35
Sample Size Example 1
Suppose the failure rate of a drug is 40%. You would like to decrease that
rate to 20% with a new treatment and design a study comparing the two
drugs. Using the table, how many subjects are needed in each group for an
alpha level of 0.05 and power of 0.80?
1. Smaller proportion:
20%
2. Difference in proportions:
20%
3. Alpha:
.05
4. Power:
.80
Answer:
At least 80 per group
36
Sample Size Example 2
Suppose the failure rate of a drug is 40%. You would like to decrease that
rate to 20% with a new treatment and design a study comparing the two
drugs. Using the table, how many subjects are needed in each group for an
alpha level of 0.05 and power of 0.90?
37
Sample Size Example 2
Suppose the failure rate of a drug is 40%. You would like to decrease that
rate to 20% with a new treatment and design a study comparing the two
drugs. Using the table, how many subjects are needed in each group for an
alpha level of 0.05 and power of 0.90?
1. Smaller proportion:
20%
2. Difference in proportions:
20%
3. Alpha:
.05
4. Power:
.90
38
Sample Size Example 2
Suppose the failure rate of a drug is 40%. You would like to decrease that
rate to 20% with a new treatment and design a study comparing the two
drugs. Using the table, how many subjects are needed in each group for an
alpha level of 0.05 and power of 0.90?
1. Smaller proportion:
20%
2. Difference in proportions:
20%
3. Alpha:
.05
4. Power:
.90
Answer:
At least 105 per group
39
Sample Size Example 3
Suppose the failure rate of a drug is 40%. You would like to decrease that
rate to 30% with a new treatment and design a study comparing the two
drugs. Using the table, how many subjects are needed in each group for an
alpha level of 0.05 and power of 0.80?
40
Sample Size Example 3
Suppose the failure rate of a drug is 40%. You would like to decrease that
rate to 30% with a new treatment and design a study comparing the two
drugs. Using the table, how many subjects are needed in each group for an
alpha level of 0.05 and power of 0.80?
1. Smaller proportion:
30%
2. Difference in proportions:
10%
3. Alpha:
.05
4. Power:
.80
41
Sample Size Example 3
Suppose the failure rate of a drug is 40%. You would like to decrease that
rate to 30% with a new treatment and design a study comparing the two
drugs. Using the table, how many subjects are needed in each group for an
alpha level of 0.05 and power of 0.80?
1. Smaller proportion:
30%
2. Difference in proportions:
10%
3. Alpha:
.05
4. Power:
.80
Answer:
At least 360 per group
42