Nonparametric Tests of Significance
Statistics for Political Science
Levin and Fox
Chapter Nine
Part One
What is a parametric test?
When a test of significance requires:
1.
2.
Normality in the population (a normal distribution) or at least large
samples so that the sampling distribution is normal.
An interval-level measure.
Non-parametric Tests
Nonparametric tests of significance have a list of requirements that do not
include normality or the interval level of measurement.
Nonparametric tests use a concept of power (power of a test): the
probability of rejecting the null hypothesis when it is false and should be
rejected. In other words, the probability of accurately claiming a
statistically significant relationship does exist between two variables.
Non-parametric Tests
Power varies from test to test:
The more powerful the test, the more likely the null hypothesis is to be
rejected when it is false and they have more difficult requirements to
satisfy.
Less powerful tests have less stringent requirements on the data and the null
hypothesis may be retained when it should be rejected.
Accuracy of different tests:
The more powerful the test is, the more likely it is to accurately determine
whether or not a statistically significant relationship does exist between
variables.
Nonparametric Tests:
Two Nonparametric Tests:
The Chi-Square Test: concerned with the distinction between expected
frequencies and observed frequencies.
The Median Test: A chi-square based test that also evaluates whether the
scores fall above or below the median.
Nonparametric Tests:
Some things to know about chi square:
1) It compares the distribution of one variable (DV) across the
category of another variable (IV)
2) It makes comparisons across frequencies rather than mean scores.
3) It is a comparison of what we expect to what we observe.
Null versus Research Hypotheses:
The null hypotheses states that the populations do not differ with respect to
the frequency of occurrence of a given characteristic, whereas a
research hypothesis asserts that sample difference reflects population
difference in terms of the relative frequency of a given characteristic.
Nonparametric Tests:
Chi Square: Example: Political Orientation and Child Rearing
Null Hypothesis:
The relative frequency or percentage of liberals who are permissive IS
the same as the relative frequency of conservatives who are
permissive.
Research Hypothesis:
The relative frequency or percentage of liberals who are permissive is
NOT the same as the relative frequency of conservatives who are
permissive.
Nonparametric Tests:
Chi Square: Example: Political Orientation and Child Rearing
Expected and Observed Frequencies:
The chi-square test of significance is defined by Expected and Observed
Frequencies.
Expected Frequencies (fe) is the frequency we would expect to get if
the hull hypothesis is true, that is there is no difference between the
populations.
Observed Frequencies (fo) refers to results we actually obtain when
conducting a study (may or may not vary between groups).
Only if the difference between expected and observed frequencies is large enough do
we reject the null hypothesis and decide that a population difference does exist.
Nonparametric Tests:
Chi Square: Political Orientation and Child Rearing: Observed Frequencies
Political Orientation
Child-Rearing
Methods
Permissive
Not Permissive
Total
Row
Marginal
Liberals
Conservatives
13
7
20
7
13
20
20
20
N = 40
Col.
Marginal
Total
Chi Square: Example: Political Orientation and Child Rearing
Since the marginals are all equal, it is easy to calculate the expected
frequencies: 10 in each cell.
Political Orientation
Child-Rearing
Methods
Permissive
Not Permissive
Total
Liberals
Conservatives
10
10
20
10
10
20
20
N = 40
20
Total
It is unusual for a study to produce row and column marginals that are
evenly split.
Chi Square: Example: Political Orientation and Child Rearing
Calculating expected frequencies when the marginals are not even:
Political Orientation
Child-Rearing
Methods
Permissive
Not Permissive
Total
Row
Marginal
Liberals
Conservatives
15
10
25
5
10
15
20
20
N = 40
Total
To determine if these frequencies depart from what is expected (null) by chance
alone, we have to calculate the expected frequencies.
If 25 of 40 respondents are permissive, than 62.5 % of them are permissive.
To then determine the expected frequency, which asserts that Libs and
Cons are the same (null) we have to calculate what would be 62.5% of 20
Libs and 20 Cons (the number of each that are in the study.
Political Orientation
Child-Rearing
Methods
Liberals
Conservatives
Permissive
15 (12.5)
10 (12.5)
25 (62.5%)
5 (7.5)
10 (7.5)
15
20
N = 40
Not Permissive
Total
20
Total
The answer is 12.5 (62.5% of 20 or .625 x 20). We then know that the expected
frequency for non permissive is 7.5 (20 – 12.5).
Calculating Expected Frequencies
fe = (column marginal)(row marginal)
N
Example:
fe = (20)(25)
40
= 500
40
= 12.5
Example: fe =
(25)(20)
40
= 500
40
= 12.5
Child-Rearing
Methods
Permissive
Not Permissive
Total
Political Orientation
Liberals
Conservatives
15 (12.5)
10 (12.5)
25 (62.5%)
5 (7.5)
10 (7.5)
15
20
N = 40
20
Total
The answer is 12.5 (62.5% of 20 or .625 x 20). We then know that the expected
frequency for non permissive is 7.5 (20 – 12.5).
The Chi-Square Test Formula
Once we have the observed and expected frequencies we can use the
following formula to calculate Chi-square.
2
( fo
fe )
fe
Where:
fo = observed frequency in any cell
fe = expected frequency in any cell
2
Nonparametric Tests: Chi-Square Tests
Observed
Expected
Subtract
Square
Divide by fe
Sum
After obtaining fo and fe, we subtract fe from fo, square the difference,
divide by the fe and then add them up.
Nonparametric Tests: Chi-Square Tests
Formula for Finding the Degrees of Freedom
df = (r-1)(c-1)
Where
r = the number of rows of observed frequencies
c = the number of columns of observed frequencies
Formula for Finding the Degrees of Freedom
Formula for Finding the Degrees of Freedom
Since there are two rows and two columns of observed frequencies in our 2 x
2 table
df = (r-1)(c-1)
df = (2-1)(2-1)
= (1)(1)
=1
Next Step, Table E, where we will find a list of chi-square scores that are
significant at .05 and .01 levels.
Table E (.05, df = 1): 3.84
Obtained2 X = 2.66
Retain null
Step by Step Chi-Square Test:
Step by Step Chi-Square Test:
1) Subtract each expected frequency from its corresponding observed
frequency
2) Square the difference
3) Divide by the expected frequency, and then
4) Add up these quotients for all the cells to obtain the chi-square value
Nonparametric Tests:
Comparing Several Groups
When comparing more than two groups, you use essentially the same
process as when comparing 2 x 2 tables.
Nonparametric Tests:
Chi Square: Example: Political Orientation and Child Rearing: 2 x 2:
Null Hypothesis:
The relative frequency of permissive, moderate, and authoritarian
child-rearing methods IS the same for Protestants, Catholics, and
Jews.
Research Hypothesis:
The relative frequency of permissive, moderate, and authoritarian
child-rearing methods is NOT the same for Protestants, Catholics, and
Jews.
Comparing Several Groups
Chi Square: Example: Political Orientation and Child Rearing
Calculating expected frequencies when the marginals are not even:
Political Orientation
Child-Rearing
Methods
Liberals
Conservatives
Permissive
15 (12.5)
10 (12.5)
25 (62.5%)
5 (7.5)
10 (7.5)
15
20
N = 40
Not Permissive
Total
20
Total
Comparing Several Groups
Chi Square: Example: Political Orientation and Child Rearing
Child-Rearing
Methods
Liberals
Conservatives
Permissive
15 (12.5)
10 (12.5)
25 (62.5%)
5 (7.5)
10 (7.5)
15
20
N = 40
Not Permissive
Total
20
Total
Correcting for Small Frequencies
Generally, chi square should be used with great care whenever some of the
frequencies are below Five (5).
Though, this is not a hard and fast rule.
Yate’s Correction
HOWEVER, when working with a 2x2 table where any expected frequency is
less than 10 but greater than 5, use Yate’s correction which reduces the
difference between the expected and observed frequencies.
2
fo
fe
.5
2
fe
The vertical indicate that we must reduce the absolute value (ignoring
minus signs) of each fo – fe by .5
Yate’s Correction
Nationality
Smoking Status
American
Canadian
Nonsmokers
15 (11.67)
5 (8.33)
20
Smokers
6 (9.33)
10 (6.67)
21
15
16
N = 36
Total
Observed
Expected
Subtract
Subtract .5
Square
Divide by fe
Sum
Requirements for the use of Chi-Square
Requirements for the use of Chi-Square:
1.
A comparison between two or more samples.
2.
Nominal data must be used.
3.
Samples should have been randomly selected.
4.
The expected cell frequencies should not be too small.
Median Test
Median Test:
Is a simply nonparametric test for determining the likelihood that two or
more random samples have been taken from populations with the
same median.
It involves conducting a Chi-Square test where one of the dimensions is
whether the scores fall above or below the median of the two groups
combined.
Median Test
Median Test: Example: Gender and Embarrassment
Experiment asked 15 men and 12 women with average singing ability to sing
in front of expert judges (sound familiar?)
Men (N = 15)
Women (N = 12)
4
6
9
11
12
15
16
18
19
21
23
24
25
26
27
1
2
3
5
7
8
10
13
14
17
20
22
Median Test
Median Test: Example: Gender and Embarrassment
Experiment asked 15 men and 12 women with average singing ability to sing
in front of expert judges (sound familiar?)
Step 1: Find Median of the two samples
Mdn = (N + 1) ÷ 2
= 27 + 1 ÷ 2
Mdn = 14
Median Test
Median Test: Example: Gender and Embarrassment
Experiment asked 15 men and 12 women with average singing ability to sing
in front of expert judges (sound familiar?)
Step 1: Find Median of the two samples
Mdn = (N + 1) ÷ 2
= 27 + 1 ÷ 2
Mdn = 14
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Median Test
Median Test: Example: Gender and Embarrassment
Experiment asked 15 men and 12 women with average singing ability to sing
in front of expert judges (sound familiar?)
Step 2: Count the Number in each sample falling above and below the
median
Gender
Median
Men
Women
Above
10
3
Below
5
9
Median Test
Median Test: Example: Gender and Embarrassment
Mdn = (N + 1) ÷ 2
= 27 + 1 ÷ 2
Mdn = 14
Men:
Below Mdn = 5
Above Mdn = 10
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Women:
Below Mdn = 9
Above Mdn = 3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Median Test
Median Test: Example: Gender and Embarrassment
Mdn = 14 minutes
Men
Women
4
6
9
11
12
15
16
18
19
21
23
24
25
26
27
1
2
3
5
7
8
10
13
14
17
20
22
14: 10 above,
5 below
14: 3 above,
9 below
Step 2: Count the Number in each sample falling above and below the
median
Gender
Median
Men
Women
Above
10 (7.22)
3 (5.78)
13
Below
5 (7.78)
9 (6.22)
15
12
14
N = 27
Total
Observed
Expected
Subtract
Subtract .5
Square
Divide by fe
Sum
Step 3: Calculate the Degrees of Freedom
df = (r-1)(c-1)
= (2-1)(2-1)
= (1)(1)
=1
We then go to Table E, which tells us that at .05 and a df = 1 chi-square must
exceed 3.84 to be considered statistically significant.
2
Obtained X = 3.13
Table E = 3.84
Retain Null hypothesis
There is insufficient that men and women differ in terms of embarrassment.