Statistics in Education
for
Mere Mortals
Inferential Statistics
Lloyd P. Rieber
Professor of Learning, Design, & Technology
The University of Georgia
© 2013 Lloyd P. Rieber
Huh?
Example of reporting a test of a statistical hypothesis:
Percentage means and standard deviations are
contained in Table 1. A significant main effect was
found on the test of learning outcomes,
F(1, 97) = 9.88, p <.05, MSerror = 190.51. Participants
given the educational game scored significantly
higher (mean=91.5%) than participants who were
not given the game (mean=71.2%).
© 2013 Lloyd P. Rieber
Running an Olympic Marathon:
No Significant Difference?
• 26 miles, 385 yards
• Times of top 2 women runners at 2012
Olympics in London:
– 1. Tiki Gelana, Ethiopia, 2:23:07
– 2. Priscah Jeptoo, Kenya, 2:23:12
• Is a difference of 5 seconds statistically
significant?
© 2013 Lloyd P. Rieber
Total votes cast for
Bush or Gore in 2000:
No Significant
Difference?
© 2013 Lloyd P. Rieber
A statistically significant
difference is not
necessarily an important
difference
© 2013 Lloyd P. Rieber
Ready to buy?
• I invented a new way to teach ________.
• When I compared my invention to the
traditional approach, students scored 5% higher
on the test.
• Are you convinced it is a better approach?
© 2013 Lloyd P. Rieber
Convinced yet?
Trial
Score of
My Invention
Score of
Traditional
Approach
1
85
80
2
86
82
© 2013 Lloyd P. Rieber
Convinced yet?
Trial
Score of
My Invention
1
85
Score of
Traditional
Approach
80
2
86
82
3
84
79
4
85
81
5
83
85
© 2013 Lloyd P. Rieber
Hypothesis Testing
An Example of Inferential Statistics
Hypothesis testing basically answers the
question:
• If we repeat the experiment how many
times, out of 100, would my invention have
to produce a higher score for you to be
convinced it is truly a better approach?
© 2013 Lloyd P. Rieber
Hypothesis Testing
An Example of Inferential Statistics
• The hypothesis that there is no difference is called
the null hypothesis.
• p is the probability that the experimental results
show a difference, when in fact there is no
difference.
• So, we hope that this probability is very small.
• How low is acceptable?
General answer: No more than 5 times out of
100, or p < .05
© 2013 Lloyd P. Rieber
Does My Invention Work?
Let’s consider the research
outcome possibilities…
© 2013 Lloyd P. Rieber
Reality
My conclusion,
based on
experiment
It truly does not
work
It works.
ERROR
It doesn’t work.
It really works!
It works.
It doesn’t work.
ERROR
© 2013 Lloyd P. Rieber
Experimental Designs
• Experimental design is used to identify cause-andeffect relationships.
• The researcher considers many possible factors that
might cause or influence a particular
condition/phenomenon. The researcher controls for
all influential factors except those having possible
effects.
• The importance of random selection and assignment
of participants.
© 2013 Lloyd P. Rieber
Independent and Dependent Variables
• Variable: any quality or characteristic in a
research investigation that has two or more
possible values.
• Independent variable: a possible cause of
something else (the manipulated variable)
• Dependent variable: a variable that is potentially
influenced by the independent variable.
© 2013 Lloyd P. Rieber
Comparing Means: Hypothesis Testing
• Comparing two means
– The t statistic
– t tests
• Comparing more than two means
– The F statistic
– Analysis of Variance
© 2013 Lloyd P. Rieber
Two Typical Uses of t Tests in Education &
Training
• Dependent (Correlated) t Test
One-group pretest-posttest design
You select one group of people at random for your evaluation.
Procedure:
• Administer a pretest (observation 1);
• Group participates in treatment (i.e. activity, intervention, etc.);
• Administer a posttest (observation 2);
Is there a difference between the pretest mean and posttest mean?
Group
Group1
Time
Obs 1
Trt
Obs 2
© 2013 Lloyd P. Rieber
Two Typical Uses of t Tests in Education &
Training
• Independent t Test
Posttest-only control group design
You select two groups of people at random for your evaluation.
Procedure:
• You randomly assignGroup
the peopleTime
to the two groups (with equal
numbers in each group);
Trt(i.e. activity,
Obs intervention, etc.);
• Group 1Random
participatesGroup1
in treatment
• GroupAssignment
2 does not; Group2
Obs
• Administer a posttest to both groups (observation);
Is there a difference between the means of the two groups?
© 2013 Lloyd P. Rieber
What If You Have More
Than Two Groups?
© 2013 Lloyd P. Rieber
Analysis of Variance
F=
Between Groups Variance
Within Groups Variance
© 2013 Lloyd P. Rieber
Sources of
Error
(think variability)
© 2013 Lloyd P. Rieber
Remember the hanging chads?
A good example of “error in measurement.”
© 2013 Lloyd P. Rieber
Quick, answer this question…
• Bob has five cookies. Jim has four
cookies. After Bob gives Jim two
more cookies, how many does he
have?
7
6
5
4
3
2
Answer choices
© 2013 Lloyd P. Rieber
Degrees of Freedom
© 2013 Lloyd P. Rieber
Understanding Degrees of Freedom
I’m thinking of 5 numbers with a mean
of 46, can you guess what they are?
1. __________
2. __________
3. __________
4. __________
5. __________
© 2013 Lloyd P. Rieber
Understanding Degrees of Freedom
I’m thinking of 5 numbers with a mean
of 46, can you guess what they are?
1. 34
2. 44
3. 93
4. 18
41
5. __________
© 2013 Lloyd P. Rieber
F(1, 97) = 9.88, p <.05, MSerror = 190.51
dftreatment
dferror
F=
=
=
Between Groups
Variance
Within Groups
Variance
Number of groups - 1
Ntotal- Number of groups
SStreatment
MStreatment
=
MSerror
=
dftreatment
SSerror
dferror
© 2013 Lloyd P. Rieber
Girl Scout Cookie Sales
Boxes of Cookies
Deviation Scores
X
X-X
x2
28
18
324
11
1
1
10
0
0
5
-5
25
4
-6
36
2
-8
64
∑X=60
∑x=0
∑x2=450
å X = 60 = 10
X=
N
6
Example taken from Spatz, 1997.
S=
å(X - X )
N
2
=
åx
N
2
=
450
= 75 = 8.66
6
© 2013 Lloyd P. Rieber
Statistics in Education
for
Mere Mortals
Inferential Statistics
Lloyd P. Rieber
Professor of Learning, Design, & Technology
The University of Georgia
© 2013 Lloyd P. Rieber