Reasoning in Psychology
Using Statistics
Psychology 138
2017
• Quizzes 1 & 2 are posted
– Quiz 2 due Fri (Feb 3)
• Don’t forget Exam 1 is coming up (Feb 8)
– In class part – multiple choice, closed book
– In labs part – open book/notes
• Today
– Nuts and bolts of experimental & observational designs
Announcements
Social Science Reasoning
Using Statistics
• Generally the process involves a number of steps:
– identification of your research questions
– identifying your variables
– specify your hypotheses (how are the variables related
to one another)
– selecting a research design
– collecting your data, analyzing your data
– drawing conclusions from your data about your
hypotheses
Designing your research
Social Science Reasoning
Using Statistics
Today’s focus
• Generally the process involves a number of steps:
– identification of your research questions
– identifying your variables
– specify your hypotheses (how are the variables related
to one another)
– selecting a research design
– collecting your data, analyzing your data
– drawing conclusions from your data about your
hypotheses
Designing your research
Social Science Reasoning
Using Statistics
Context
•
•
•
Research
design
Measurement
scale
What is known
about the
distributions
Social Science Reasoning
Using Statistics
Statistical
tests
•Which tests to
conduct
•How to set up
your datafile
Design Decision Tree
• Issue: What is the best way to study for a test?
Cram the night before
- or Spread your studying out over several nights
• How could we investigate this
question?
– What research methods should
we use?
Selecting your research method
Social Science Reasoning
Using Statistics
• Issue: What is the best way to study for a test?
Observational
– involve examining variables
as they already are
Experimental
– involve manipulation of
variables
• How could we investigate this
question?
– What research methods should
we use?
Selecting your research method
Social Science Reasoning
Using Statistics
• Issue: What is the best way to study for a test?
Observational
– Randomly select
individuals
– Watch their study
habits
– See how they do on a
test
Experimental
– Randomly select
individuals
– Randomly assign to
groups
• Crammed study group
• Distributed study group
– See how they do on a
test
Selecting your research method
Social Science Reasoning
Using Statistics
• Issue: What is the best way to study for a test?
Observational
– Randomly select
individuals
– Watch their study
habits
– See how they do on a
test
• Here our analysis looks for a
relationship between the
explanatory (study method)
and response (test
performance) variables
• The scale of measurement
used for the variables is
important
Observational designs
Social Science Reasoning
Using Statistics
• Why random assignment?
• To try to minimize bias,
potential systematic
differences between the
people in the different groups
Experimental
– Randomly select
individuals
– Randomly assign to
groups
• Crammed study group
• Distributed study group
– See how they do on a
test
Experiments
Social Science Reasoning
Using Statistics
• Why 2 groups?
– All experiments make some sort
of comparison.
Experimental
– Randomly select
individuals
– Randomly assign to
groups
• Crammed study group
• Distributed study group
– See how they do on a
test
Experiments
Social Science Reasoning
Using Statistics
• Why 2 groups?
– All experiments make some sort
of comparison.
If only one group:
– There is natural variability in your DV.
– No comparison group, so hard to see if:
change in the IV
change in the DV
– Should not make causal claims
•
This is an observational design called “one-shot case study design”
?? Experiment ??
• Randomly select
individuals
• Tell 10 students to
distribute their studying
• See how they do on a
test
Special case: when our “comparison”
group is the population
Experiments
Social Science Reasoning
Using Statistics
• Why 2 groups?
– All experiments make some sort
of comparison.
The systematic reduction and control of
variability of the IV(s) allows us to examine
whether they are sources of (causes) variability in
the DV
Experiment
If two groups:
– Still some natural variability in your DV.
– But have constrained one source of the
variability (crammed or distributed studying)
– Can see if there is an effect of the
manipulation
change in the IV
change in the DV
We will discuss these designs with
two groups later in the semester
Experiments
Social Science Reasoning
Using Statistics
• Randomly select
individuals
• Randomly assign to groups
– Crammed study group
– Distributed study group
• See how they do on a test
• Issue: What is the best way to study for a test?
• Variables
– Independent
– Dependent
– Extraneous
• Control
• Random
• Confound
Experiments
Social Science Reasoning
Using Statistics
Experiment
• Randomly select
individuals
• Randomly assign to groups
– Crammed study group
– Distributed study group
• See how they do on a test
• Issue: What is the best way to study for a test?
• Variables
– Independent
– Dependent
– Extraneous
• Control
• Random
• Confound
Experiment
– These are the variables that •are Randomly
manipulated
by the
select
experimenter
individuals
• Randomly
• A number of ways to manipulate
your IVassign to groups
– Crammed
study group
– Event/Stimulus manipulations
– manipulate
– context,
Distributed
characteristics of the stimuli,
etc.study group
• See how
they dogroups
on a test
– Instructional manipulations
– different
are given different instructions
– Subject manipulations – there are (pre-existing
mostly) differences between the subjects in the
different conditions (typically results in quasiexperimental designs)
– Will typically use a Categorical Scale of Measurement
– Nominal or Ordinal – names corresponding to
different levels of the IV
Experiments
Social Science Reasoning
Using Statistics
• Issue: What is the best way to study for a test?
• Variables
1 IV (factor): study type
– Independent
– Dependent2 levels:
study
Distributed study
–Crammed
Extraneous
Study all
in one night
• Control
Study over five nights
• Random
• Confound
Experiments
Social Science Reasoning
Using Statistics
Experiment
• Randomly select
individuals
• Randomly assign to groups
– Crammed study group
– Distributed study group
• See how they do on a test
• Issue: What is the best way to study for a test?
• Variables
Experiment
– Independent
• Randomly select
– Dependent • These are the variables that are measured
individualsby
the experimenter, they are “dependent”
onassign to groups
– Extraneous
• Randomly
the independent variables
• Control
– Crammed study group
• Random • For most of our designs these will
– be
Distributed study group
measured on Continuous Scales
of how they do on a test
• Confound
• See
Measurement (Ratio or Interval)
Test performance
Experiments
Social Science Reasoning
Using Statistics
• Issue: What is the best way to study for a test?
• Variables
Experiment
– Independent
– Dependent
– Extraneous
• Randomly select
individuals
• Randomly assign to groups
• Control
– Crammed study group
– Holding things constant
• Random
– Distributed
- Controls for excessive random
variabilitystudy group
• Confound
• See how they do on a test
Crammed study
Experiments
Study for 5 hours
Social Science Reasoning
Using Statistics
Distributed study
Study for 1 hour each night
Total study time = 5 hours
• Issue: What is the best way to study for a test?
• Variables
– Independent
– Dependent
– Extraneous
Experiment
• Randomly select
individuals
• Randomly assign to groups
• Control
– Crammed study group
• Random - May freely vary, to spread variability
– Distributed
study group
equally across
• Confound all experimental conditions
• See how they do on a test
Experiments
Social Science Reasoning
Using Statistics
• Issue: What is the best way to study for a test?
• Variables
Experiment
– Independent
– Dependent
– Extraneous
• Control
• Random
• Confound
• Randomly select
individuals
• Randomly assign to groups
– Crammed study group
– Distributed study group
– Other variables, that haven’t• been
Seeaccounted
how they for
do on a test
(manipulated, measured, randomized, controlled) that
co-vary with the IV(s) and can impact changes in the
dependent variable(s)
Experiments
Social Science Reasoning
Using Statistics
• Issue: What is the best way to study for a test?
• Variables
– Independent
Underlying third variable
– Dependent
that varies with the IV
– Extraneous
• Control
• Random
• Confound
Crammed study
Distributed study
• Randomly select
Experiment
individuals
• Randomly
assign
Studied vocabulary
Studied
onlyto groups
study group
and practice problems– Crammed
vocabulary
Test performance
– Distributed study group
Is this difference
duethey
to study
• See how
do onmethod?
a test
Or is it due to what was studied?
85%
Experiments
Social Science Reasoning
Using Statistics
73%
• Which research method is best? - it depends
Observational
Advantages
- May see patterns of complex behaviors
- Good first step
- May learn about something unexpected
Disadvantages
- Shouldn’t make causal claims
- Directionality of the relationship isn’t
known
- Threats to internal validity due to lack
of control
- Sometimes the results are not
reproducible
Experimental
- Precise control possible
- Precise measurement possible
- Can make causal claims
- Artificial situations may restrict
generalization to “real world”
- Complex behaviors may be difficult
to measure
Selecting your research method
Social Science Reasoning
Using Statistics
Your design largely determines
your statistical analysis
Social Science Reasoning
Using Statistics
Design Decision Tree
Your design largely determines
your statistical analysis
Social Science Reasoning
Using Statistics
Design Decision Tree
Which test do I use?
•Students often find this one of
the hardest parts of the course –
the key is understanding the
context
SPSS
• Today’s lab
– Practice developing and evaluating experiments
• Questions?
Statistics Learning Centre Choosing which statistical test to use (~10 mins)
Erich Goldstein Choosing a Statistical Test (~13 mins)
RStatsInstitute Independent vs. Paired t Tests Setup in SPSS (~3 mins)
Dr. Todd Grande Comparison of Independent-Samples T
Test and Paired-Samples T Test using SPSS (~20 mins)
Wrap up
Social Science Reasoning
Using Statistics