Chapter Ten:
Designing, Conducting,
Analyzing, and Interpreting
Experiments with Two Groups
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Experimental Design: The Basic
Building Blocks
• Experimental Design
• The general plan for selecting participants, assigning participants
to experimental conditions, controlling extraneous variables, and
gathering data.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Principle of Parsimony (Occam’s Razor)
• The belief that explanations of phenomena and events should
remain simple until the simple explanations are no longer valid.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Independent Variable (IV)
• A stimulus or aspect of the environment that the experimenter
directly manipulates to determine its influences on behavior.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Independent Variable (IV)
• A stimulus or aspect of the environment that the experimenter
directly manipulated to determine its influences on behavior.
• Chapters 10 and 11 deal with research designs that have one
IV.
• Chapter 12 deals with research designs that have more than
one IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Independent Variable (IV)
• A stimulus or aspect of the environment that the experimenter
directly manipulated to determine its influences on behavior.
• Dependent Variable (DV)
• A response or behavior that is measured. It is desired that changes
in the DV be directly related to manipulation of the IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• How Many Groups?
• Although an experiment can have only one IV, it must have at least
two groups.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• How Many Groups?
• Although an experiment can have only one IV, it must have at least
two groups.
• The simplest way to find out whether our IV caused a change
in behavior is to compare some research participants who have
received our IV to some others who have not received the IV.
• If those two groups differ, and we are assured that we
controlled potential extraneous variables, then we conclude
that the IV caused the participants to differ.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Extraneous Variables
• Undesired variables that may operate to influence the DV and,
thus, invalidate an experiment.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Levels
• The most common manner of creating two groups with one IV is to
present some amount or type of IV to one group and to withhold
that IV from the second group.
• Thus, the presence of the IV is contrasted with the absence of the IV.
• These differing levels of the IV are referred to as the levels (also
known as treatment conditions) of the IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Experimental Group
• In a two-group design, the group of participants that receives the
IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Experimental Group
• In a two-group design, the group of participants that receives the
IV.
• Control Group
• In a two-group design, the group of participants that does not
receive the IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Assigning Participants to Groups
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Assigning Participants to Groups
• Random Assignment
• A method of assigning research participants to groups so that
each participant has an equal chance of being in any group.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Assigning Participants to Groups
• Random Assignment
• A method of assigning research participants to groups so that
each participant has an equal chance of being in any group.
• Random assignment is not the same as random selection.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Assigning Participants to Groups
• Random Assignment
• A method of assigning research participants to groups so that
each participant has an equal chance of being in any group.
• Random assignment is not the same as random selection.
• When we randomly assign participants to groups, we have
created what are known as independent groups.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Assigning Participants to Groups
• Random Assignment
• A method of assigning research participants to groups so that
each participant has an equal chance of being in any group.
• Random assignment is not the same as random selection.
• When we randomly assign participants to groups, we have
created what are known as independent groups.
• When we wish to compare the performance of participants in
these two groups, we are making what is known as a betweensubjects comparison.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Independent Groups
• The participants in one group have absolutely no ties or links to the
participants in the other group.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Independent Groups
• The participants in one group have absolutely no ties or links to the
participants in the other group.
• Between-Subjects Comparison
• Refers to a contrast between groups of participants who were
randomly assigned to groups.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Confounded Experiment
• An experiment in which an extraneous variable varies
systematically with the IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Confounded Experiment
• An experiment in which an extraneous variable varies
systematically with the IV.
• Confounding makes drawing a cause-and-effect relation
impossible.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Confounded Experiment
• An experiment in which an extraneous variable varies
systematically with the IV.
• Confounding makes drawing a cause-and-effect relation
impossible.
• Confounding may occur if participants are not equal before the
start of the experiment.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Nonrandom Assignment to Groups
• Random assignment tends to create equal groups in the long run.
As groups get larger, we can place more confidence in random
assignment achieving what we want it to.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Nonrandom Assignment to Groups
• Random assignment tends to create equal groups in the long run.
• As groups get larger, we can place more confidence in random
assignment achieving what we want it to.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Nonrandom Assignment to Groups
• Random assignment tends to create equal groups in the long run.
• As groups get larger, we can place more confidence in random
assignment achieving what we want it to.
• If we are faced with a situation in which we have few potential
research participants and we are worried that random assignment
may not create equal groups, what can we do?
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Correlated Assignment
• A method of assigning research participants to groups so that there
is a relationship between small numbers of participants.
• These small groups are then randomly assigned to treatment
conditions (also known as paired or matched assignment).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Correlated Assignment
• Matched pairs
• Research participants in a two-group design who are measured
and equated on some variable before the experiment.
• Typically we measure a variable that could result in
confounding if not controlled.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Correlated Assignment
• Matched pairs
• Research participants in a two-group design who are measured
and equated on some variable before the experiment.
• Typically we measure a variable that could result in
confounding if not controlled.
• After we have measured this variable, we create pairs of
participants that are equal on this variable.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Correlated Assignment
• Matched pairs
• Research participants in a two-group design who are measured
and equated on some variable before the experiment.
• Typically we measure a variable that could result in
confounding if not controlled.
• After we have measured this variable, we create pairs of
participants that are equal on this variable.
• After we have created our matched pairs, we then
randomly assign participants from these pairs to the
different treatment conditions.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Correlated Assignment
• Matched pairs
• Repeated measures
• The same participants are tested in both treatment conditions
of our experiment.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Correlated Assignment
• Matched pairs
• Repeated measures
• The same participants are tested in both treatment conditions
of our experiment.
• The matched pairs are perfectly equal because they consist of
the same people or animals tested across the entire experiment.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Correlated Assignment
• Matched pairs
• Repeated measures
• The same participants are tested in both treatment conditions
of our experiment.
• The matched pairs are perfectly equal because they consist of
the same people or animals tested across the entire experiment.
• No extraneous variables should be able to confound this
situation because any difference between the participants’
performance in the two treatment conditions is due to the IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Correlated Assignment
• Matched pairs
• Repeated measures
• The same participants are tested in both treatment conditions
of our experiment.
• The matched pairs are perfectly equal because they consist of
the same people or animals tested across the entire experiment.
• No extraneous variables should be able to confound this
situation because any difference between the participants’
performance in the two treatment conditions is due to the IV.
• In this type of experiment, participants serve as their own
controls.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Correlated Assignment
• Matched pairs
• Repeated measures
• Natural pairs
• Pairs of participants are created from naturally occurring pairs
(e.g. biologically or socially related).
• For example, psychologists who study intelligence often
use twins as their research participants.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Within-Subjects Comparison
• Refers to a contrast between groups of participants who were
assigned to groups through matched pairs, natural pairs, or
repeated measures.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Two-Group Design
• Within-Subjects Comparison
• Refers to a contrast between groups of participants who were
assigned to groups through matched pairs, natural pairs, or
repeated measures.
• We are essentially comparing scores within the same
participants (subjects).
• Although this direct comparison is literally true only for
repeated-measures designs, participants in matched or natural
pairs are the same with regard to the matching variable.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Comparing Two-Group Designs
• Choosing a Two-Group Design
• Random assignment should should equate your groups adequately
(assuming that you have large groups).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Comparing Two-Group Designs
• Choosing a Two-Group Design
• Random assignment should should equate your groups adequately
(assuming that you have large groups).
• If you are using 20 or more participants per group, you can feel
fairly safe that randomization will create equal groups.
• If you are using 5 or fewer participants in a group,
randomization may not work.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Comparing Two-Group Designs
• Choosing a two-group design
• Advantages of correlated groups designs
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Comparing Two-Group Designs
• Choosing a two-group design
• Advantages of correlated groups designs
• Control issues
• The three methods for creating correlated-groups designs give
us greater certainty of group equality.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Comparing Two-Group Designs
• Choosing a Two-Group Design
• Advantages of Correlated Groups Designs
• Control issues
• Statistical issues
• Correlated-groups designs can help reduce error variation.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Comparing Two-Group Designs
• Error Variability
• Variability in DV scores that is due to factors other than the IV –
individual differences, measurement error, and extraneous
variation (also known as within-groups variability).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Comparing Two-Group Designs
• Choosing a Two-Group Design
• Advantages of Correlated Groups Designs
• Advantages of Independent-Groups Designs
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Comparing Two-Group Designs
• Choosing a Two-Group Design
• Advantages of Correlated Groups Designs
• Advantages of Independent-Groups Designs
• Simplicity
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Comparing Two-Group Designs
• Choosing a Two-Group Design
• Advantages of Correlated Groups Designs
• Advantages of Independent-Groups Designs
• Simplicity
• Use of correlated-groups designs is impossible in some situations.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Comparing Two-Group Designs
• True Experiment
• An experiment in which the experimenter directly manipulates the
IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Comparing Two-Group Designs
• True Experiment
• An experiment in which the experimenter directly manipulates the
IV.
• Ex Post Facto Research
• A research approach in which the experimenter cannot directly
manipulate the IV but can only classify, categorize, or measure the
IV because it is predetermined in the participants (e.g. IV = sex).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• The Relation Between Experimental Design and Statistics
• Selecting the appropriate experimental design determines the
particular statistical test you will use to analyze your data.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• The Relation Between Experimental Design and Statistics
• Selecting the appropriate experimental design determines the
particular statistical test you will use to analyze your data.
• You should determine your experimental design before you
begin collecting data to ensure there will be an appropriate
statistical test you can use to analyze your data.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• Analyzing Two-Group Designs
• For a two-independent groups design, use a t test for independent
samples to analyze your data.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• Analyzing Two-Group Designs
• For a two-independent groups design, use a t test for independent
samples to analyze your data.
• For a two-correlated-groups design, analyze your data with a t test
for correlated samples (a.k.a dependent t test, a within-groups t
test, or a paired t test).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• Interpretation: Making Sense of Your Statistics
• Statistics are a tool to help you understand the data from your
experiment.
• Statistics are useless if you don’t know how to interpret them.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• Interpreting Computer Statistical Output
• The t test for independent samples
• Homogeneity of variance
• The assumption that the variances are equal for two (or
more) groups you plan to compare statistically.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• Interpreting Computer Statistical Output
• The t test for independent samples
• Homogeneity of variance
• The assumption that the variances are equal for two (or
more) groups you plan to compare statistically.
• Heterogeneity of variance
• Occurs when we do not have homogeneity of variance.
• This means that our two (or more) groups’ variances
are not equivalent.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• Interpreting Computer Statistical Output
• The t test for independent samples
• Generally speaking, t tests are robust with regard to the
assumption of homogeneity (Kirk, 1968).
• A robust test is one that can tolerate violations of its
assumptions and still provide accurate answers.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• Interpreting Computer Statistical Output
• The t test for independent samples
• Translating statistics into words
• If two equal groups began the experiment and they are
now unequal, to what can we attribute the difference?
• If our controls have been adequate, our only choice is to
assume that the difference between the groups is due to
the IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• Interpreting Computer Statistical Output
• The t test for independent samples
• Translating statistics into words
• For example, if you were writing an interpretation of the
results from the sample experiment in your text, you
might write something like the following:
• Salesclerks who waited on well-dressed customers (M
= 43.38, SD = 10.11) took significantly less time, t(14) =
2.61, p = .021, to respond to customers than
salespeople who waited on customers dressed in
sloppy clothing (M = 63.25, SD = 12.54). The effect
size, estimated with Cohen’s d, was .92.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• Interpreting Computer Statistical Output
• The t test for correlated samples
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
Statistical Analysis: What Do
Your Data Show?
• Translating Statistics Into Words
• Here is the example from your text:
• Salespeople who waited on well-dressed customers (M = 48.38, SD
= 10.11) took significantly less time, t(7) = 5.47, p = .001, to respond
to the customers than when they waited on customers dressed in
sloppy clothes (M = 63.25, SD = 12.54). The effect size, estimated
with Cohen’s d, was 1.93.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Continuing Research
Problem
• It is rare for a psychologist to conduct a single research project
and stop at that point because that one research project had
answered all the questions about the particular topic.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Continuing Research
Problem
• Let’s review the steps involved in choosing a research design
(using the hypothetical example from the text).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Continuing Research
Problem
• After reviewing relevant research literature, we chose our IV
(style of dress) and our DV (salesclerk response time).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Continuing Research
Problem
• After reviewing relevant research literature, we chose our IV
(style of dress) and our DV (salesclerk response time).
• Because we were conducting a preliminary investigation into
the effects of clothing on salesclerks’ reactions, we decided to
test only one IV (the style of dress).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Continuing Research
Problem
• After reviewing relevant research literature, we chose our IV
(style of dress) and our DV (salesclerk response time).
• Because we were conducting a preliminary investigation into
the effects of clothing on salesclerks’ reactions, we decided to
test only one IV (the style of dress).
• Because we wanted to determine only whether clothing style
can affect the performance of salespeople, we chose to use only
two levels of the IV (dressy clothing vs. sloppy clothing).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Continuing Research
Problem
• After reviewing relevant research literature, we chose our IV
(style of dress) and our DV (salesclerk response time).
• Because we were conducting a preliminary investigation into
the effects of clothing on salesclerks’ reactions, we decided to
test only one IV (the style of dress).
• Because we wanted to determine only whether clothing style
can affect the performance of salespeople, we chose to use only
two levels of the IV (dressy clothing vs. sloppy clothing).
• If we have a large number of participants available, then we can
use random assignment, which yields independent groups. In
this case, we would use the two-independent-groups design
and analyze the data with a t test for independent groups.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Continuing Research
Problem
•
•
•
Because we wanted to determine only whether clothing style can affect the
performance of salespeople, we chose to use only two levels of the IV (dressy
clothing vs. sloppy clothing).
If we have a large number of participants available, then we can use random
assignment, which yields independent groups. In this case, we would use the
two-independent-groups design and analyze the data with a t test for
independent groups.
If we expect to have a small number of participants and need to exert the
maximum degree of control, we choose to use a design with repeated measures
or matched groups, thus resulting in correlated groups. Therefore, we would
use a two-correlated-groups design for the experiment and analyze the data
with a t test for correlated groups.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education
The Continuing Research
Problem
• If we have a large number of participants available, then we can
use random assignment, which yields independent groups. In
this case, we would use the two-independent-groups design
and analyze the data with a t test for independent groups.
• If we expect to have a small number of participants and need to
exert the maximum degree of control, we choose to use a design
with repeated measures or matched groups, thus resulting in
correlated groups. Therefore, we would use a two-correlatedgroups design for the experiment and analyze the data with a t
test for correlated groups.
• We concluded that salespeople responded more quickly to
customers in dressy clothes than customers dressed in sloppy
clothes.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education