Week 9
Experimental Research Strategy




Bailey Chapter 9
Pp. 217-230
Babbie, Chapter 8
Pp. 228-236
Experimental Research Strategy

Goal: to establish the existence of a cause-andeffect relationship between two variables.

To go beyond correlation

Demonstrate that changes in one variable are
directly responsible for changes in another
variable.
Kinds of research we can do using
experiments:




Specific clear and limited focus
Explanatory rather than descriptive or exploratory
research questions
Possibility of quantitative measurement
Prerequisite for an experimental research:
CLEARLY ARTICULATED CAUSAL
HYPOTHESI(E)S
Prejudice example
Reducing prejudice against African-Americans
 1. give a prejudice test and measure level of
prejudice against homosexuals
 2. have participants to watch a film about
homosexual people with particular achievements
 3. measure prejudice levels again
The Classical Experiment
Three essential components
Independent
Variable(s)
&
Dependent
Variable
Studies the effects of an
independent variable on a
dependent variable.
Operational definitions
are essential.
Pre-test
&
Post-test
Dependent variable is
measured, a stimulus is
presented, and dependent
variable is re-measured.
Changes in the dependent
variable are attributed to
the stimulus.
Experimental
Groups
&
Control
Groups
There are two groups
of comparable
subjects. Only one
group is exposed to the
stimulus.
Differences in the
changes in dependent
variable are attributed
to the stimulus.
Classical Experiment

Independent and dependent variable:

Effect of an independent v. on dependent variable


İndependent variable: exposure to movie on prejudice
Dependent variable : prejudice

Reduced prejudice is an effect of watching the film
Classical Experiment

Pretesting and posttesting

Pretesting occurs first when subjects are measured on
dependent variable
Then subjects are exposed to a stimulus representing an
independent variable
Finally in posttesting they are remeasured on their
dependent variable
Pretesting – giving the prejudice test
Posttest – giving the prejudice test again




Classical Experiment


Treatment (stimulus)
Exposure to the movie on homosexuals and their
achievements
Classical Experiment





Experimental and control groups
Experimental group to which stimulus is administered
Control group which does not receive the experimental
stimulus
Using a control group allows the researcher to detect any
effects of the experiment itself
If the posttest shows that level of prejudice in the control
group goes down as well then decreased level of prejudice
may be result of an extranous factor
Select subjects randomly
from the population
Staging
True Experiments
Pre-test
Randomly assign subjects to groups
Manipulation of independent variable
Manipulation check <
Post-test: Measurement of dependent variable
Comparison of post-test scores of each group
Selecting subjects




Not terribly representative
Probability sampling is seldom used
Use logic of random sampling when we assign
subjects to groups
2 methods:


Randomization
matching
Selecting subjects
Randomization




Randomization is used to assign participants to treatment
groups.
Might be accomplished by numbering all subjects and
selecting numbers by means of random number table
Offers a method for controlling a multitude of
variables simultaneously and does not require specific
attention to each extraneous variable.
Still “chance” can produce a biased outcome in the
long run if use a small sample
Selecting subjects: Matching groups
Group A
Red
Short
21yrs
Group B
matched
Red
Short
21yrs

Matched groups


Blue
tall
23yrs
Green
average
22yrs
Brown
tall
22yrs
matched
matched
matched
Blue
tall
23yrs
Green
average
22yrs
Brown
tall
22yrs

Trying to create
quivalent groups
Also trying to reduce
some of the overall
variability
Eliminating variability
from the variables that
you matched people on
Color
Height
Age
Variations on experimental design



Cross sectional designs (between subjects)
Longitudinal designs (within subjects) Post test only
designs
One-and Two-group experimental designs



Before and after experiment with no control group
Two-group design
Multiple group designs


Two experimental groups with one control group
Factorial design
Variations on experimental design

Between subjects design

Have treatment and control
groups and compare their
scores

Within subjects (repeated
measures) design:

Compare the same group of
participants’ scores before
and after a treatment is
introduced
The Structure of a Between-Subjects Experiment.
 The key element is that separate groups of
participants are used for the different treatment
conditions.


Have treatment and control groups and compare their
scores
Example: one group is assigned to teaching method A,
a separate group to teaching method B and their scores
are compared.
The Structure of a Within-Subjects Design.
The same sample of individuals participate in all of the
treatment conditions. Because each participant is
measured in each treatment, the design is sometimes
called a repeated-measures design. Note: All participants
go through the entire series of treatments but not
necessarily in the same order


Compare the same group of participants’ scores before and after
a treatment is introduced.
Example: Effect of an anger management program
Before and after group with no control group
1.
2.
3.
4.
5.
select subjects
select experimental environment
pretest
Experimental stimulus
Post-test
Two-group designs
Experimental group
1.
2.
3.
4.
5.
Select subjects
Select experimental
environment
Pretest
Experimetal stimuli
Posttest
Control Group
1.
2.
3.
4.
Select subjects
Select experimental
environemt
Pretest
Posttest
Posttestcontrol –pretestcontrol=diffcontrol
Posttestexp-pretestexp=diffexp
Causal effect=Diffexp- Diffcontrol
Multiple Group Designs
Two experimental groups with one control group
 Causal variable differs in value or intensity in the
two experimental groups
 Example : population density and anxiety
Exp. Group 1 ---high density environment
Exp. Grp 2 – low density environment
Control group – no treatment

Multiple Group Designs

Factorial Designs:
FACTORIAL DESIGNS

When an experimenter is interested in the effects of two
or more independent variables, it is usually more efficient
to manipulate these variables in one experiment than to
run a separate experiment for each variable.

Consider a hypothetical experiment on the effects of a
stimulant drug on the ability to solve problems. There
were three levels of drug dosage: 0 mg, 100 mg, and 200
mg. A second variable, type of task, was also
manipulated. There were two types of tasks: a simple
well-learned task (naming colors) and a more complex
task (finding hidden figures in a complex display).
DOSE
0 mg
100 mg
200 mg
Simple Task Complex Task
Question


Alcohol decreases reaction time.
Coffee increases reaction time.

You serve coffee to your guests after a dinner where
you drank a bottle of wine. What is going to happen
to your guests’ reaction time? Is it going to be slower
or faster than normal?

HOW WOULD YOU TEST THIS QUESTION?
Example:
The purpose of the experiment is to examine how different
combinations of alcohol and caffeine affect reaction time in a
simulated emergency driving situation.
Main effect of alcohol
What can go wrong in experiments?
Experimental effects
may be due to
something else
Internal Validity
Problems
Experimental effects
may not be
generalizable
External Validity
Problems
Internal validity - results due to the
independent variable and not other variables
IV or Treatment
Causes
Behavior Change
Causes?
Extraneous/confounding
variables
Extraneous (confounding) variables that may have influenced
results are threats to internal validity
Sources of internal validity







History
Maturation
Testing
Instrumentation
Statistical regression
Selection biases
Experimental mortality
External validity problems

Sample problems



Non-representative samples
Control and experimental groups may not be
comparable
Experimental situation does not generalize to the
real world



Unrealistic lab circumstances
Stimulus acts differently in the real world
Interaction between pretest and stimulus
Advantages of Experiments

Establishing causality




Enables to meausre the dependent variable, introduces
the cause (independent variable) and observe whether
any change occurs on dependent variable
Generally longitidunal
Control
Longitidunal analysis

Study change over time
Disdavnatages of experiments


Artificial environment
Experimenter effect


Lack of control



Double-blind experiments
Placing subjects in a lab will alter the very behavior you want to
measure
Closure:
 The extent to which we can control the relevant variables
Sample size
Quasi-Experimental
Strategies
Quasi-experimental Research Strategy

Sometimes you just can’t perform a fully
controlled experiment



Because of the issue of interest (sex, depression)
Limited resources (not enough subjects, observations
are too costly, etc).
Then: design a research strategy similar to an
experiment but fails to satisfy at least one of the
requirements of the true experiment.
Quasi-experimental Research Strategy

Data are collected from pre-existing groups. Independent
variables are measured and used to create groups or
conditions
• Study naturally occuring events or existing groups



A second variable is measured to obtain a set of scores
within each condition
Extraneous variables are NOT controlled through either
experimental control (keeping variables constant) or
randomization.
Because the IV is NOT directly manipulated and
extraneous variables are NOT controlled, a (real) causal
relationship can NOT be inferred
Quasi-experimental Research Strategy
General types
1) An event occurs that the experimenter doesn’t manipulate
• Attitudes toward turban in private and public
universities
• Gender and reading ability
• Self esteem scores of children of divorced and
undivorced parents
• Time (age) is used as a variable (developmental studie
2) Research on programs that is implemented to achieve
some positive effect on a group of individuals.(effects of
immunization campaign in different regions in Turkey)
QUASI EXPERIMENTATION
one-group-posttest only
----- X
O
post-test only with non-equivalent groups
exp:--X
O’
control:
----O’
one-group pre-test-post-test design
exp:
O
X
prestest-posttest with 2-groups-design
exp:
O
X
control: O
---
O’
O’
O’
Quasi-experimental Research Strategy

Core virtues:




“Natural” look at how variables relate
Less control = less reactivity than experimental designs
Can model very complex phenomena
Core Drawbacks:
Confounds!; unmeasured 3rd variable problem
 Causality; simple correlation may confuse cause & effect.
Dealing with confounds: Use complex measurements or samples to
eliminate alternate hypotheses.


 Trade off internal for external validity
True experiments:
Quasi-experiments:
Participants assigned to groups Use existing group(s)
 Random or matching
 Non-random assignment
 Participants & experimenter
 Participants not blind
Blind to assignment
 Control group may not be possible
Control study procedures
 Create / manipulate IV
 Control procedures & measures
Control often not possible
 No true IV; observe event(s)
 Partial control of procedures &
measures
Emphasize internal validity
 Assess cause & effect (in relatively
Emphasize external validity
 Describe “real” / naturally
occurring events
 Clear to exploratory hypotheses
artificial environment)

Test clear, a priori hypotheses