1
Experiments with More Than
One Variable
Dr. Stefanie Drew
[email protected]
2
Image adapted from graciousindian.com/catalog/index.php?cPath=25 on April 16, 2013
Image adapted from earth911.com/recycling/metal/aluminum-can/ on April 15, 2013
Review
• Simple experiments
• Experimental Example:
Company develops new energy
drink Blue Hen
▫ Allow us to see a difference
across two levels of a
singular IV
▫ Does drink make a significant
difference in energy levels?
▫ Treatment group &
Comparison Group
Pretest
Random
Assignment
Pretest
Posttest
IV
Level
2
Posttest
Going one step further
• What if still want to know if
Blue Hen is effective
• …but also want to know it
works differently for different
genders?
▫ E.g. Maybe men and women
have different metabolisms
that affects the effect of the
drink
• Asking about two IVs:
▫ Effect of drink on energy
▫ Effect of gender on drink
effectiveness
4
Interactions
• Interaction: effect of one IV
differs depending on level of
another IV
▫ In terms of two IVs: is there a
difference in differences
• Experimental Example: the
effect of Blue Hen on depends
on the gender of the
participant
• Types to consider
▫ Crossover Interaction
▫ Spreading interaction
The effect of one
A Difference
Interaction =
= IV depends on
In Differences
level of other IV
Image adapted from bestclipartblog.com/clipart-pics/people-clip-art-4.jpg on April 16, 2013
5
Visualizing Interactions:
Crossover Interactions
• Two IVs
▫ Meal judging (Dinner or Dessert)
▫ Flavor of Food (Savory or Sweet)
• One DV
▫ Liking rating
6
Visualizing Interactions:
Spreading Interactions
• Can be graphed several ways
• Can describe verbally or
mathematically
7
8
How do we test for interactions
• Factorial design: two or more IVs (called
“factors”)
▫ Crossed factorial design: researchers cross
two or more IVs and study possible combinations
▫ Nested factorial design: study with more than
one IV where levels of one variable are nested
under the levels of another higher order IV
NOTE: for this text, we will not be studying nested!
9
With our Experimental Example
Blue
Hen
Energy
Drink
No
Blue
Hen
Energy
Drink
Men
Men w/
Blue
Hen
Women
Women
w/ Blue
Hen
Whole thing is called a 2 x 2 factorial design!
Men w/o
Blue
Hen
Women
w/o Blue
Hen
Four possible conditions
(a.k.a. “cells”)
10
Participant Variables
• Participant variables: variables where levels
are measured (a.k.a. “selected”) not
manipulated
• Age, gender, sex, ethnicity, culture and not true
IVs, but often called that
Image adapted from abramsresearch.tumblr.com/post/7850140625 on April 16, 2013
11
Uses of Factorial Designs
• Testing limits
▫ External Validity
▫ Moderators
• Testing theories
Image adapted from www.sodahead.com/living/do-you-know-why-cats-are-so-attracted-to-eating-mice/question-266823/ on April 16, 2013
12
Testing Limits: External Validity
• Testing if IV in multiple groups = testing if effect
generalizes
▫ If IV affects groups in same way, effect
generalizes
16
14
12
10
8
6
4
2
0
Blue Hen
No Blue Hen
13
Testing Limits: External Validity
• Testing if IV in multiple groups = testing if effect
generalizes
▫ If groups respond differently, effect may not
generalize
16
14
12
10
8
6
4
2
0
Blue Hen
No Blue Hen
14
Testing Limits: Testing for Moderators
• Remember moderators?
IV
DV
Moderating
Variable
• In terms of factorial design…
▫ Moderator: IV that changes
relationship between other IV and
DV
▫ Moderators interactions
“Effect of one IV depends on the
level of another IV”
OR
“Effect of one IV is moderated by
the level of another IV”
15
Testing Theories
• Theories often address how
different variables interact
with one another
▫ To look how variables
combine, can use a factorial
design!
DV:
Amount of drink
consumed
IV1: Beverage
IV2:
Gender
17
Men
Women
Can with
Logo
Can without
Logo
9
8
7
• Experimental Example: What
if you have a theory that the
icon appeals more to men?
16
Factorial design: Check. Now what?
• Analyzing a factorial design with two
independent variables, consider three results
▫ Two Main Effects
▫ One Interaction Effect
17
Main Effects
• Main effects: overall effect
of on IV the DV, averaging
over levels of other IV
• Main effects are simple
differences
DV:
Amount of drink
Consumed
IV1: Beverage
• Marginal means: means for
each level of one IV, averaging
over the levels of another IV
▫ If sample sizes are equal, can do
simple means
▫ If sample sizes not equal, have to
weigh the means
Can with Logo
Can without Logo
Main Effect of IV2
IV2: Gender Men
17
7
12
Main Effect of IV1
13
Women
9
8
7.5
8.5
18
Main Effects & Statistical Significance
• Examine marginal means to
inspect main effects
• Statistics used to determine if
difference in the marginal
means is statistically
significant
▫ If not significant, conclude
that difference is no larger
than what could be expected
by chance
19
Interactions
Social Interaction
• Interaction: effect of one IV
differs depending on level of
other IV
• Interaction is the difference of
differences
Image adapted from waitbutwhy.com/2014/01/the-great-perils-of-social-interaction.html on April 20, 2015
20
Interaction Tables (1)
• Compute two differences for two levels of IV1
DV:
IV1: Size of Can
Amount of drink consumed
Small
IV2:
Price
On Sale
Regular
300
302
The difference is 2
(302 – 300 = 2)
Large
400
302
The difference is 98
(400 – 302 = 98)
The two differences are significantly
different (2 < 98). There is an interaction.
• Test to see if difference between two differences is
statistically significant
21
Interaction Graphs
• Easier to detect in a graph
• Check to see if lines are parallel
▫ If lines are parallel, probably no interaction
▫ If lines are not parallel, probably an interaction
• Confirm with statistics
22
Interaction Tables vs. Graphs
• Interactions Tables
• Interactions in graphs
Difference of differences
23
24
16
Gender Main Effect : No
Logo Main Effect 2: No
Interaction: Yes
14
12
10
8
6
4
2
0
Can with logo
Men
DV:
Amount of drink
Consumed
IV1: Beverage
Can without logo
Women
Can with Logo
Can without Logo
Main Effect of IV2
IV2: Gender Men
14
6
10
Main Effect of IV1
10
Women
6
14
10
10
25
16
Gender Main Effect : Yes
Logo Main Effect 2: No
Interaction: No
14
12
10
8
6
4
2
0
Can with logo
Men
DV:
Amount of drink
Consumed
IV1: Beverage
Can without logo
Women
Can with Logo
Can without Logo
Main Effect of IV2
IV2: Gender Men
14
14
14
Main Effect of IV1
10
10
Women
6
6
6
26
16
Gender Main Effect : No
Logo Main Effect 2: Yes
Interaction: No
14
12
10
8
6
4
2
0
Can with logo
Men
DV:
Amount of drink
Consumed
IV1: Beverage
Can without logo
Women
Can with Logo
Can without Logo
Main Effect of IV2
IV2: Gender Men
14
6
10
Main Effect of IV1
14
6
Women
14
6
10
27
16
Gender Main Effect : Yes
Logo Main Effect 2: Yes
Interaction: Yes
14
12
10
8
6
4
2
0
Can with logo
Men
DV:
Amount of drink
Consumed
IV1: Beverage
Can without logo
Women
Can with Logo
Can without Logo
Main Effect of IV2
IV2: Gender Men
14
14
14
Main Effect of IV1
14
10
Women
14
6
10
28
18
Gender Main Effect : Yes
Logo Main Effect 2: Yes
Interaction: Yes
16
14
12
10
8
6
4
2
0
Can with logo
Men
DV:
Amount of drink
Consumed
IV1: Beverage
Can without logo
Women
Can with Logo
Can without Logo
Main Effect of IV2
IV2: Gender Men
12
4
8
Main Effect of IV1
14
5.5
Women
16
7
11.5
29
7
Gender Main Effect : No
Logo Main Effect 2: No
Interaction: No
6
5
4
3
2
1
0
DV:
Amount of drink
Consumed
IV1: Beverage
Can with logo
Men
Can without logo
Women
Can with Logo
Can without Logo
Main Effect of IV2
IV2: Gender Men
6
6
6
Main Effect of IV1
6
6
Women
6
6
6
30
14
Gender Main Effect : No
Logo Main Effect 2: Yes
Interaction: Yes
12
10
8
6
4
2
0
Can with logo
Men
DV:
Amount of drink
Consumed
IV2: Gender Men
Women
Main Effect of IV1
IV1: Beverage
Can without logo
Women
Can with Logo
Can without Logo
Main Effect of IV2
8
8
8
10
6
12
4
8
31
14
Gender Main Effect : Yes
Logo Main Effect 2: No
Interaction: Yes
12
10
8
6
4
2
0
Can with logo
Men
DV:
Amount of drink
Consumed
IV1: Beverage
Can without logo
Women
Can with Logo
Can without Logo
Main Effect of IV2
IV2: Gender Men
12
8
10
Main Effect of IV1
8
Women
4
8
8
6