PS52005C Design and Analysis of Psychological Studies
SPSS Lab 2 – Unplanned (post hoc) comparisons
Outline:
•
•
•
•
•
Introduction
The Worksheet
Design 1 – Two way independent measures (2x3 factorial design)
Design 2 – Two way repeated measures (2x2 factorial design)
Design 3 – Two way mixed (2x4 factorial design)
Introduction
The first lab looked at the omnibus ANOVA tests for the three basic two-way factorial
designs. This lab introduces a number of simple procedures for conducting the follow-up
analyses for looking at the differences between means responsible for any main effects or
interactions uncovered in the omnibus tests.
This lab goes over the steps for the three basic factorial designs, as well as the procedures
for conducting simple effects and pairwise comparisons. This involves the use of t-tests and
one-way ANOVAs.
This lab also gives a step-by-step guide for two useful techniques to enable you to create
new variables and select specific data:
•
•
Split File – to restrict analyses to specific treatment levels
Compute – to create new variables from existing variables
You will not need to enter any data, all the data files you need are provided on learn.gold.
You will find these in the folder marked ‘Lab 2 Documents’.
Each design has its own data file, make sure you open the appropriate data file for each
design. Pay careful attention to these, the data has to be entered in the appropriate format
to be able to run the relevant analysis.
The Worksheet
You need to complete and upload the worksheet related to this lab. This can be found on
learn.gold, in the folder marked ‘Lab 2 Documents’.
1
PS52005C Design and Analysis of Psychological Studies
You need to write a brief paragraph on Design 2, reporting the effects in the appropriate
format, and explaining the results found. An example of the format is given, using Design 1
as a guide to the information you need to give.
You will be given feedback on the worksheet, so it is important that you upload this
worksheet at the end of the lab.
Design 1 – two way independent measures design
This study looks at the effect of feedback on memory. 30 participants are randomly assigned
to one of six cells in the design.
Factor 1 is Feedback condition – half the participants receive feedback, the other half do
not.
Factor 2 is the level of complexity of the memory task (low, medium, high). Equal numbers
are assigned to each task.
The DV is the score (out of 10) on a subsequent memory test.
Open the data file labelled Tutorial_2_Design_1_Two_way_independent_measures.sav, it
will look like this:
2
PS52005C Design and Analysis of Psychological Studies
Rows are always participants –
there should be as many rows as
there are participants in the study.
Dependent variables (DVs) have
only 1 column for between
participants factors.
Additional columns are needed to
specify which condition each
participant belongs to.
Run the two way between subjects ANOVA
Select Analyze
Go down to General Linear Model
Move across and click on Univariate
3
PS52005C Design and Analysis of Psychological Studies
Move the DV into the Dependent
Variable box
Move the IVs into the Fixed
Factor(s) box
Click OK
This will give you the following output table, giving the main effects of task difficulty and
feedback condition, as well as the interaction between the two factors.
Tests of Between-Subjects Effects
Dependent Variable:Test Score
Type III Sum of
Source
Squares
df
Mean Square
F
Sig.
Corrected Model
122.967
a
5
24.593
30.742
.000
Intercept
1020.833
1
1020.833
1276.042
.000
difficul
97.867
2
48.933
61.167
.000
feedcond
14.700
1
14.700
18.375
.000
difficul * feedcond
10.400
2
5.200
6.500
.006
Error
19.200
24
.800
Total
1163.000
30
142.167
29
Corrected Total
a. R Squared = .865 (Adjusted R Squared = .837)
4
PS52005C Design and Analysis of Psychological Studies
The table below shows the main effects that you need to consider when looking at the
results.
The rows show the simple main effect of task difficulty overall for each level of feedback –
the effect of task difficulty overall with feedback, and with no feedback.
The columns show the main effect of feedback overall within each level of task difficulty –
the overall effect of feedback for a low difficulty task, for medium difficulty, and for a hard
task.
Task difficulty
B1
Feedback
5.0
6.6
8.0
6.53
(n = 15)
Family of simple main
effects of task difficult
within each level of the
feedback factor
For two one-way
ANOVAs: (0.05/2 =
0.025)
Simple main effect of
task difficulty within
Feedback condition
B2
No feedback
2.0
5.6
7.8
5.13
(n = 15)
Simple main effect of
task difficulty within no
feedback condition
Column
means (main
effect of
condition)
3.5
6.1
7.9
(n = 10)
(n = 10)
(n = 10)
Grand
mean =
5.43
Simple
effect of
feedback
within High
task
difficulty
Simple
effect of
feedback
within
Medium
task
difficulty
Simple effect
of feedback
within Low
task difficulty
A1
Hard
A2
Medium
A3
Low
Row means
(main
effect of
difficulty)
B*
Feedback
Condition
A*
Family of
simple effects
of feedback
within each
level of the
task difficulty
factor
(0.05/3 =
0.0167)
Following up the simple effects of feedback
In order to look at the effect of feedback under each difficulty level, separate t-tests
(feedback vs no feedback) are needed for each level of task difficulty (high, medium, low).
It is possible to do all three tests at once, by using the Split File option:
5
PS52005C Design and Analysis of Psychological Studies
Click on Data
Select Split File
Select Organize output by groups
Move Task Difficulty to the ‘Groups
Based’ on box (this is the grouping
variable, allowing us to make
comparisons in each level of this
variable)
Click OK.
6
PS52005C Design and Analysis of Psychological Studies
You will now be in the output window, and it will appear as though nothing has happened.
The next step is to run the t-tests.
Click on Analyze
Go down to Compare Means
Click on Independent-Samples T
Test
Move the DV (score) to the Test
Variable(s) box
Move feedback into the Grouping
Variable box
Click on Define Groups
Specify the levels of the IV to be
compared.
Here there are only two levels, 1
and 2.
Click on Continue
This returns you to the previous
screen, click OK
7
PS52005C Design and Analysis of Psychological Studies
The t-test will be carried out three times, one for level of task difficulty.
Note: all tests will be carried out three times, until you turn off the Split File function. You
do this by going to Analyze, Split File, and this time selecting ‘Analyze all cases, do not create
groups.
Testing the simple main effects of task difficulty
We also want to look at the simple main effects of task difficulty in each level of the
feedback condition. As there are three levels of task difficulty, we need to carry out one-way
ANOVAs at each level of the feedback condition.
The first step is to split the file, but this time based on the feedback condition.
Click on Data
Select Split File
8
PS52005C Design and Analysis of Psychological Studies
This time feedback is the grouping
variable
Run the one-way ANOVA:
Select Analyze
Go down to General Linear Model
Move across and click on Univariate
9
PS52005C Design and Analysis of Psychological Studies
The score goes in the DV box
Difficulty goes in the Fixed Factor(s)
box
IMPORTANT: Feedback should NOT
be included, as you have already
dealt with this by splitting the file
You can see from the output that the simple main effects of Task Difficulty are significant both in
the feedback and no feedback conditions. We now need to find out which differences are
significant, by carrying out pairwise comparisons.
The table below shows the comparisons that need to be made (shown by the arrows):
Task difficulty
A1
Hard
A2
Medium
A3
Low
Row means
(main
effect of
difficulty)
B*
Feedback
Condition
B1
Feedback
5.0
6.6
8.0
6.53
(n = 15)
Treat each level of
Feedback as a separate
‘family’ by comparing
your alpha against
(0.025/3 = 0.0083) for 3
simple comparisons
B2
No feedback
2.0
5.6
7.8
5.13
(n = 15)
alpha = (0.025/3 =
0.0083) for 3 simple
comparisons
Column
means (main
effect of
condition)
3.5
6.1
7.9
(n = 10)
(n = 10)
(n = 10)
Grand
mean =
5.43
A*
As you can see, three t-tests are needed at each level of the feedback condition: low vs
medium difficulty, low vs high, and medium vs high.
10
PS52005C Design and Analysis of Psychological Studies
Click on Analyze
Go down to Compare Means
Click on Independent-Samples T
Test
Move the DV (score) into the Test
Variable(s) box
Move difficulty (the IV) into the
Grouping Variable box
Define the two levels to be
compared.
As t-tests only do pair-wise comparisons, you will need to run the t-test three times. The
first time you will compare Groups 1 and 2, the second time Groups 2 and 3, and the third
time Groups 1 and 3.
The output will give two t-tests each time, one for each level of the feedback condition.
11
PS52005C Design and Analysis of Psychological Studies
NOTE:
If the interaction had not been significant but the main effects had, we would still have
calculated the simple effects for the levels of task difficulty to show which comparison was
responsible for this main effect (as there are 3 levels of this IV). However, we would not
have calculated the simple effects for the feedback condition as there are only two levels to
this factor, and therefore this main effect is explained by the omnibus test alone.
Note on Type-1 errors
When unplanned comparisons are carried out, it is necessary to adjust the significance level
to control for the increased likelihood of a Type 1 error. However, full explanation of why
this is necessary is not covered here. Look at your lecture handouts to fully understand the
procedures involved.
12
PS52005C Design and Analysis of Psychological Studies
Design 2 – Two way repeated measures design
This is a hypothetical study examining the relief of acute pain.
Opioids are derivatives of opiates and have obviously been used for pain relief for many
years. More recently, cannabinoids have been developed for the same reason, as they seem
to mimic some of the properties of opioids.
This study involves the use of both opioids and cannabinoids in two different doses to
evaluate their relative efficacy (this study was also looked at in Lab 1).
There are two factors, with two levels in each:
Factor 1 is Treatment – opioid vs cannabinoid
Factor 2 is Dosage – single dose vs double dose
The DV is the score on a perceived level of pain scale – the higher the score, the higher the
level of pain.
Open the data file labelled Tutorial_2_Design_2_Two_way_repeated_measures.sav, it will
look like this:
13
PS52005C Design and Analysis of Psychological Studies
Run the two way repeated measures ANOVA
Click on Analyze
Move down to General Linear
Model
Select Repeated Measures
Name each factor (IV) and give the
number of levels
Click on Add after naming each one
There should be two factors (two
IVs) – the treatment (2 levels) and
the dosage (2 levels)
Click on Define
14
PS52005C Design and Analysis of Psychological Studies
Move the conditions to the WithinSubjects Variables window
Make sure you enter these in the
correct order (here they go over in
the order they are listed)
Once all options are selected, click
OK
Finding the right order for entering variables:
Additional options:
Look at the factor names in the brackets above the WithinSubjects Variables box, this is the order to enter the
variables.
Click on Options for the mean
scores (see below)
After each _?_ you will see two numbers in brackets, this
tells you which variable is needed – so the first is
treatment 1 dose 1 (opioid single dose), the next is
treatment 1 dose 2 (opioid double dose), etc
15
Click on Plots for a plot of the
interaction (see below)
PS52005C Design and Analysis of Psychological Studies
Click on Descriptive statistics – this
will give you the mean score for
each condition
Click Continue
This will return you to the previous
window
Move ‘dose’ to the Horizontal Axis
box
Move ‘treatment’ to the Separate
Lines Box
Click Add
Click Continue
16
PS52005C Design and Analysis of Psychological Studies
This shows the interaction of the
two factors.
This plot would not be included in
your Results section, as you can see
the scale makes it misleading.
However, it does give you an
overview of how the variables
interact
Following up on the significant interaction
As the interaction was significant, you need to carry out further tests to find which
differences are significant.
The table below shows the comparisons that you need to make:
Dosage
B*
Single
Treatment
A*
Double
Row
means
(main
effect of
dose)
Opioid
63.0
56.8
?
Cannabinoid
54.4
54.8
?
Column means (main
effect of Treatment)
?
?
Family of simple effects
of Treatment within
each level of the Dosage
factor
For two paired samples
t-tests: (0.05/2 = 0.025)
Simple effect
of Treatment
within Single
dose level
Simple effect
of Treatment
within Double
dose level
17
Family of simple effects of
Dosage within each level of
the Treatment factor
For two paired samples ttests: (0.05/2 = 0.025)
Simple effect of Dosage
within the Opioid level
Simple effect of Dosage
within the Cannabinoid
level
PS52005C Design and Analysis of Psychological Studies
As this is a repeated measures design, it should be followed by repeated measures t-tests.
Click on Analyze
Go down to Compare Means
Click on Paired-Samples T Test
You need to select 4 pairs – these
are indicated in the table shown
previously.
Click on the variables in the left box
and move them over to the Paired
Variables box
Click OK
Calculating row and column means with repeated measures designs
Look again at the table on the previous page. You will note that this table does not include
the row and column means (the means for the main effects). If you look at the descriptive
statistics table from the repeated measures ANOVA you will see that this does not give
these means either.
These figures (as well as standard deviations are needed in order to correctly report the
main effect results in APA format.
18
PS52005C Design and Analysis of Psychological Studies
Here you need to calculate the means and standard deviations for the following:
•
•
•
•
Overall opioid
Overall cannabinoid
Overall single dose
Overall double dose
You will need to carry out the following steps to find these. This shows you how to do this
for the overall opioid variable, you simply repeat this, changing the variable names, for the
rest.
You should then have four new variables in the Data View.
Click on Transform
Select Compute Variable
19
PS52005C Design and Analysis of Psychological Studies
Name the new variable in the Target
Variable box
In the Numeric Expression box, you need
to calculate the mean of the two opioid
conditions. This should therefore read:
It is important to make this meaningful –
the first variable you want is the total
opioid score, so you could call this
totopioid
(opioid1 + opioid2)/2
It is a good idea to move the variables
across from the left and add the
appropriate brackets etc – in that way,
you will be sure that the variable name is
spelt correctly, to ensure SPSS
recognises it.
Repeat this to get the remaining scores (overall cannabinoid, overall single dose, and overall
double dose).
20
PS52005C Design and Analysis of Psychological Studies
You can now calculate the means and standard deviations for these new variables:
Click on Analyze
Move down to Descriptive Statistics
Click on Descriptives
Move the 4 variables you have
created to the Variable(s) box
Click on Options
Make sure Mean and Standard
Deviation are selected
Click on Continue to return to the
previous screen
Click OK
21
PS52005C Design and Analysis of Psychological Studies
Design 3 – Two way mixed ANOVA
This study evaluates four exam tests (1, 2, 3 and 4) to ensure they are of an equivalent
standard. Participants (N=8) take part in each test (repeated measures). Gender was also
taken into account, with 4 males and 4 females (independent measures).
This is an extended version of the study used in Lab 1.
Open the data file labelled ‘Tutorial_2_Design_3_Two_way_mixed.sav’, it will look like this:
Run the two-way mixed ANOVA
Click on Analyze
Move down to General Linear
Model
Click on Repeated Measures
22
PS52005C Design and Analysis of Psychological Studies
Name the within subjects factor
(test) and given the number of levels
(in this case 4)
Click Add
Click Define
Move the four within subjects
variables to the Within-Subjects
Variables box
Move the between subjects factor
(gender) to the Between-Subjects
factor(s) box
Select options (see below)
Once complete, click OK
As with the previous design, you can
get descriptive statistics by clicking
on Options
A plot of the interaction is available
by clicking on Plots
23
PS52005C Design and Analysis of Psychological Studies
Simple main effects
We want to look at the simple main effects for males and females, so we will need to carry
out one-way repeated measures ANOVAs for each.
Click on Data
Click on Split File
24
PS52005C Design and Analysis of Psychological Studies
Select Organize output by groups
Move Gender to the Groups Based
on box
We can now run one-way ANOVAs on the four levels of test:
Click on Analyze
Go down to General Linear Model
Click on Repeated Measures
25
PS52005C Design and Analysis of Psychological Studies
Move the test variables over to the
Within-Subjects Variables box
Make sure you do NOT put gender
in the Between-Subjects Factor(s)
box
Click OK
Note: we are comparing our results against a Bonferroni corrected alpha of .025 (as it is .05
/ 2).
You will see from the output that the simple main effect of Test is significant for females,
but not for males. This means that we now need to carry out pairwise comparisons for the
female group only, to find out which differences are significant.
Keep the Split File turned on, so that the analyses are separate for males and females.
Although you will get output for males as well, this can be ignored.
Click on Analyze
Go down to Compare Means
Click on Paired-Samples T Test
26
PS52005C Design and Analysis of Psychological Studies
Select all possible pairwise
combinations of the test variable:
As there are 6 tests, you need a
Bonferroni corrected significance
level:
Test 1 / Test 2
Test 1 / Test 3
Test 1 / Test 4
etc
.025 / 6 = .0042
There should be 6 pairs in total
Click on OK
Simple effects of gender
We also want to compare the simple effects of gender at each test separately.
Make sure the Split File is turned off first – go to Data, Split File, and select ‘Analyze all
cases, do not create groups’.
27
PS52005C Design and Analysis of Psychological Studies
Click on Analyze
Go down to Compare Means
Click on Independent-Samples T
Test
Move the four test variables over to
the Test Variable(s) box
Move gender to the Grouping
Variable box
Click on Define Groups
Put groups as 1 and 2
Click Continue to return to previous
screen
Click OK
The output will give you four t-tests. This means you need to compare these to a Bonferroni
corrected alpha – .05 / 4 = .0125
28