Organizing Data Files in SPSS
Z All data for one subject entered on the same line
Š Identification data
Š Between-subjects manipulations: variable to identify group
level for that subject
Analysis of Data
Š Measures for Within-subjects manipulations: each level of a
within subject variable appears as a new variable in the data file
Claudia J. Stanny
PSY 6217 – Research Design
ID
SEX
AGE
HAND
WD1
WD2
WD3
PIC1
PIC2
PIC3
RTW1
RTW2
RTW3
RTP1
RTP2
RTP3
1
1
2
2
12
13
11
14
16
15
617
725
710
550
513
490
2
1
1
2
11
15
10
16
20
19
982
880
798
682
773
825
3
2
1
1
4
2
2
1
5
1
2
1
6
2
1
2
Descriptive Statistics
Measures of Association
Z Biased and Unbiased Estimates of Population
parameters
Z Measures of Central Tendency
Z Pearson product-moment correlation (r)
Z Point-biserial correlation
Š Equivalent to a between-subjects t-test
Š Operational definitions of a “typical” score
Š Mean, median, mode
Z Spearman rank order correlation (rs )
Z Phi coefficient
Z Measures of Variability
Š Range, interquartile range
Š Variance
Š Standard deviation
Interpreting Correlations
8
6
4
2
0
0
1.5
2
2.5
3
HS GPA
3.5
4
1
2 3 4
5 6 7
8 9 10 11 12
Patient's Belief in Doctor's Competence
12
Crop Yield (bushels)
100
80
60
40
20
0
0
25
50
75
Temperature in Pensacola
100
10
30
30
25
25
Scores on a Test
2.5
10
Scores on a Test
3.5
1.5
ATM transactions
Effects of Outliers
12
D ay s in T h e rap y
College GPA
4.5
20
15
10
20
15
10
8
6
5
4
2
5
0
0
0
5
10
Inches of Rainfall per Month
15
0
0
2
4
Hours Studied
6
0
2
4
6
Hours Studied
1
Effects of Outliers
Restricted Range
100
75
75
50
50
25
25
0
25
50
75
100
70
60
60
50
50
40
30
20
10
25
50
75
100
0
5
125
65
75
65
75
10
10
9
9
8
8
7
7
6
5
4
3
Š Score (DV) = µ + random error
µ1 = µ2
Z Assume that the experimental conditions
produce systematic changes in the scores of
participants on the dependent measure
Š H0 :
Š Score (DV) = µ + treatment effect + random error
15
5
4
3
2
1
0
0
20
40
60
80
100
Age at Death of Spouse
Inferential Statistics
Z Logic of Inferential Statistical Tests
Z Assume that all participants are selected from
the same population
10
6
1
0
Height (inches)
5
Age in years
2
55
0
Data From Different Groups
Women
Men
Height (inches)
Š H1 :
15
Interest in Remarriage
290
265
240
215
190
165
140
115
90
10
Age in years
Interest in Remarriage
Weight (lbs)
Weight (lbs)
55
30
10
Combining Data From Different Groups
290
265
240
215
190
165
140
115
90
40
20
0
0
125
80
70
0
0
0
80
Height (in)
100
Height (in)
125
125
Widowers
Widows
0
20
40
60
80
100
Age at Death of Spouse
Types of Statistical Errors
Z Type I Error
Š Probability = alpha
Š Rejection of H0 when H0
is true
Z Type II Error
Š Probability = beta
Š Failure to reject H0 when
H1 is true
Š Failure to detect a true
treatment effect
µ1 ≠ µ2
2
Parametric Statistics
Z Random selection and/or random assignment of
scores from the population of interest
Z Sampling distribution is normal
Š Central Limit Theorem: sampling distribution will
become normal as sample size increases, even when
raw data are not distributed normally
Z Homogeneity of variances within groups
Š Problems can arise when observations in one
condition are more variable than observations in other
conditions
Z Testing these assumptions statistically
Partitioning Variance in ANOVA
Total Variability
Systematic
Variability
Error Variability
(Within Groups)
(Between Groups)
Interpreting a Significant F Ratio
Z Need to evaluate pair-wise comparisons of means
following a significant F ratio
Z Planned comparisons
Š A priori comparisons
Š Comparisons based on specific predictions hypothesized
before data collection
Z Unplanned comparisons
Š Post hoc tests
Š Need to control Experiment-wide Error Rate (family-
wise error rate)
Š Probability of at least one Type I Error increases as the
number of comparisons increases
Analysis of Variance
Z One-way designs
Š One independent variable (factor)
Š two levels of a factor – a t-test will give equivalent
results
Š three or more levels of a factor requires use of
ANOVA
Z Partitioning Variance
Š Between-groups variability
Š Within-groups variability
Creating an F Ratio for ANOVA
Z SStotal
= SStreatment + SSerror
Z Mean Square = SS / df
Z Expected MSBG = Var TX + Var error
Z Expected MS WG = Var error
F Ratio = MS BG / MS WG
= (Var TX + Var error)/Varerror
Z Value of F ratio when H0 is true?
Z Value of F ratio when H0 is false?
Planned Comparisons
Z Contrasts should be orthogonal
Š Outcome of each contrast should be independent
Š Knowing the result for one comparison does not
constrain the decision for other comparisons
Z Example: Planned comparisons for a three
group design: Drug A, Drug B, Control
Š Contrast 1: Average of Drug versus No Drug
• Combines mean for drug A and drug B
Š Contrast 2: Drug A versus Drug B
3
Orthogonal Contrasts Produce Independent
Outcomes
Independence of Contrast Outcomes (continued)
Z Assume that the comparison of the Drugs versus
Control produces no difference
Mean for Drugs = 5 Mean for Control = 5
Z 3 outcomes possible for the comparison of Drug A
with Drug B
Z Assume that the comparison of the Drugs versus
Control produces a significant difference
Mean for Drugs = 4 Mean for Control = 8
Z 3 outcomes possible for the comparison of Drug
A with Drug B
Š No Difference
Š Drug A > Drug B
Š Drug A < Drug B
Drug A = 5
Drug A = 7
Drug A = 2
Drug B = 5
Drug B = 3
Drug B = 8
Š No Difference
Š Drug A > Drug B
Š Drug A < Drug B
Drug A = 4
Drug A = 6
Drug A = 1
Drug B = 4
Drug B = 2
Drug B = 7
Z Knowing the decision about the first contrast does not
Z Knowing the decision about the first contrast does not
provide information about the outcome of the second
contrast
provide information about the outcome of the second
contrast
Post Hoc Tests
Z Scheffe
Š Corrects for all possible comparisons, whether or not
these are examined
Z Bonferroni procedure
Š Divide alpha by the number of comparisons to be
made
Š Use the critical value for the smaller alpha value for
all tests
Z Dunnett
Š Specifically designed to compare each experimental
group with a single control group
ANOVA – Two Factor Designs
Post Hoc Tests
ZTukey HSD (default post hoc in SPSS)
Š Controls EER over a set of pair-wise comparisons
(adjusts critical value for significance)
Š Specialized procedures available for calculation for
means based on unequal sample sizes
Z Newman-Keuls
Š Less conservative than Tukey HSD
Š Means in a comparison set are rank ordered
Š Critical value for a pair-wise comparison varies with the
number of “steps” between means in the ordered set
Z Duncan
Partitioning Variance in the BetweenSubjects Design
Z Additional factors require partitioning variance
in different ways
Z Nature of the partitioning will change
depending on the type of design
Š Both factors manipulated between-subjects
Š Both factors manipulated within-subjects
Š Mixed designs
Z Logic of the test remains the same: Isolate the
systematic effect in the numerator of the F ratio
4
Partitioning Variance in the WithinSubjects Design
Partitioning in a Mixed Design
Fixed Effects & Random Effects
Interaction Effects
Z Fixed Effects
Š All levels of a factor are in the design
• men & women
Š All relevant levels of a factor are in the design
• zero stress, moderate stress, high stress
Z Random Effects
Š Levels of a factor represent a sample of the possible
levels in the design
• Individual subjects as levels of a factor
• Unique stimulus materials (e.g., photos, sentences, etc. used to
create a manipulation)
Z Additive Model (no interaction)
Score = µ + TXa + TXb + error
Z Non-Additive Model (interaction)
Score = µ + TXa + TXb + TXab + error
Z Meaning of interactions: effect of a treatment
varies depending on conditions created by
manipulation of a second variable
Š Analysis problems when 2 or more factors are random
effects
2 x 3 Factorial with No Interaction
2 x 2 Factorial with Significant Interaction
14
35
Men
Women
30
10
M ean Recall
M ean W ords Recalled
12
8
6
4
2
25
sad at test
happy at test
20
15
10
5
0
0
sad
neutral
Emotional Value of Words
happy
sad
happy
Mood at Study
5
Other Forms of Interactions
25
Factors that Affect Statistical Power
Z Selection of alpha level
20
20
15
15
Š Comparing alpha = .05 to alpha = .01
Š Comparing one tailed tests to two-tailed tests
10
10
5
5
0
0
1
2
3
4
5
6
7
8
9
10
60
50
40
30
20
10
0
1
2
1
2
35
30
25
20
15
10
5
0
Z Size of the treatment effect produced
Z Variability of the sampling distribution for the
statistical test
Š Effects of changes in sample size
Š Effects of changes in control of extraneous variables
1
2
Power Changes with Size of Effect Created
Power Changes with Variability in the
Sampling Distribution
Measuring Effect Sizes (Abelson)
Interpreting Statistical Significance
Z Raw effect size
Š Differences created measured in the same units of
measurement used for the dependent measure
Š Difference in IQ, number of words recalled, etc.
Š Difficulty of knowing what counts as a “large” effect
Z Standardized effect size
Š Effect sizes converted to z-scores
Š Creates equivalent scales for comparing different
dependent measures
Š Useful for meta-analysis
Š Cohen’s d, variance explained (r 2), omega squared
Z Large sample sizes can provide powerful tests
in which small numeric differences are
statistically reliable
Z How should these be interpreted?
Z Numerically small effects can be impressive
Š Created with minimal manipulation of the IV
Š Created when the DV is difficult to change at all
Š Effects of any size have an important theoretical
interpretation
6
Statistical Significance vs Practical Significance
Z What are the practical implications of an
effect?
Š An educational program reliably produces a 2 point
increase in the IQ of 10-year-old children
Š A medical intervention reduces the risk of heart
attack from 2 per 10,000 (p(heart attack) = .0002) by
half (to p(heart attack) = .0002)
• In a population of 10 million people, this will
translate into a change from 1,000 heart attacks to
500 heart attacks
Non-Linear Data Transformations
Z Square Root transformation
Š Used when larger means correlate with larger variances
(common with count data)
Š Makes variances more homogeneous
Š Reduces skew of distributions
Z Arcsine transformation
Š Used when data consist of proportions (e.g., percent
correct data)
Data Transformations
Z Linear transformations
Š Add a constant value
Š Shifts location of distribution without changing the
variance of the distribution
Š Can make data analysis easier (1K vs 1000)
Z Other transformations change both the mean
and the variance of the distribution
Š Transforming data can produce data that meet the
distribution assumptions for statistical tests
Non-Linear Data Transformations
Z Log transformation
Š Reduces skew in distributions
Š Common with sensory data – gives manageable ranges for
plotting results
Z Reciprocal transformation
Š X = 1/X
Š Reciprocal data are less variable than raw data
Š Transforms alter the interpretation of the measure
• Reciprocal of a time measure is a speed measure
Š New measure is sometimes easier to interpret
• GSR gets smaller as stress increases
• EDR (1/GSR) increases with increased stress
Non-Parametric Tests
Z Do not make assumptions about the
underlying distributions to compute
probabilities
Z Useful when data do not meet assumptions for
normality, etc.
Z Can be more powerful tests than parametric
tests when assumptions have been violated
Z Non-parametric analogs exists for nearly every
parametric test
7