Regression and Test Bias
PSY 395
Outline
•
•
•
•
Regression Example
Errors in Prediction
Group Differences
Test Bias
Regression Example
Regression Example
Cavanaugh, M. A., Boswell, W. R., Roehling, M. V., & Boudreau, J. W.
(2000) Self-reported stress among U.S. managers. Journal of Applied
Psychology, 85, 65-74
Does job stress predict job
satisfaction?
A 5-item “hindrance-related job
stress” measure is correlated with
an “overall job satisfaction”
measure
x = job stress
y = overall job sat
1
Regression Example
Regression Example (cont’)
The authors report that
rxy = -.52 (lower stress, higher sat)
x: Job Stress
Mean = 2.71, SD = .76
y: Job Satisfaction
Mean = 0.00, SD = 2.60
Job Stress Predicting Job Satisfaction
r = -.52, N = 150
8
job satisfaction
6
4
2
0
-2
-4
-6
0
1
2
3
4
5
jo b stress
Regression Example
Regression Example (cont’)
The prediction equation is
y’ = bx + c
Have x and y data;
y’ is the value predicted from the reg.
line
Need to calculate:
b (slope) and
c (intercept)
2
Regression Example
Calculating b (slope)
rxy = -.52 (lower stress, higher sat)
x: Job Stress - Mean = 2.71, SD = .76
y: Job Satisfaction - Mean = 0.00, SD = 2.60
b=
scrit
2.60
× rxy =
× −.52 = −1.78
s pred
0.76
Every 1-point increase in job stress
translates into a 1.78-point predicted
decrease in job satisfaction
Regression Example
Calculating c (intercept)
Well, now we know b = -1.78, and
c = M crit − b × M pred
so
c = 0.00 − ( −1.78) × 2.71 = 4.82
Note that this intercept value makes sense:
“If job stress were zero (well below the
mean), then satisfaction would be
predicted to be 4.82 (well above the
mean)”
Regression Example
Prediction Equation
The equation for predicting job satisfaction
from job stress is
b
c
y’ = -1.78x + 4.82
As x increases y’ decreases
because b is negative
low stress score Æ high predicted satisfaction
high stress score Æ low predicted satisfaction
3
Job Stress Predicting Job Satisfaction
r = -.52, N = 150
8
c = 4.82
job satisfaction
6
b = -1.87
4
Increasing x by one
job stress unit
means decreasing
y’ by 1.87
satisfaction units
2
0
-2
-4
-6
0
1
2
3
4
5
jo b stress
Errors in Prediction
Errors in Prediction
• Prediction always imperfect (whenever
criterion-related validity isn’t -1.0 or 1.0)
• Example – Say we know Jane has
A job stress (x) score of 1.2
A job satisfaction (y) score of 3.6
Her predicted score (y’) is:
y’ = bx + c
y’ = -1.78(1.2) + 4.82 = 2.7
Job Stress Predicting Job Satisfaction
r = -.52, N = 150
8
job satisfaction
6
…overprediction
or underprediction?
actual sat
= 3.6
4
2
predicted sat
= 2.7
0
-2
-4
-6
0
1
stress score
= 1.2
2
3
4
5
jo b stress
4
Errors in Prediction
Errors in Prediction
• Prediction and reality aren’t exactly
the same. The error in predicting
her score is
Error = Actual – Predicted
3.6 – 2.7 = .9
• In other words, we underpredicted
her job satisfaction score by .9
Errors in Prediction
Errors in Prediction (cont’)
What if Jane’s job satisfaction score
was 2.0 instead?
We would have overpredicted her
actual job satisfaction score by .7
(because 2.0 – 2.7 = -.7).
Job Stress Predicting Job Satisfaction
r = -.52, N = 150
8
…overprediction
or underprediction?
job satisfaction
6
predicted sat
= 2.7
4
2
actual sat
= 2.0
0
-2
-4
-6
0
1
stress score
= 1.2
2
3
4
5
jo b stress
5
Group Differences
Major group differences
• Intelligence and Achievement Tests
– Mean differences in racial group means
•
•
•
•
Asians score (@105)
Caucasians (@100)
Hispanics (@ 97)
African Americans (@ 94)
– The differences are real – the causes are hotly
debated and not well understood.
– These are group differences….cannot use these to
draw interferences about the individuals in the group.
Group Differences
Major group differences
• Gender differences in intelligence?
– Means are nearly identical for all abilities
– Male distributions are generally more
variable…more men in the extremes in both ends
Group Differences
Major group differences
• Interest differences across gender?
– No difference on Data versus Ideas
– Big difference on People versus Things!
• Males prefer things and females prefer people
– No clear racial differences
– No clear age differences…fairly stable across
the lifespan
6
Group Differences
Major group differences
• Personality Differences
– No strong racial or gender differences in
personality
– Females scores slightly higher on
Conscientiousness
– Males score slightly lower on Neuroticism
Test Bias
Test Bias
• Test bias (unlike test fairness) is a statistical
concept that occurs when a test does not
function equally well across racial or gender
groups.
• The issue of test bias began with the 1964 Civil
Rights Act.
– Minorities were not being selected for employment
positions at the same rate as majorities because the
minority test scores tended to be ½ to 1 SD below the
majority mean. This led to the concept of ADVERSE
IMPACT.
Test Bias
Test Bias
• However, mean differences across groups
in test performance is only bias if those
mean differences aren’t related
performance.
• 2 methods for evaluating test bias
– Differential Validity
• Equality of correlations across groups
– Differential Prediction
• Equality of regression-line slopes across groups
7
Test Bias
Moderator Effects in Regression
• Sometimes you’ll find that instead of
one overall regression line, you get
better prediction if you divide a larger
group into smaller subgroups based on
a moderator variable – such as age,
race, gender – and do regression within
each subgroup.
subgroup regression lines
Y
Group 1
Group 2
X
overall regression line
Goldilocks and the 3 Regression Scenarios:
Scenario 1
Y
Group 1
Group 2
X
INTERCEPT BIAS
The overall regression line underpredicts for
Group 1…
8
Goldilocks and the 3 Regression Scenarios:
Scenario 1
Y
Group 1
Group 2
X
INTERCEPT BIAS
…and overpredicts for Group 2
Goldilocks and the 3 Regression Scenarios:
Scenario 2
Y
Group 1
Group 2
X
SLOPE BIAS – found less often
The overall regression line doesn’t represent
either group well
Goldilocks and the 3 Regression Scenarios:
Scenario 2
Y
Group 1
X
SLOPE BIAS – found less often
The overall regression line doesn’t represent
either group well (e.g., under- and overpredict
for group 1)
9
Goldilocks and the 3 Regression Scenarios:
Scenario 2
Y
X
SLOPE BIAS – found less often
The overall regression line doesn’t represent
either group well (e.g., under- and overpredict
for group 2)
Goldilocks and the 3 Regression Scenarios:
Scenario 3
Y
Group 1
Group 2
X
JUST RIGHT
The overall regression line works well for both
Group 1 and Group 2; predicts similarly
regardless of group membership
Test Bias
Same Slopes? Same Intercepts?
Moderated Multiple Regression (MMR)
• To get accurate estimates of slope
and intercept differences, you need
• Large sample sizes for both groups
• Reliable measures for both groups
• A construct-valid test – measures
the same construct in both groups
10