Nonequivalent Groups:
Linear Methods
Kolen, M. J., & Brennan, R. L. (2004). Test
equating, scaling, and linking: Methods and
practices (2nd ed.). New York, NY: Springer.
EPSY 8225
1
Equating
Common
Population
Observed
Score
Anchor Test
True Score:
CTT/IRT
Observed
Score
EPSY 8225
True Score:
CTT/IRT
2
Nonequivalent Groups
• Only one test form is administered at a time.
• The different groups cannot be assumed to be
randomly equivalent.
• Each group comes from a different population.
• To allow the linking of the two forms, a set of
common items is included on each form.
• Common items could be internal (scored) or
external (not part of the score on the form).
EPSY 8225
3
Notation
• Y = score on the old form
• X = score on the new form
• V = score on the common item set
EPSY 8225
4
The Synthetic Population
• The design involves two populations.
• But the equating function is defined for a
single population.
• We define a synthetic population, where each
of the independent populations is weighted by
w1 and w2 where
w1 + w2 = 1 and w1 , w2  0
EPSY 8225
5
Linear Equating
• The linear conversion is defined by setting
standardized deviation scores equal
x  ( X ) y  (Y )

( X )
(Y )
 x  ( X ) 
lY ( x)  y  (Y ) 
 (Y )

 ( X ) 
EPSY 8225
6
Linear Equating Function
 x  s ( X ) 
lYS ( x)  y   s (Y )
   s (Y )
 s ( X ) 
Where s indicates the synthetic population
EPSY 8225
7
Synthetic Population Parameters
s(X) = w11(X) + w22(X)
s(Y) = w11(Y) + w22(Y)
 ( X )  w  ( X )  w2 ( X )  w1w2 [1 ( X )   2 ( X )]
2
s
2
1 1
2
2
 (Y )  w  (Y )  w2  (Y )  w1w2 [1 (Y )   2 (Y )]
2
s
2
1 1
2
2
EPSY 8225
8
2
2
Synthetic Population Parameters
• In population 1, Y is not administered
• In population 2, X is not administered
• The following parameters cannot be estimated
directly:
1 (Y ),  (Y ), and  2 ( X ),  ( X )
2
1
2
2
EPSY 8225
9
Parameters Estimated from Data
• Form X administered to population 1
1 ( X ) and  ( X )
2
1
• Form Y administered to population 2
 2 (Y ) and  (Y )
2
2
EPSY 8225
10
Parameters Estimated from Assumptions
• Form X moments in Population 2
 2 ( X ) and  ( X )
2
2
• Form Y moments in Population 1
1 (Y ) and  (Y )
2
1
EPSY 8225
11
Nonequivalent Group Equating
• What sets apart these equating methods is
the set of statistical assumptions used.
• Assumptions must be introduced to estimate
the parameters not observed.
• Different methods employ different
assumptions.
EPSY 8225
12
Tucker Method
• Assumption 1: regression of total scores on
common-item scores
– The regression of X on V is the same linear function
for both populations 1 and 2
– The same assumption is made for the regression of Y
on V
The slope and regression intercept are assumed to be
the same for the observed data with each population
and the unobserved parameters in the other population
EPSY 8225
13
Tucker Method
• Assumption 2: conditional variances of total
scores given common-item scores
– Conditional variance of X given V is the same for
population 1 and 2
– The same assumption is made for the conditional
variance of Y given V
EPSY 8225
14
Tucker Method
• The conditional mean score on the new form
increases linearly with scores on the anchor
– Use a simple formula to estimate the conditional
mean in the synthetic population
• The conditional standard error is the same at
all levels of the anchor score
– Estimate a single value for the conditional SE
• Need: Mean and SD of anchor scores in
synthetic population
EPSY 8225
15
Tucker Method
• Result: the synthetic population means and
variances for X and Y are adjusted to directly
observable quantities.
• The adjustment is a function of the differences
in means and variances for the common items
across the two populations.
• If 1(V) = 2(V) and 12 (V )   22 (V )
the synthetic parameters would equal
observable means and variances.
EPSY 8225
16
Estimating Synthetic Pop Values
s(X) = 1(X) - w2γ1[1(V) - 2(V)]
s(Y) = 2(Y) - w1γ2[1(V) - 2(V)]
Where γ1 and γ2 are the regression slopes from
the regression of X on V for the two populations.
2
2

(
X
)
and

Similarly, we can estimate s
s (Y )
EPSY 8225
17
Weighting Options
• w1 + w2 = 1 and w1 , w2  0
• One could conceive of the synthetic population to be
the new population: w1 = 1 and w2 = 0
• Weights can be proportional to population sizes:
w1 = N1/(N1 + N2) and w1 = N2/(N1 + N2)
• Weights can be equal, combining both populations
equally: w1 + w2 = .5
EPSY 8225
18
Tucker Issues
• Problems in the equating can occur if
– the ability distributions between those who take
the different forms differ a great deal
– When the anchor is not strongly correlated with
the test scores
– The test scores and the anchor scores do not yield
near perfect reliabilities
EPSY 8225
19
Levine Method
• Assumes that X, Y, and V are all measuring the
same thing so that TX and TV as well as TY and TV
are perfectly correlated in both populations.
• Assumptions about true scores are made in
terms of the linear regression of X on V and Y
on V
• Assumptions about the error variances
(measurement error) are made similarly
EPSY 8225
20
Levine
• The method is one that relates observed
scores from form X to the scale of observed
scores on form Y.
• However, the assumptions underlying Levine
method are about the true scores, TX, TY, and
TV. These are related to observed scores as in
CTT, where the error has E[ε] = 0 and ρεT = 0.
EPSY 8225
21
Levine
• The assumptions are that X, Y, and V all
measure the same thing, which implies
• True scores of X and V as well as True scores of
Y and V are perfectly correlated in both
populations 1 and 2.
• ρ1 = (TX, TV) = ρ2(TX, TV) = 1
• ρ1 = (TY, TV) = ρ2(TY, TV) = 1
EPSY 8225
22
Levine True Score Method
• Similar assumptions about true scores are made
in this method
• Instead of equating observed scores, true scores
are equated
 t x   s (TX ) 
lYS (t x )  y   s (TY ) 
   s (TY )
  s (TX ) 
• Although the derivations employ true scores, the
equating is actually done on observed scores.
EPSY 8225
23
Levine True Score Method
In CTT, observed score means = true score means
 s (TY )
t x   s ( X )   s (Y )
lYS (t x )  y 
 s (TX )
And based on previous derivation results:
2
lY (t x )  y  t x  1 ( X )   2 (Y )   2 1 (V )   2 (V )
1
Observed scores are used in place of true scores.
The results do not rely on the synthetic population.
EPSY 8225
24
Levine True-Score Property
• Turns out, using observed scores in Levine’s
true score equating function for the commonitem nonequivalent groups design results in
first-order equity, under the congeneric model.
• For the population with a given true score on Y,
the expected value for the linearly transformed
scores on X equals the expected value of the
scores on Y, for all true scores on Y.
EPSY 8225
25
Nonequivalent Groups:
Equipercentile Methods
EPSY 8225
26
Frequency Estimation
• f (x, v ) is the joint distribution; the probability that
X = x and V = v.
• f (x) is the marginal distribution of scores on X; the
probability of obtaining a score of x on X.
• h( v ) is the marginal distribution of scores on V.
• f (x| v ) is the conditional distribution of scores on
Form X for examinees with a particular score on V.
EPSY 8225
27
The conditional expectation is
f ( x, v )
f ( x v) 
h (v )
It follows that
f ( x, v)  f ( x v)h(v)
EPSY 8225
28
Synthetic Populations
• fs(x) = w1 f1(x) + w2 f2(x)
• gs(y) = w1 g1(y) + w2 g2(y)
• Because form X is not administered to
population 2, f2(x) is not directly estimable
• Because form Y is not administered to
population 1, g1(y) is not directly estimable
EPSY 8225
29
Assumptions
• For both Form X and Form Y, the conditional
distribution of total score given each score,
V = v, is the same in both populations.
f1 ( x v)  f 2 ( x v), for all v
g1 ( y v)  g2 ( y v), for all v
EPSY 8225
30
Estimation
• These assumptions can be used to find
expressions for f2(x) and g1(y) using quantities
for which direct estimates are available.
EPSY 8225
31
Chained Equipercentile Equating
Angoff (1971)
• Form X scores are converted to scores on the
common items using examinees from
population 1
• Scores on the common items are equated to
form Y scores using examinees from
Population 2.
• These conversions are chained together to
produce a conversion of Form X to Form Y
EPSY 8225
32