Published in: Statistische Hefte, 22 (1981), pp. 94-121.
Seiichi
Kawasaki/Klaus
F.
Zimmermann
M e a s u r i n g R e l a t i o n s h i p s in the L o g - L i n e a r P r o b a b i l i t y M o d e l
by Some C o m p a c t M e a s u r e s of A s s o c i a t i o n
I. I n t r o d u c t i o n
I)
Recent d e v e l o p m e n t s of l o g - l i n e a r p r o b a b i l i t y models have
p r o v i d e d m a n y r e s e a r c h e r s p o w e r f u l tools for i n v e s t i g a t i n g
r e l a t i o n s h i p s a m o n g q u a l i t a t i v e v a r i a b l e s . The l o g - l i n e a r
model has several a t t r a c t i v e p r o p e r t i e s . For one thing a
complex multivariate relationship among categorical variables
can be d e c o m p o s e d into s e v e r a l groups of i n t e r p r e t a b l e e f f e c t
p a r a m e t e r s . And we can e a s i l y f o r m u l a t e m a n y a l t e r n a t i v e
m o d e l s c o r r e s p o n d i n g tO v a r i o u s h y p o t h e s e s such as i n d e p e n dence or e q u i p r o b a b i l i t y , e s t i m a t e such models, for instance,
by the m a x i m u m - l i k e l i h o o d method, and p e r f o r m the s t a t i s t i c a l
tests of the a l t e r n a t i v e h y p o t h e s e s .
In this p a p e r we are c o n c e r n e d w i t h the f o l l o w i n g d i f f i c u l t y
in a p p l y i n g l o g - l i n e a r models. For i n s t a n c e c o n s i d e r the bivariate interaction parameters between a four-category
v a r i a b l e and a f i v e - c a t e g o r y variable. As the result of
e s t i m a t i o n we shall have t w e n t y p a r a m e t e r e s t i m a t e s and a
t w e n t y - b y - t w e n t y c o v a r i a n c e matrix. T h e s e p a r a m e t e r s give us
d e t a i l e d c a t e g o r y - b y - c a t e g o r y i n f o r m a t i o n on the b i v a r i a t e
r e l a t i o n s h i p . However, the i n f o r m a t i o n is s c a t t e r e d o v e r so
m a n y p a r a m e t e r s that it is d i f f i c u l t to get m o r e d i r e c t inf o r m a t i o n b e t w e e n the two v a r i a b l e s such as w h e t h e r or not
the two v a r i a b l e s are g e n e r a l l y p o s i t i v e l y (negatively) associated. The p u r p o s e of this p a p e r is to o v e r c o m e this difficulty t h r o u g h a s p e c i a l a p p l i c a t i o n of a s s o c i a t i o n m e a s u r e s
to the l o g - l i n e a r model. 2)
In s e c t i o n 2, in o r d e r to p r o v i d e a t h e o r e t i c a l framework, we
d i s c u s s some basic p r o p e r t i e s of the l o g - l i n e a r model, inc l u d i n g e s t i m a t i o n and s t a t i s t i c a l tests. S e c t i o n 3 p r e s e n t s
a l o g - l i n e a r a n a l y s i s of a set of actual data, in w h i c h the
d i f f i c u l t y s t a t e d above is i l l u s t r a t e d . In s e c t i o n 4 and 5 we
d e v e l o p a p r o c e d u r e to e v a l u a t e c o m p a c t m e a s u r e s of a s s o c i a t i o n
as a s o l u t i o n to the a b o v e problem. We first d e c o m p o s e the
p r o b a b i l i t y into several c o m p o n e n t s , and then we apply some
a s s o c i a t i o n m e a s u r e s , w h i c h are w e l l f o u n d e d in the l i t e r a t u r e
on the c o n t i n g e n c y table, to these c o m p o n e n t s . An a s y m p t o t i c
s a m p l i n g theory for these m e a s u r e s is also developed. In
s e c t i o n 6 we apply this m e t h o d to the e x a m p l e d i s c u s s e d in
s e c t i o n 3.
82
2.
Basic
Properties
of
t h e Loc{-Linear M o d e l
In this sectlon, we c o n s i d e r the b a s i c p r o p e r t i e s of loql i n e a r m o d e l s as the f o u n d a t i o n of our study to be d e v e l o p e d
below. For the d e t a i l e d study of g e n e r a l l o g - l i n e a r m o d e l s
see, for e x a m p l e , [ B i s h o p et al., 19751 , ] Bock, 19751 , [ Goodman, 19701 , [ Haberman, 1974a] , and [ N e r l o v e and Press, ]976] .
S u p p o s e that we have one d i c h o t o m o u s r a n d o m v a r i a b l e A and
one t r i c h o t o m o u s random v a r i a b l e B. We d e f i n e i n d e x e s of A
and B by i A and i B such that
iA = I, 2
i B = I, 2, 3.
We also i n t r o d u c e an index i w h i c h is g e n e r a t e d
the pairs (i A, i B) l e x i c o g r a p h i c a l l y
such that
by o r d e r i n g
i = I, 2 . . . . , 6.
A c o n t i n g e n c y table of A and B is a set of f r e q u e n c i e s n of
the cells (i A, i B) or (i). An e l e m e n t of ~ may be e x p r e s s e d
either
two-dimensionally
In a g e n e r a l case
c o n t i n g e n c y table
for
of
by niAiB
or o n e - d i m e n s i o n a l l y
by n i-
where there are q c a t e g o r i c a l v a r i a b l e s , a
c o n s i s t s of n i l i 2 . . . i q w h e r e i i = I, 2...,I i
j = I, 2 , . . . , q
and I. is the total n u m b e r of c a t e g o r i e s
3
variable. The index i is g i v e n by
the j-th
i =
(ii-I)I213...I q + (i2-1)I314...Iq+...+(iq_1-1)Iq+i
Therefore,
n.
=
q.
n.
l
lli2...i q
for
i = I, 2 , . . . , Q
where
Q = lli~I I i.
Now we a s s u m e that the c o n t i n g e n c y
ties of A and B is f a c t o r e d in the
table of joint
f o l l o w i n g way:
log P(iA=I , iB=1)
= ~ + ~A(1)
+ eB(1)
+ BAB(I,I)
log P(iA=1 , iB=2)
= ~ + CA(1)
+ eB(2)
+ ~AB(I,2)
log
P(iA=I , iB=3)
= ~ + ~A(1)
+ eB(3)
+ 8AB(1,3)
log P ( i A = 2 , iB=1)
= ~ + eA(2)
+ ~B(1)
+ 8AB(2,1)
probabili-
83
log P(iA=2,
iB=2)
= H + aA(2)
+ eB (2) + BAB(2, 2)
log P(iA=2,
iB=3)
= U + eA(2)
+ ~B (3) + ~AB(2, 3)
(I)
F. F~ P
= I and O < PiAiB < I for all i A and i B.
i A i B iAiB
where
Or in vector notation,
U
log PAB =
1 1 0 1 0 0 1 0 0 0 0 0 -
eA (I)
1 1 0 0 1 0 0 1 0 0 0 0
CA(2)
1 1 0 0 0 1 0 0 1 0 0 0
~B(1)
1 0 1 1 0 0 0 0 0 1 0 0
eB(2)
1 0 1 0 1 0 0 0 0 0 1 0
eB(3)
1 0 1 0 0 1 0 0 0 0 0 1
8AB(I, I )
(2)
~AB(I, 2 )
8AB(I,3)
OAB(2, I )
BAB(2, 2 )
8AB(2, 3 )
This formulation of a log-linear model is similar to an ANOVA
model. A set of log-probabilities is d e c o m p o s e d into the grand
mean effect (U), main effects (~), and interaction effects
(8). For example, CA(1) represents the impact of the first
category of variable A on the log-probability vector because
~A(1) appears in the equation only when i A = I. And 8AB(I,2)
represents the joint impact (the interaction effect) of the
first category of A and the second category of B on log PAB
because
8AB(I,2)
occurs only when i A = I and i B = 2.
I D this model, however, there are many more paramet,~rs than
equations (cells). Hence, we introduce usual ANOVA constraints
such that
2
ZJA = I sA(JA ) = O
3
Zj
84
= I ~B(JB ) = O
2
s
= I gAB(JA'JB ) = O
for all
JB
~j3
B = I BAB(JA'JB ) = O
for all
JA"
Eliminating
those
JB = 3 from
(2) and
solvable
parameters
for w h i c h
(3), we o b t a i n
the
(3)
either
JA = 2 or
following
uniquely
model:
I
I
I
0
1
m
Oi
I
I
0
I
0
I
I
I -I
-I
-I
-I
eB(1)
0
eB(2)
-p
CA(1)
log PAB =
I -I
I
0 -I
I -I
0
I
0
-I
I -I
-I
-I
I
I
(4)
8AB(I ,I)
BAB(I ,2)_
N o w each p a r a m e t e r is m e a s u r e d in r e l a t i o n to the p o i n t w h e r e
the A N O V A c o n s t r a i n t s
(3) hold, or w h e r e the a p p r o p r i a t e sums
of p a r a m e t e r s
(or t h e i r mean) are set to zero. I m p l i c i t param e t e r s , w h i c h a p p e a r in (2) but do not a p p e a r in (4), can be
c a l c u l a t e d from the r e s t r i c t i o n s
(3) if we k n o w all the exp l i c i t p a r a m e t e r s in (4).
In g e n e r a l , any m u l t i v a r i a t e
l o g - l i n e a r m o d e l for q v a r i a b l e s
(I I •215
Iq) has a r e p r e s e n t a t i o n as in e q u a t i o n (4):
log P =
Qxl
0 9 0
=
QxQ Qxl
e 9 p +
Q~I
U
Qx(Q-1)
0
(5)
(Q-l~xl
q
where
rank
(U) = Q
(Q = H
Ii) ,
i=I
is the g e n e r a l p a r a m e t e r
v e c t o r of U w h i c h c o n s i s t s
vector, and e is the
of o n l y one's.
Since P denotes a probability,
ing r e s t r i c t i o n s :
we w a n t
first
to i m p o s e
the
column
follow-
Q
P
i=I
1
= I
O < P. < I
l
for all
i.
85
T h e s e two c o n s t r a i n t s
are a u t o m a t i c a l l y
p o s e one a p r i o r i r e s t r i c t i o n on It such
satisfied
that
if we
im-
0
= - log
(E
cxp (Z i)))
(6)
i=I
where
This
Z(i)
can
= log Pi
be
shown
- I,, or
as
Pl
follows.
= exp(l, + Z(i))
From
for all
i.
(6),
0
O = u + log(R
e x p ( Z (i)) ).
i=I
Exponentiating
both
sides,
we
obtain
0
exp(O)
= I = exp(p)
exp(log
E
exp(Z(i)))
i=I
0
= e x p (ll)
(E
exp(Z(i)) )
i=I
=
Q
~i
exp(it + Z(i)).
i=I
Q
Therefore
E
i=I
P. = I.
i
0
Since
that
P. = e x p ( u
1
O < P
l
+ Z(i))
< I for
> O,
the
equation
F
i=I
all
P. = I i m p l i e s
l
i.
In the e q u a t i o n
(5), the r a n k of U is Q. S u c h a m o d e l is
c a l l e d a s a t u r a t e d m o d e l . It c o n t a i n s as m a n y i n d e p e n d e n t
parameters
as t h e r e are c e l l s in the c o n t i n g e n c y
table.
T h e r e f o r e , the s a t u r a t e d m o d e l a l w a y s fits e x a c t l y the obs e r v e d data. But we m a y o f t e n i m p o s e a s p e c i a l s t r u c t u r e on
a m o d e l by s u p p r e s s i n g
certain parameters.
We c a l l such _models
unsaturated
m o d e l s ; t h e s e are m o d e l s in w h i c h I =< r a n k (U)<Q.
It is p o s s i b l e to c o n s t r u c t v a r i o u s k i n d s of u n s a t u r a t e d
models based upon different hypotheses.
The simplest model
c o n t a i n s o n l y the o v e r a l l e f f e c t . In the c a s e of two v a r i a b l e s
it is g i v e n by
log
This
same
86
PiAiB
= H.
m o d e l i m p l i e s t h a t all the
and e q u a l to I / ( I A - I B ) .
(7)
cell
probabilities
are
the
Including the main effects of variables A and B, we obtain a
model representing independence between random variables A
and B:
log PiAiB
(8)
U + eA(iA ) + eB(iB )-
The main effects eA and ~B measures the portion of the logprobability which is explained neither by the overall scale
factor nor by the association between A and B. In the absence
of interaction effects, the main effects represent the marginal distributions 9 The independence model (8) can be easily
converted to the more familiar form consisting of the product
of two marginals in the following manners. First note that
I = IA'ZiB'~ PiAiB = exp(u)'iAZ exp(~A(iA))'i~
exp(eB(iB)),
I
so that exp(u) = iA~ exp(eA(iA)). ZIB exp(eB (iB))
The marginal distributions
Pi A- = i~ PiAi B
---
of model
(~) are
~
exp(u) exp(eA(iA))
.
.
P.i B . i~ PiAi
B . exp(u)
exp(eB(iB))
Therefore,
PiAiB
=
9
i~ exp(eB(iB))'
and
.E exp(~A(iA))iA
we obtain
exp(~)'exp(eA(iA))
exp(~B(iB))
= PIA. P iB.
.
A log-linear model can be in general written as
log
Q•
P
=
U
Q•
--~
S•
=
~
e
Q~I
+
U
Qx(S-1)
8
(S-I~•
(9)
where I < S < Q
=
=
Q
exp(Z(i)) )
u = -log(E
i=I
Z(i) = log Pi = ~"
87
Now let us c o n s i d e r e q u a t i o n (9) in terms of a vector space.
The space of io@ P in (9) is s~anned by a set of S l~nearly
i n d e p e n d e n t column vectors of U. Thus !o@ P belongs to an Sd i m e n s i o n a l l i n e a r s u b s p a c e of R Q. We call such a model space
M. And we also call U a basis m a t r i x of M and a column v e c t o r
u. of U a basis v e c t o r of M.
--3
R e t u r n i n g to model (4), we notice that there are four different groups of parameters; the o v e r a l l effect ~, main effects eA(iA) and ~B(iB), and b i v a r i a t e i n t e r a c t i o n effects
8AB(iA,iB).
exactly
We call
such a group of p a r a m e t e r s
the same v a r i a b l e s
that involves
a configuration.
A partial o r d e r among c o n f i g u r a t i o n s can be d e f i n e d by the
inclusion r e l a t i o n s h i p of the set of involved variables. For
example, {~} < {eA(iA)} < {~AB(iA,iB)} b e c a u s e ~ C { A } C { A , B } .
A model is c a l l e d a h i e r a r c h i c a l model if the model includes
all p o s s i b l e l o w e r - o r d e r c o n f i g u r a t i o n s of any c o n f i g u r a t i o n
in the model. Model (4) and model (8) are h i e r a r c h i c a l models.
But the f o l l o w i n g model is not a h i e r a r c h i c a l model because
it lacks a l o w e r - o r d e r c o n f i g u r a t i o n {eA(iA)} of the configuration {SAB(iA,iB)}:
log PiAiB
= ~ + ~B(iB)
+ 8AB(iA,iB).
In this paper we are only c o n c e r n e d
with h i e r a r c h i c a l
models.
We assume that the r a n d o m v e c t o r n is m u l t i n o m i a l l y distributed. Our d i s c u s s i o n deals with ~he simple m u l t i n o m i n a l
model. For d e t a i l e d study of s a m p l i n g d i s t r i b u t i o n s of loglinear models, see [ B i s h o p et al., 1975, Ch. 3, Ch. 131 and
[Haberman, 1974, Ch. 11. T h e r e f o r e , the p r o b a b i l i t y d e n s i t y
function is given by
n
!
n.
f({ni} ; {mi} ) =
~i~i
~i~1
ni
!
Q
where
n
= 7
i=I
n.
1
E(n i) = m i > O
m
=
P l9
= -n
i = I, 2,...,Q
n
m.
1
~8
i = I, 2,
""
. , Q.
p.
l
l
(10)
We note that in this model the t o t a l sample size
and is e q u a l to the total e x p e c t e d f r e q u e n c y m..
l i k e l i h o o d f u n c t i o n of this m o d e l is
log L(n,
where
m(8_))
log P = ~
P =--
n. !
Q
Q
+ ~i=I
H n
!
i=I i
= log
n. is fixed
The log-
n i log Pi'
(11)
+ U ~,
I
m.
Now we o b t a i n the first p a r t i a l d e r i v a t i v e s and the s e c o n d
p a r t i a l d e r i v a t i v e s of the l i k e l i h o o d f u n c t i o n log L ( ~ , m ( ! ) )
w i t h r e s p e c t to the p a r a m e t e r s i:
log
0
log m
L = ;)log
L
~ log m
~ 8_
Q
log m [ ~
Q
i=I
n
n~
=
(n
m)
log m . - n
log
l
l
9
T
~
i=I
mi]
~ log m
~ 0
--
(log n + log P)
EiQ1m i
=
(n - _m) T ~
=
(n - m) T U
(log n + ~ e + He_)
(12)
~21og L
~ 3 log L
~ UT
~)O i)O T
= ~00_T '
= ~- [
n.
(n
m.
i=I Z
n.
[u T
log m
(n
m)l
Q
~ log m
)]
--
Q
~8_
(m i )
i=I
= - uT
where
D
m
I
(Dm - ~T. _m m_T)
is a QxQ d i a g o n a l
matrix
U
(13)
whose
diagonal
elements
are m_.
~9
Since
this Hessian
for example,
has a unique
matrix
~21~~0 T L is p o s i t i v e - d e f i n i t e
20
[Bock, 1975~ p. 525]),
m a x i m u m at 9 such that
(fi _ ~ ( ~ ) ) T
the likelihood
(see,
function
U = O.
(14)
The m a x i m u m - l i k e l i h o o d
estimate 9 is a s y m p t o t i c a l l y normally
distributed. The asymptotic covariance matrix is given by the
negative
inverse
of the Hessian
matrix
~2 loa L
~0
0. Hence,
asymptotic
of the diagonal
standard
elements
deviations
of - (
evaluated
at
~0 T
are the square
-I
roots
32 log L)
~0
~0 T
By dividing each element of 0 by its c o r r e s p o n d i n g standard
deviation, we obtain asymptotic t-rations for the estimates.
Hypotheses about the parameters can also be tested by the
large-sample l i k e l i h o o d - r a t i o test. For instance, we obtain
the values of the likelihood function maximized under two
different p a r a m e t e r structures, O_O and ~I ' such that -O0 involves
constraints
upon ~I"
Then the q u a n t i t y
chi-squared
of freedom
obtain
in addition
distribution
equal
under
the number
the structure
~O"
to the constraints imposed
L(0_O)
- 2 log ~
has an asymptotic
the hypothesis
of restrictions
~O"
The degrees
imposed
We may use Pearson's
on ~I to
goodness-of-f~t
test, which is a s y m p t o t i c a l l y equivalent to the likelihood
test. Haberman [ 1974, Ch. 41 and Bishop et al. [ Iq75, Ch. 14]
studied the asymptotic properties of log-linear models.
3. A Problem
in the A p p l i c a t i o n
of Log-Linear
Models
As an a p p l i c a t i o n of the log-linear model d i s c u s s e d above, we
consider a set of data used by Srole et al. (1962). Respondents in the Midtown Manhattan are c r o s s - c l a s s i f i e d according
to the levels of their mental health and p a r e n t a l socioeconomic status (Table I). For this data the following 4x6
saturated log-linear model is estimated;
log Pij = ~ + aM(i)
where i and j denote
socioeconomic status
+ as(J)
the indexes of mental health
(S) respectively such that
i = I, 2, 3, 4 and
j = I, 2, 3, 4, 5, 6.
9O
(15)
+ ~MS (i'j)
(M) and
The indexes are in d e s c e n d i n g o r d e r such that i = I and j = I
d e n o t e the h i g h e s t s o c i o e c o n o m i c status and the best m e n t a l
health condition respectively,
Table
I: D i s t r i b u t i o n s of R e s p o n d e n t s on M e n t a l H e a l t h
d i t i o n s by P a r e n t a l S o c i o e c o n o m i c S t a t u s
Con-
Parental Socioeconomic Status
Mental Health
Condition
I. (highest)
2.
3.
4.
5.
6. (lowest)
I. Well
64
57
57
72
36
21
2. Mild sympton
formation
94
94
105
141
97
71
3. Moderate sympton
formation
5;~
54
65
77
54
54
4.
46
40
~0
94
78
7]
Impaired
Total number of observations: 1660
Source: Stole et al. (|962), p. 213.
The m a x i m u m - l i k e l i h o o d e s t i m a t e s of the e f f e c t s p a r a m e t e r s
are given in T a b l e 2. These e s t i m a t e s give us d e t a i l e d inf o r m a t i o n on the r e l a t i o n of the two v a r i a b l e s to the obs e r v e d p a t t e r n of p r o b a b i l i t i e s . The main e f f e c t s eM and ~S
r e p r e s e n t the p o r t i o n s that are e x p l a i n a b l e o n l y by one
v a r i a b l e w i t h o u t r e f e r e n c e to the o t h e r v a r i a b l e . The interaction e f f e c t s 8MS i n d i c a t e the p o r t i o n due to the association
between
mental
health
and
socioeconomic
status.
A c c o r d i n g to the p a r a m e t e r e s t i m a t e s g i v e n in T a b l e 2, the
s e c o n d main e f f e c t of m e n t a l h e a l t h has a r a t h e r large value
w h i l e the first has a small value. This implies that the
s a m p l e shows a c o n c e n t r a t i o n at the second m e n t a l h e a l t h level
and a d e p r e s s i o n at the first level due to some u n k n o w n factors i n d e p e n d e n t of the o t h e r i n c l u d e d v a r i a b l e ( s o c i o e c o n o m i c
status). In the same sense the p a r e n t a l s o c i o e c o n o m i c status
has a c o n c e n t r a t i o n at the fourth level and a d e p r e s s i o n at
the sixth level. E s t i m a t e d b i v a r i a t e i n t e r a c t i o n effects show
that there are s t a t i s t i c a l l y s i g n i f i c a n t p o s i t i v e r e l a t i o n s
b e t w e e n the first levels of the two v a r i a b l e s and b e t w e e n
their last levels, and that n e g a t i v e r e l a t i o n s e x i s t b e t w e e n
the first m e n t a l h e a l t h level and the last s o c i o e c o n o m i c
status, b e t w e e n the last m e n t a l h e a l t h level and the second
s o c i o e c o n o m i c status, and b e t w e e n the last m e n t a l h e a l t h
level and the first s o c i o e c o n o m i c status. In this way the loglinear m o d e l p r o v i d e d e t a i l e d c a t e g o r y - b y - c a t e g o r y i n f o r m a t i o n
on the r e l a t i o n s h i p s a m o n g q u a l i t a t i v e v a r i a b l e s .
However, we o f t e n require some more g e n e r a l i n f o r m a t i o n . Forexample, we m a y want to k n o w w h e t h e r or not m e n t a l h e a l t h is
p o s i t i v e l y (negatively) a s s o c i a t e d with s o c i o e c o n o m i c status,
what the d e g r e e of the a s s o c i a t i o n is, or w h i c h is the most
d o m i n a n t c o m p o n e n t of the three c o n f i g u r a t i o n s i n c l u d e d in
91
Table
2: E s t i m a t e d
Effect Parameters
of M o d e l
(15)*
Main Effects of Mental Health (M)
I
-0.30824
(5.992)
2
0.42126
(10.641)
3
-0.07559
(1.647)
4
-0.03743
(O.816)
Main Effects of Socioeconomic Status (S)
I
-0.01871
(0.320)
2
-0.10047
(1.654)
3
0.07491
(1.332)
4
0.36161
(7.145)
5
-0.04054
(0.676)
6
-0.27681
(4.067)
Bivariate Interaction Effects of Mental Health (M) and Socioeconomic
Status (S)
S
!
M
2
3
4
5
6
i
0.31931
(3.105)
0.28524
(2.662)
0.10986
(1.O50)
0.05677
(0.594)
-0.23422
(1.925)
-0.53695
(3.584)
2
-0.02578
(O.291)
0.05598
(0.622)
-0.00874
(O.103)
-0.00063
(0.008)
0.02747
(0.309)
-0.04830
(0.478)
3
-0.01178
(O.115)
-0.00148
(O.O14)
0.00854
(0.087)
-0.10874
(1.194)
-0.06141
(0.581)
0.17486
(1.583)
4
-0.28174
(2.565)
-0.33974
(2.932)
-0.10966
(1.O90)
0.05259
(0.607)
0.26816
(2.785)
0.41039
(3.962)
* t-values in parentheses
the model. The p a r a m e t e r e s t i m a t e s such as given in T a b l e 2,
at least as they are, are not v e r y h e l p f u l in a n s w e r i n g these
q u e s t i o n s , e s p e c i a l l y w h e n the n u m b e r of c a t e g o r i e s (levels)
of q u a l i t a t i v e v a r i a b l e s are large. This p r o b l e m f r e q u e n t l y
o c c u r s in the a p p l i c a t i o n of l o g - l i n e a r models. L.A. G o o d m a n
(1973) s t u d i e d c a u s a l r e l a t i o n s h i p in l o g - l i n e a r m o d e l s and
p r o p o s e d to s u m m a r i z e the r e l a t i o n s h i p a m o n g c a t e g o r i c a l
v a r i a b l e s by a d i a g r a m a n a l o g o u s to the p a t h diagram. W h e n
the i n v o l v e d v a r i a b l e s are all d i c h o t o m o u s , there are no diff i c u l t i e s in s u c h a r e p r e s e n t a t i o n b e c a u s e there is o n l y one
free e f f e c t p a r a m e t e r for each c o n f i g u r a t i o n . H o w e v e r , the
d i a g r a m b e c o m e s r a t h e r c o m p l i c a t e d and less i n f o r m a t i v e w h e n
the i n v o l v e d v a r i a b l e s have m a n y c a t e g o r i e s .
One s o l u t i o n to this p r o b l e m , w h i c h we p r o p o s e , is s u m m a r i z i n g
the i n f o r m a t i o n s c a t t e r e d over m a n y e f f e c t p a r a m e t e r s by a
single a s s o c i a t i o n m e a s u r e for e a c h c o n f i g u r a t i o n . If this is
p o s s i b l e , the above l o g - l i n e a r m o d e l is c o m p a c t l y r e p r e s e n t e d
o n l y by three a s s o c i a t i o n m e a s u r e s . In the f o l l o w i n g s e c t i o n s
we d e v e l o p a m e t h o d to c o n s t r u c t such c o m p a c t m e a s u r e s . This
p r o c e d u r e c o n s i s t s of two stages. First, the p a r a m e t e r
e s t i m a t e s are c o n v e r t e d into sets of p r o b a b i l i t i e s , each of
w h i c h r e p r e s e n t s o n l y one c o n f i g u r a t i o n . Next, from these
p r o b a b i l i t i e s , some m e a s u r e s of a s s o c i a t i o n are c a l c u l a t e d .
We may choose a m o n g s e v e r a l a l t e r n a t i v e m e a s u r e s of
92
association. The selection depends upon the nature of data
and the special hypothesis about association that the investigator is concerned with.
4. The Concept of the Component Probability
The first step of our procedure is to decompose the expected
probability of a log-linear model into several components,
each of which represents one configuration 3). The saturated
two-way log-linear model can be rewritten as follows:
Pij = exp{~+~A (i) +aB (J) +BAB (i 'J ) }
{Zexp(~A(i) ) }" {Zexp(eB(j)) }. {Z Z exp(SAB(i,j)
i
j
i j
E E exp{~A(i)+~B(j)+~AB(i,j)}
i j
exp(~A(i) )
exp(eA(i))
i
l
]
exp(eB(j))
exp(SAB(i,j) )
~ exp(eB(j))
j
Z ~ exp(SAB(i,j
ij
}
(16)
l]
exp(~A(i))
where PA
1
E
exp(eA(i))
'
i
pB
]
H
exp(~B(j))
exp(eB(j))
'
J
pAB
exp (8AB (i, j ) )
ij - Z Z exp(SAB(i,j))
i j
and
C
{.Zexp(eA(i)) }. {~exp(~B(j)) }- {Z~exp(SAB(i,j)
i
~
ij
Z exp{~A(i) + eB(j) + ~.AB(i,j)}
i j
}
A
B
and pAB
We call Pi'
Pj'
ij in the above formulation component
probabilities. The sum of a set of component probabilities
over the indexes is one, and each element of component
probability is positive by definition. Thus, the component
probability'satisfies the formal requirement of a probability.
Each of the component probabilities represents exactly one
configuration of effect parameters. For example, the component
93
probability
PAl contains
probabilities
include
only cA(i) , and no o t h e r
aA(i).
Although
C involves
component
all effect
parameters, it is constant given the values of parameters.
Furthermore, note that each of the c o m p o n e n t p r o b a b i l i t i e s
r e p r e s e n t s the p r o b a b i l i t y for a c o n f i g u r a t i o n when the param e t e r s of all o t h e r c o n f i g u r a t i o n s are zero. For instance, if
eA(i) = O and eB(j) = O for all i and j, then it is clear,
pAB
that Pi3
ij" T h e r e f o r e , the c o m p o n e n t p r o b a b i l i t y repre9
=
sents a pure or partial p r o b a b i l i t y in the sense that the influence of all o t h e r c o n f i g u r a t i o n s is s i m u l t a n e o u s l y removed. In c o n t r a s t to this, the m a r g i n a l p r o b a b i l i t y represents a gross r e l a t i o n s h i p including not only the config u r a u i o n c o n c e r n e d but also all other c o n f i g u r a t i o n s . For
example,
P..
(marginal
probability
of A) = Z P
l
9
exp(eA(i))
Zexp(eB(j)
.
i]
+ BAB(i,j))
J
Z Z exp(eA(i)
i j
+ r/B(j) + HAB(i,j))
'
exp(~A(i)
while
PA (component
1
probability
of A) =
X exp(aA(i))
i
In o r d e r to make the m e a n i n g of the c o m p o n e n t p r o b a b i l i t y
clearer, we c o n s i d e r the case where two variables are independent. Then, the i n t e r a c t i o n p a r a m e t e r s BAB(i,j) are all
zero.
This
are e x a c t l y
ly.
implies
Furthermore,
equation
that
the c o m p o n e n t
the m a r g i n a l
(16)
probabilities
probabilities
PA and PB
i
3
of A and B r e s p e c t i v e -
C 9 pAB = I for all i and j. Therefore,
13
to
reduces
P
13
= P A 9 pB.
i
]
(17)
This is the c l a s s i c a l f o r m u l a t i o n of independence. Equation
(16) is a general e x p r e s s i o n of the p r o b a b i l i t y c o n t a i n i n g
e q u a t i o n (17) as a special case.
One i m p o r t a n t p r o p e r t y of the c o m p o n e n s p r o b a b i l i t y is that
the e l e m e n t s are e q u i p r o b a b l e if and only if the values of
the included effect p a r a m e t e r s are all zero. In proving this
p r o p o s i t i o n we c o n s i d e r only a b i v a r i a t e case b e c a u s e the
g e n e r a l i z a t i o n to other cases is s t r a i g h t f o r w a r d . First, if
~ij = O for all i and j, then it is obvious that
94
pAB =
13
exp(Bij)
J
~ exp(Bij)
i=I 3=I
Conversely,
Then,
_
I
I
I.J
for all i and j
"
assume
it follows
that pAB = pAB
for all i, j, i', and j'
ij
i'j'
that 8i5 = 8i,~, for all i n v o l v e d indexes.
I
J
~
8i
=
~ 8ij = O. Therei=I
J
3=I
fore, 8. 9 = O for all i and j. Thus, in this sense, the equi13
p r o b a b i l i t y case serves as a c r i t e r i o n to e v a l u a t e the imp o r t a n c e of a c o n f i g u r a t i o n in c o m p o n e n t p r o b a b i l i t y .
F r o m the A N O V A r e s t r i c t i o n s ,
As shown above, the l o g - l i n e a r model can be e x p r e s s e d in
terms of c o m p o n e n t p r o b a b i l i t i e s . This mode of r e p r e s e n t a t i o n
plays an i m p o r t a n t role in m e a s u r i n g the i n f o r m a t i o n contained in a c o n f i g u r a t i o n ; it enables us to apply w e l l d e v e l o p e d
m e a s u r e s of a s s o c i a t i o n in the f r a m e w o r k of the l o g - l i n e a r
model.
5. A p p l i c a t i o n of Some M e a s u r e s of A s s o c i a t i o n to the LogL i n e a r Model and the A s y m p t o t i c D i s t r i b u t i o n s of the
Estimated Measures
For many years various m e a s u r e s of a s s o c i a t i o n have been developed and w i d e l y used for the s t a t i s t i c a l a n a l y s i s of categ o r i c a l data. While there exists a vast a m o u n t of l i t e r a t u r e
on this topic, we may cite, as recent i m p o r t a n t works, G o o d m a n
and K r u s k a l (1954, 1959, 1963, and 1972), K e n d a l l (1970),
Somers (1962), and W i l s o n (1974) a m o n g others. One reason for
this a b u n d a n c e of m e a s u r e s is that there is no unique defin i t i o n of a s s o c i a t i o n . The choice of m e a s u r e s d e p e n d s p a r t l y
on the n a t u r e of data such as nominal or o r d i n a l data and
p a r t l y on the aspects of a s s o c i a t i o n that the r e s e a r c h e r is
c o n c e r n e d w~th.
So far these m e a s u r e s of a s s o c i a t i o n have b e e n m a i n l y c o m p u t e d
from o b s e r v e d data. I n s t e a d we a p p l y these m e a s u r e s to the
c o m p o n e n t p r o b a b i l i t y c o n s t r u c t e d from the e f f e c t p a r a m e t e r s
of the l o g - l i n e a r model; since the a s s o c i a t i o n m e a s u r e is
c a l c u l a t e d for a set of f r e q u e n c i e s or p r o b a b i l i t i e s , it is
also p o s s i b l e to c o m p u t e such m e a s u r e s from a set of c o m p o n e n t
p r o b a b i l i t i e s . In this way we can c o m p a c t l y s u m m a r i z e the inf o r m a t i o n c o n t a i n e d in a c o n f i g u r a t i o n in t h e l o g - l i n e a r
model. As is clear from the d i s c u s s i o n about the c o m p o n e n t
p r o b a b i l i t y , the a s s o c i a t i o n m e a s u r e d e r i v e d from the comp o n e n t p r o b a b i l i t y r e p r e s e n t s a pure or p a r t i a l r e l a t i o n s h i p
of the c o n c e r n e d c o n f i g u r a t i o n in the sense that the i n f l u e n c e
of all o t h e r c o n f i g u r a t i o n s are removed. F r o m among m a n y prop o s e d m e a s u r e s , we c o n s i d e r the f o l l o w i n g six measures; the
P h i - s q u a r e (~2), the s y m m e t r i c and a s y m m e t r i c L a m b d a ' s (I)
for n o m i n a l c a t e g o r i c a l variables; the s y m m e t r i c gamma (y), e,
95
and the a s y m m e t r i c d for o r d i n a l c a t e g o r i c a l variables. Although our s e l e c t i o n is never e x h a u s t i v e , they cover a wide
range of a l t e r n a t i v e h y p o t h e s e s and data. Besides, it is
g e n e r a l l y quite s t r a i g h t f o r w a r d to apply the p r o p o s e d method
to any w e l l - d e f i n e d a s s o c i a t i o n measure.
Pearson's c o e f f i c i e n t ~2 m e a s u r e s the d i f f e r e n c e b e t w e e n a
set of p r o b a b i l i t i e s Pk and their e x p e c t a t i o n s Pk of a c e r t a i n
r e s t r i c t e d model:
~2
K
=
(Pk-Pk)2
~
k=1
,
(18)
Pk
w h e r e K is the number of cells for the d i s c r e t e
Pk" A l t h o u g h Pk is u s u a l l y a s s u m e d to r e p r e s e n t
model,
we use the e q u i - p r o b a b i l i t y
probabilities
an i n d e p e n d e n c e
as Pk: the e q u i - p r o b a b i l i t y
case in c o m p o n e n t p r o b a b i l i t y c o r r e s p o n d s to the zero values
of effect p a r a m e t e r s as d i s c u s s e d in section 4. So in our
framework ~2 m e a s u r e s how d i f f e r e n t the a s s o c i a t i o n for a
given c o n f i g u r a t i o n is from the case where the values of effect p a r a m e t e r s are all zero. As o f t e n p o i n t e d out, a disadvantage of ~2 is that it has no upper bound. On the other
hand this m e a s u r e can be a p p l i e d not o n l y to b i v a r i a t e interaction effects but also to main effects and any h i g h e r - o r d e r
interactions.
Some popular a l t e r n a t i v e s to ~2 are Lambda's 4), w h i c h are
based upon the p r o p o r t i o n a l - r e d u c t i o n - i n - e r r o r
(PRE) logic.
Assume that we have two c a t e g o r i c a l variables A (with index
i=I,2,...,I) and B (with index j=1,2,...,J). What p r e d i c t i o n
can be made for the c a t e g o r y of A of a r a n d o m l y s e l e c t e d obs e r v a t i o n w i t h o u t k n o w i n g the c a t e g o r y of B of the a b s e r v a t i o n ?
In this case one can decide for the c a t e g o r y with the largest
m a r g i n a l p r o p o r t i o n . Then, the p r o b a b i l i t y of m a k i n g errors
under this scheme is
EP(1)
= I - P .
m
where
Pm"
~ max
i
(Pi")"
On the other hand, if the i n v e s t i g a t o r knows the c a t e g o r y
v a r i a b l e B, the error p r o b a b i l i t y is given by
EP(2)
of
= I - Z Pmj"
J
1A is d e f i n e d
96
as the r e d u c t i o n
in e r r o r b e t w e e n
the two cases;
(1-P
IA = EP(1)
By c h a n g i n g
- EP(2)
EP(1)
the
(a-P.m)
IB =
=
variables,
-
) m-
(I - Z Pmj)
j
I - P
"
m.
(19)
we o b t a i n
(1-E P i m )
i
I - P
(20)
.m
1A
and
1B are
asymmetric
in t h a t
their
values
d i f f e r e n t for a g i v e n c o n t i n g e n c y
table.
of I c a n be d e r i v e d by the same logic;
{I
I .
-
.
~I( P m . + P . m ) }
.
.
.
-
[I - 89
J
are
in g e n e r a l
A symmetric
Pmj +~. Pim)}
1
version
(21)
I - 89
These
lity.
l's
can
be d i r e c t l y
computed
from
the
component
probabi-
For o r d i n a l m e a s u r e s of a s s o c i a t i o n
we use G o o d m a n a n d K r u s k a l ' s g a m m a 5), S o m e r s ' d 6), and W i l s o n ' s e 7). A s s u m e that
t h e r e are two o r d i n a l c a t e g o r i c a l
v a r i a b l e s A and B w i t h ind e x e s i and j r e s p e c t i v e l y .
C h o o s e r a n d o m l y any two observations,
(i, j) a n d (i', j'), w i t h r e p l a c e m e n t .
Then, the
probability
of o b t a i n i n g a c o n c o r d a n t p a i r (PS) is P r { i < i '
~ n d j<j', or i>i' and j>j'}, and the p r o b a b i l i t y
to get a
discordant
p a i r is P r { i < i ' and j>j', or i>i' and j<j'}. The
probabilities
of t i e s are d e f i n e d as
and
TA
= Pr{i
= i'
and
j ~ j'},
TB
= Pr{j
= j'
and
i ~ i'},
TAB
= Pr{i
= i' a n d
j = j'},
T
= Pr[i
= i' or
j = j'}
= T A + TB
+ TAB.
G a m m a , d, and e all m e a s u r e s the d i f f e r e n c e b e t w e e n the concordant probability
and d i s c o r d a n t p r o b a b i l i t y ,
while they
d i f f e r in the d e n o m i n a t o r ;
PS - PD
= PS + PD
'
(22)
dAB
PS - PD
= PS + PD + T A
'
(23)
dBA
PS - P D
= PS + PD + T B
'
(24)
97
and e
=
PS - PD
PS + PD + TA + T B "
(25)
We note that u and e are s y m m e t r i c , but that d is a s y m m e t r i c
so that there are two v e r s i o n s of d. M e a s u r e e is s t r i c t e r
than y in that e is not n e c e s s a r i l y equal to its u p p e r b o u n d I
(or its lower bound -I) e v e n if u is I (or -I) 8). We can
e a s i l y c a l c u l a t e these three m e a s u r e s from a set of c o m p o n e n t
probabilities.
K a w a s a k i (1979, 1980) d e v e l o p e d an a s y m p t o t i c s a m p l i n g t h e o r y
for the g a m m a c o e f f i c i e n t in the l o g - l i n e a r model. The same
m e t h o d can be e a s i l y a p p l i e d to o t h e r a s s o c i a t i o n measures.
The a s y m p t o t i c d i s t r i b u t i o n of the e s t i m a t e d a s s o c i a t i o n
m e a s u r e s can be d e r i v e d by the d e l t a m e t h o d b e c a u s e these
m e a s u r e s are f u n c t i o n s of p a r a m e t e r s w h i c h are a s y m p t o t i c a l l y
n o r m a l l y d i s t r i b u t e d if, for instance, the m a x i m u m - l i k e l i h o o d
m e t h o d is employed. For the g e n e r a l t h e o r e m on the delta
method, see, for example, C r a m ~ r (1946, pp. 366-367) and
B i s h o p et al. (1975, pp. 492-494). Since we e s t i m a t e the
p a r a m e t e r s ~ of the l o g - l i n e a r m o d e l by the m a x i m u m - l i k e l i hood m e t h o d and n e c e s s a r y r e g u l a r i t y c o n d i t i o n s are s a t i s f i e d ,
the t e r m /n (~n - ~) is a s y m p t o t i c a l l y n o r m a l l y d i s t r i b u t e d
w i t h m e a n zero and a ~ o s i t i v e d e f i n i t e c o v a r i a n c e Z. The
e s t i m a t e d m e a s u r e V (~n) is c l e a r l y a t o t a l l y d i f f e r e n t i a b l e
f u n c t i o n of ~n"
Then,
d i s t r i b u t i o n of V
by the d e l t a method,
the a s y m p t o t i c
(Sn)_ is a n o r m a l w i t h m e a n V(8)
I .~V.T
and v a r i a n c e
~V
N o t e that a m e a s u r e i n v o l v e s o n l y one c o n f i g u r a t i o n . T h u s it
s h o u l d be u n d e r s t o o d that 8 and Z in the a b o v e formula of the
v a r i a n c e are in a p p r o p r i a t e l y a b r i d g e d forms.
S i n c e the t e r m ~Z can be a p p r o x i m a t e d by the n e g a t i v e i n v e r s e
n
of the H e s s i a n m a t r i x of the l o g - l i k e l i h o o d f u n c t i o n e v a l u a t e d
at the m a x i m u m , we have o n l y to c o n s i d e r the p a r t ~V/~ B in
o r d e r to d e r i v e a c o m p u t a b l e e x p r e s s i o n for the v a r i a n c e of
the measures. The a p p e n d i x c o n t a i n s the d e r i v a t i o n s for the
selected association measures.
6. A S o l u t i o n to the P r o b l e m of S e c t i o n
3
N o w e q u i p e d w i t h the m e t h o d d i s c u s s e d above, we return to the
e x a m p l e c o n s i d e r e d in s e c t i o n 3. Table 3 p r e s e n t s several ass o c i a t i o n m e a s u r e s c a l c u l a t e d from c o m p o n e n t p r o b a b i l i t i e s as
d e s c r i b e d in s e c t i o n 4 and 5. r
i n d i c a t e s that all three conf i g u r a t i o n s (the m a i n e f f e c t s of m e n t a l h e a l t h and socioe c o n o m i c status and t h e i r i n t e r a c t i o n effect) are s t a t i s t i c a l ly s i g n i f i c a n t l y d i f f e r e n t from zero. In o t h e r words, the
98
Table
3: A s s o c i a t i o n M e a s u r e s C a l c u l a t e d
P r o b a b i l i t i e s o f M o d e l (15)*
from
Component
Configurations
Measures
r
1
Main effects of
Mental Health
0.08037
(5.166)
-
l
Main effects of
Socioeconomic
Status
Bivariate Interdction between
Mental Health and
Socioeconomic Status
0.04217
(3.685)
0.04151
(3.2~7)
-
0.06061
(6.219)
-
0.07883
(5.928)
-
0.04420
(4.267)
A
I
B
u
d
-
+
0.18698
(6.469)
-
+
0.14060
(6.44O)
-
+
0.15623
(6.439)
+
0.12247
(6.420)
AB
d
BA
* t-values in parentheses.
s t r u c t u r e o f o b s e r v e d c o n t i n g e n c y t a b l e m u s t be e x p l a i n e d n o t
o n l y b y the i n t e r a c t i o n b e t w e e n m e n t a l h e a l t h
and socioeconomic
status but also by some factors which determine the distribution
of e a c h v a r i a b l e s e p a r a t e l y a n d i n d e p e n d e n t l y .
We also note
t h a t the m a i n e f f e c t o f m e n t a l h e a l t h is b y f a r the s t r o n g e s t
of the t h r e e c o n f i g u r a t i o n s .
Alternative
n o m i n a l m e a s u r e s l's
c o n f i r m o n e o f a b o v e r e s u l t s t h a t the b i v a r i a t e i n t e r a c t i o n
is s t a t i s t i c a l l y
s i g n i f i c a n t . As for o r d i n a l m e a s u r e s , t h e y
a l l s h o w t h a t t h e r e is s t a t i s t i c a l l y
significant, moderately
positive associationbetween
mental health and parental socioe c o n o m i c s t a t u s . A l t h o u g h the i n s p e c t i o n o f i n t e r a c t i o n p a r a m e t e r s in T a b l e 2 g i v e s us an i m p r e s s i o n o f p o s i t i v e a s s o c i a t i o n , s u c h i n f o r m a t i o n is v a g u e , i n t i t u i t i v e ,
and sometimes misleading.
Furthermore,
the pattern of interaction parameters
is o f t e n
n o t so c l e a r as in t h i s case. O n t h e o t h e r h a n d , o u r m e t h o d
provides concrete numerical estimates and their asymptotic
standard errors.
99
Appendix
Derivation of the A s y m p t o t i c
A s s o c i a t i o n Measures
A. General
variances
of the Selected
Framework
As we pointed out in section
a s s o c i a t i o n measure V is
I
,~V,T
~_8)
5, the asymptotic
variance
of an
~V
~ (~_~) .
(26)
Since the term ~% can be a p p r o x i m a t e d by the negative inverse
n
of the Hessian matrix of the l o g - l i k e l i h o o d function evaluated at the maximum, we have only to consider the part ~ V / ~
in order to derive a computable e x p r e s s i o n for the variance
of the chosen a s s o c i a t i o n measure.
First We define a few terms n e c e s s a r y for the explicit
derivation of ~ V / ~ . We must d i s t i n g u i s h the explicit parameters 8, which are directly estimated, and the implicit parameters ~I' which are calculated from the explicit parameters
by means of the ANOVA restrictions. We denote the parameter
vector c o n s i s t i n g of both ~ and ~I by ~s" And we also note
that P denotes
By the chain
the component
probability
rule we decompose
~V _ ~ s
~V/~
in this
appendix.
as
~P ~V
(27)
where
T
8 = [81 I 81 2 "
--s
. .
8 IJ ~21
.
81(j_i)
821
.
.
8ijl
'
T
--8 = [ B11
812
...
P
P12
"'" PIJP21
Pij
= [P11
= exp(Sij)/
..-
8(i_I) (j_1)]
"'" PIJ IT'
~ ~ exp(Si,
),
i' j'
J'
i, i' = 1,2 ..... I
,
with
j, j' = 1,2 ..... J,
and
V
Since
(27)
1 O0
= ~2 , IAB,
the
first
are common
IA,
IB, y , dAB,
two terms
dBA'
C~Ss/~ ~ and
to all measures,
we
e.
gP/~Ss ) of expression
first discuss
the
derivation
maining
of ~ s / ~
term ~ V / ~
i) ~ s / ~ :
and D ~ / ~ s
and then consider
for each association
The ANOVA constraints
I-I
= - I]
i=I
J-1
= - Z
8(I,j)
~(i,J)
the re-
measure.
imply that
~(i,j),
~(i,j) ,
and
j=1
I-I
~(I,J)
= Z
J-1
~
i:I
~(i,j).
j:1
We also note that the ranges of indexes
Bs(i', j') of ~s and Z(i,j) of [~:
are different
i' = I, 2, ..., I;
j' = I, 2, .... J, and
I
j
= I, 2, ..., I-I;
Therefore,
for
= I, 2 . . . . , J-1.
we get
I
I if
' = i and j' = j,
I if i' = I and j' = J,
~ s ( i ' , j ')
3B(i,j)
-I if
' = i and j' = J,
-I if
' = I and j' = j,
(2S)
= O otherwise.
ii)
~/~s:
~Pi2
:
~-
Pi'j'
Pij
if i#i' or j#j'
~
Pi,j,(1-Pij)
(29)
~Si'j'
otherwise.
B. Derivation of 8V/8~ for Nominal Measures
i) The Phi-square
~2 :
is defined as
ZI
JE
i=I 3=I
As discussed
of Association
in section
(Pij - Pij)
P..
13
5, we assume that P.. denotes
13
element of the equi-probable model, so that
an
101
l]
Therefore,
we
IJ
"
obtain
~V
B~2
~P..
l]
~P..
lJ
In o r d e r to c a l c u l a t e
~
e f f e c t s o f the v a r i a b l e s
I = I.
With
ii)
these
The
2
l's
are
2IJ (Pij
and the
A o r B,
restrictions,
a
IAB
-
the
defined
I
(30)
- i7).
variance
o f it for the m a i n
w e h a v e to f i x J = I o r
formulas
are
the
same.
as
Pam + Z Pmb - P 9m - P mb
=
2 - P
- P
9m
Pmb
m-
- Pm.
b
hA
I - P
m.
=
~B
Z p
- p
am
-m
a
I - P
.m
where
Pam
= maxb
[Pab}'
Pmb
= maXa { P a b } '
P.m
= max
b
{ P . b }'
Pm.
= max
a
First
let
ib =
Then
102
us
define
some
new
[Pa. }"
indexes:
the
i index
of
Pib
= max
a
{ P a b },
Ja = the
j index
of
Paj
= max
b
{ P a b },
im = the
i index
of
Pi.
= max
a
{Pa. }'
Jm = t h e
j index
of
P.j
= m abx
{P " b } .
we
decompose
-~
such
that
and
~V
_
~I
~P
aN a l
=
~P
(31)
~P 8H ,
where
= HAB' HA' ~B'
and
T
~AB = [ P-m + P m. aE Pam + ~ Pmb ] '
HA
= [ Pm-
HB
= [P
The first terms
T
bE Pmb ] ,
E Pam ]
b
9m
~/~P
of decomposition
~HAB = [ "~---~--m~P
+ ~Pm.
aPij
~ij
~P''I3
~H
~p..
A =,
~P
13
~
B = ,
~P..
9J
13
~P
E P
~ E
a am +
b Pmb ] ,
aP..Ij
~P'lj
~ Z Pmb
b
],
~p..
13
m.
L ~p..
(31) for the l's are
(32)
B E P am
.m
a
~P, . 1,
~P..
13
13
where
a ~ Pam
a
I
if i = a and j = Ja
~Pij
O
otherwise,
i
if j = b and i = i b
O
otherwise,
E Pmb
F
b
=
~Pij
~Pm"
~
I
if i = i m
~Pij
= [
O
otherwise,
~P.m
= ~
I
if j = Jm
~-Pij
~
O
otherwise.
103
The
second
al/aH o f (31) are
terms
alAB
I Pam + g Pmb - 2
__
= [a
b
aRAB
(2 - P . m - P m -)2
g
- I
b Pmb
alA
I
(2 - P 9 m - Pm
)]'
I
(33)
aHA
= [
al B
_[a
(I
I P
aHB
C.
I - P
=
of
I - P.m
~V/~P
for O r d i n a l
in s e c t i o n
PS - PD
PS + PD
5, the
Measures
selected
of A s s o c i a t i o n
ordinal
'
=
PS - PD
PS + P D + T A
I
dBA =
PS - PD
PS + PD + TB
J
e
=
PS - P D
PS + PD + T A + TB'
=
2 g
dAB
]'
1 ] .
(I - P . m )2
As d i s c u s s e d
d e f i n e d as
m.
-I
am
Derivation
y
- Pm.)
2
where
PS
Z Pij
{
i j
PD
=
2 Z
~ P i 4J
i j
TA
TB
=
=
=
i'>i
j'<j
{
Pi,j, },
Pi,j,},
Z
(Pi-) 2 _ Z I
i j
(Pij) 2,
Z (P.4) 2 _ Z g (Pij) 2,
J
~ (Pi')
i
We d e c o m p o s e
ij
2
2
+ Z (P.j)
j
9
- ~ E (Pi.)-.j
ij
av/ a~ as is d o n e for I;
aV
~P
104
g
j'>j
i
J
T
Z
i'>i
aH
~P
aV
~H'
measures
are
where
V = y, dAB, dBA, e,
I = Iy,
,
I
IdAB
IdBA'
e
and
I
= [ PS PD]
T
,
T
IdA B = [ PS PD TA]
,
T
HdB A = [ PS PD TB]
I
Then,
i)
=
e
[PS
PD
,
TA
+
TBI
each term of ~V/~[ can be derived as follows:
~H/~P:
~p~
9.
,~PS
~PD
=t~[.
93
~.-l,
13
13
~IdAB
,~PS
~p..
=[-~[.
13
13
~PD
~p..
13
~TA 1
~P.. ,
13
(34)
~HdBA
~Pij
,~PS
=[~--~T..
lJ
~le
~Pij
= [~PS
~-~ij
where
SPD
~P i3.
~TB
~Pi--~'] '
~PD
~Pij
~TA
~P' "
13
~TB
~P' '] '
lj
9)
~PS
~P..
13
~PD
-
2
Z
E
i'>i
-
2
lJ
=
2
3TB
~P..
L]
=
2
+ 2
Z
i'<i
Pi'j'
~
i'>i
3TA
~Pij
Pi'j'
j'>j
j'<j
+ 2
[
i'<i
[ Pi'j'
j '<j
'
'Z Pi'j'
j'>j
P. - P..),
i"
13
(p.j
-
Pij)"
105
ii)
~V/~H:
~u
~H
= ~' 2(I
Y
~dAB
= [TA
~HdAB
~dBA
~HdBA
De
~H
-
PS
-2PD)
-2
(I - T)
= [TB
+
2(I
-
PS
-
,
T)
-TA
-
2 PS
I - T - 2 PS]
(I - T + TA) 2
(I - T + TA) 2
(I - T + TA) 2 '
+ 2(I
- TB - 2 PS
I - T-
- PS - T)
(I - T + T B ) 2
= [ TA+TB+2(I-PS-T)
2
(I - T + T A + T B )
e
PS2]
(I - T)
2 PS ]
(I - T + TB) 2 (I - T + TB) 2 '
-TA-TB-
2 PS
2
(I- T + T A + T B )
I - T-
2 PS]
(I-T+TA+TB) 2 "
(35)
Remarks
I) T h i s w o r k has b e e n d o n e as a p a r t of a r e s e a r c h p r o j e c t
on q u a l i t a t i v e b u s i n e s s m i c r o - d a t a a n a l y s i s o r g a n i z e d by
some m e m b e r s o f U n i v e r s i t y of M a n n h e i m , N o r t h w e s t e r n U n i v e r s i t y , and INSEE. T h i s p r o j e c t is p a r t l y f i n a n c e d by
the D e u t s c h e F o r s c h u n g s g e m e i n s c h a f t
(Grant 2 1 9 / 1 0 to H e i n z
K~nig). M a r c N e r l o v e , G e b h a r d Flaig, J o h n Link and Q u a n g
v u o n g p r o v i d e d h e l p f u l c o m m e n t s on p a r t s of a p r e v i o u s
d r a f t , b u t the a u t h o r s r e t a i n s o l e r e s p o n s i b i l i t y
for any
remaining errors.
2) K a w a s a k i (1979, 1980) used this m e t h o d J'or C o o d m a n and
K r u s k a l ' s g a m m a c o e f f i c i e n t . T he p r e s e n t p a p e r d e v e l o p s
the p r o c e d u r e f u r t h e r , and e x t e n d s it to a w i d e r a n g e of
association measures.
3) A c o n f i g u r a t i o n d e n o t e s a set of e f f e c t p a r a m e t e r s w h i c h
h a v e the same d e g r e e of i n t e r a c t i o n a n d i n v o l v e e x a c t l y
the s a m e set of v a r i a b l e s .
4) See G u t t m a n (1941) and G o o d m a n and K r u s k a l (1954).
5) See G o o d m a n a n d K r u s k a l (1954).
6) See S o m m e r s (1962).
7) See W i l s o n (1974).
8) As W i l s o n (1974) s t a t e s , u
d, and e c o r r e s p o n d to the
w e a k c o r r e l a t i o n , a s y m m e t r i c c o r r e l a t i o n , and s t r i c t correlation respectively.
9) ~ p s / ~ P i i and ~ P D / ~ P i i are the e x p r e s s i o n s d e r i v e d by G o o d man
and Kruskal
(1963,
p.
362).
References
B i s h o p , Y . M . M . , F i e n b e r g , S.E., and H o l l a n d , P.W. (1975),
D i s c r e t e M u l t i v a r i a t e A n a l y s i s : T h e o r y and P r a c t i c e . T h e
M I T P r e s s , C a m b r i d g e , Mass.
106
Bock, R.D. (1975), Multivariate Statistical Methods in Behavioral Research. McGraw-Hill, New York.
Cram~r, H. (1946), Mathematical Methods of Statistics.
Princeton University Press, Princeton.
Goodman, L.A. (1970), The multivariate analysis of qualitative
data: interaction among multiple classifications. J. Amer.
Statist. Assoc. 65, 226-256.
(1973), Causal Analysis of data from panel studies and
other kinds of surveys. Amer. J. Sociol. 78, 1135-1191.
and Kruskal, W.H. (1954), Measures of association for crossclassifications. J. Amer. Statist. Assoc. 49, 732-764.
and Kruskal, W.H. (1959), Measures of association for crossclassifications II: further discussion and references.
J. Amer. Statist. Assoc. 54, 123-163.
and Kruskal, W.H. (1963), Measures of association for crossclassifications III: approximate sampling theory. J. Amer.
Statist. Assoc. 5~, 310-364.
and Kruskal, W.H. (1972), Measures of association for crossclassifications IV: simplification of asymptotic variances.
J. Amer. Statist. Assoc. 67, 415-421.
Guttman, L. (1941), An outline of the statistical theory of
prediction, in Horst, P. and others (ed.), The Prediction
of Personal Adjustment, Bulletin 45, Social Science Research Council, New York, 253-318.
Haberman, J.S. (1973), Log-linear models for frequency data:
sufficient statistics and likelihood equations. Ann.
Statist. I, 617-632.
(1974), The Analysis of Frequency Data. The University of
Chicago Press, Chicago.
Kawasaki, S. (1979), Application of Log-linear Probability
Models in E c o n o m e t r i c s . P h . D . dissertation, Department of
Economics, Northwestern University.
(1980), An Extension of Goodman and Kruskal's Gamma Coefficient in the Context of the Log-linear Probability Model,
Discussion Paper No. 140/80, University of Mannheim.
Kendall, M.G. (1970), Rank Correlation Methods, 4th ed.,
Hafner, New York.
Nerlove, M. and Press, S.J. (1973), Univariate and Multivariate Log-Linear and Logistic Models. Rand Corporation
Report R-1306-EDA/NIH, Santa Monica, Cal.
- (1976), Multivariate Log-Linear Probability Models for the
Analysis of Qualitative Data. Discussion Paper No. I,
Center for Statistics and Probability, N o r t h w e s t e r n University, Evanston, Illinois.
Reynolds, H.T. (1977), The Analysis of Cross-Classifications.
The Free Press, New York.
Somers, R.H. (1962), A new asymmetric measure for ordinal
variables. Amer. Soc. Rev. 27, 799-811.
Srole, L., Langner, T.S., Michael, S.T., Opler, M.K. and
Rennie, T.A.C. (1962), Mental Health in the Metropolis:
The Midtown Manhattan Study, New York.
Upton, G.J.G. (1978), The Analysis of Cross Tabulated Data.
John Wiley and Sons, New York.
Wilson, J.P. (1974), Measures of association for bivariate
ordinal hypotheses, pp. 327-342 in H.M. Blalock (ed.) ;
Measurement in the Social Sciences: Theories and Strategies.
Aldine Publishing Company, Chicago.
-
-
-
-
-
-
-
107
Zusammenfassung
S. Kawasaki/K.F.
Zimmermann:
Das Messen von Beziehungen im
log-linearen Wahrscheinlichkeitsmodell durch einige kompakte
AssoziationsmaBe
Auf der Basis eines log-linearen hierarchischen Modells wird
das Kontingenztableau zweier q u a l i t a t i v e r Merkmale mit Hilfe
yon Assoziationsma~en ausgewertet:
Unter multinomialen V e r t e i l u n g s a n n a h m e n liefert die MaximumL i k e l i h o o d - S c h ~ t z m e t h o d e die Haupt- und bivariaten Interaktionseffekte. Diese i.a. unHbersichtliche Information wird
mit Hilfe bekannter AssoziationsmaBe verdichtet. Deren asymptotisches Verhalten wird an einem empirischen Datensatz demonstriert.
Summary
S. Kawasaki/K.F.
Zimmermann:
Measuring Relationships in the
Log-linear P r o b a b i l i t y Model by
Some Compact Measures of Association
On the basis of a log-linear hierarchical model, the contingency table of two qualitative variables is evaluated with
the help of some compact measures Of association. Under assumptions of multinomial distribution the maximum likelihood
estimation method provides the principal and bivariate effects
of interaction. The resulting information, unclear as it is in
general, is condensed with the help of well-known measures of
association. Their asymptotic behavior is shown with the help
of empirical data.
R&sum&
S. Kawasaki/K.F.
Zimmermann:
Mesurant des relations dans le
module log-lin~aire de probabilit& par quelques mesures compactes
d'association
Le tableau de contingence de deux caract~ristiques qualitatives
est exploit& ~ l'aide de mesures d'association sur la base d'un
module hi~rarchique log-lin~aire:
La m&thode d'estimation maximum likelihood fournit les effets
d'interactions principaux et bivariates sous des hypotheses
de distribution multinomiale. Cette information complexe en
g&n~ral est comprim~e ~ l'aide de mesures d'association connues, le caract~re asymptotique desquelles est montr~ basant
sur une serie de donn~es empiriques.
108
Pe3~oMe
So K,~wn:~nki / K.?. Zimm,,rm:~nn:
H 3 M e p e H H e O T H 0 m e H H ~ B flOF- ~ H H e ~ H O ~ B e p O H T H O C T H O ~ MOA e n H npH H O M O ~ H H e c K o ~ B K H X
K O M H & K T H N X Mep a C C O H H a ~ H H
Ha OCHOBe ~or-~HHe~HOfi H e p a p x H q e c K o ~ Mo~e~H O~eHHBaDTCH
np~ n O M O m ~ Mep a C C O U H a u H ~ T a O ~ H U ~ e o n p H K a c a H H ~ A B y X
KaqeCTBeHHNX HpH~HaKOB.
Cpe~H UOHHHOMHa~BHNX npeAnonomeHHH pacnpe~eneHHa BepoaTH O C T H N ~ M e T O A o ~ e H K H M a K C H M y M A a e T 0 C H O B a T e Z B H ~ e H ~HB a p M a T H N e H H T e p a K U H O H H N e p e 3 y ~ S T a T ~ . [)Ta B o 6 M e M H e H C H a H
H H ~ O p M a ~ H a H O H C H H e T C H HpH H O M O ~ H H 3 B e C T H N X M e p a c e o ~ H a HHH. A C H M H T O T H q e C K o e n o B e ~ e H H e Mep a C C 0 H H a H H H H 3 o O p a ~ s eTCH HpH HOMO~}i S M H H p H q e c K o ~ TeOpeMsI A a H H N X .
109