I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
'e
I
I
SIMULTANEOUS TEST PROCEDURES SOME THEORY OF MULTIPLE COMPARISONS
by
K. R. Gabriel
University of North Carolina
Institute of Statistics Mimeo Series No. 536
June 1967
Original version September 1963
First revision December 1964
This res,earch was supported in part by the National
Institut~;~of Health (Institute of General Medical
Sciences) Grant No. GM-12868-03.
DEPARTMENT OF BIOSTATISTICS
UNIVERSITY OF NORTH CAROLINA
Chapel Hill, N. C.
I
SIMULTANEOUS TEST PROCEDURES SOME THEORY OF MULTIPLE COMPARISONS*
I.
'I
by
K. R. Gabriel
I
I
University of North Carolina
and
Hebrew University, Jerusalem
June 1967
:1
Original version September 1963
First revision December 1964
~I
,I
1.
Introduction and Summary.
When a hypothesis is tested by a significance test and is not rejected?
it is generally agreed that all hypotheses implied by that hypothesis (its "com ponents") must also be considered as non-rejected.
However, when the hypothesis
is rejected the question remains as to which components may also be rejected.
I
~I
Various writers have given attention to this question and have proposed a variety
of multiple comparisons methods based either on tests of each one of the components
or on simultaneous confidence bounds on parametric functions related to the
various hypotheses.
'An approach to such methods, apparently originally due to Tukey [27J, is
to test each component hypothesis by comparing its statistic with the
critical value of the statistic for the overall hypothesis.
I
I
I·
I
~
level
This is called a
Simultaneous Test Procedure (STP for short) as all hypotheses may be tested simultaneously and without reference to one another.
An STP involves no stepwise
* This research was supported in part by the National Institutes of Health
(Institute of General Medical Sciences) Grant No. GM-12868-03.
I
-2-
testing of the kind employed by some other methods of multiple comparisons for
means, in which subsets are tested for equality only if they are contained in
sets which have already been found significant.
(See [3], [4], [10], [18]).
A general formalization of STPs is attempted in this paper.
Section 2
introduces the requisite concepts of families of hypotheses and the implication
relations between them, as well as the monotonicity relations between the related
statistics.
Section 3 defines STPs and shows conditions for coherence and con-
sonance of their decisions, these properties being that hypothesis implication
relations are preserved in the decisions of the STP.
Section 4 discusses com-
parison of various STPs for the same hypotheses and shows the advantages of the
Union-Intersection type of statistics and of reducing the family of hypotheses
tested as much as possible.
Section 5 translates all these results to simul-
.'I
I
I
I
I
I
I
taneous confidence statements after introducing the definitions necessary to allow
such translation.
The analogy between simultaneous test and confidence methods
I;
is of special importance as it brings a wide spectrum of methods within this
framework, most of which was originally formulated in confidence region terms.
This covers the original work of Tukey [27] and Scheff~[25] and continues with
that of Roy and his associates [24] and most recently Krishnaiah [12J,. [13].
A
general discussion of this confidence approach has been given by Aitchison [1]
since the first draft of the present paper.
ed out in section 5, it is not
In view of the close analogies point-
surprising that Aitchison arrives at the require-
ment of a constant critical value for all his tests, exactly as in an STP.
In
fact his approach and the present one are complementary.
The formal theory is illustrated with ANOVA examples to clarify the concepts.
No essentially new techniques are presented in this paper though this
approach has been used elsewhere by the author to derive a number of practically useful
11I
,
I:
1
I;
I
I.
I
-3 -
procedures [6], [7], [8].
2.
Families of hypotheses and statistics
II
I
I
I
I
I
I_
Let Y be a (vector) random variable having density function fe(Y) with
respect to a cr-finite
set w.
n
=
measure~,
where the parameter(s) are assumed to lie in a
An overall hypothesis w (cw) is being considered, as well as a family
o
{w./ iEI} of hypotheses w. implied by w, i. e., w c w. c w if
~
~
00- ~
an index set, not necessarily denumerable.
iEI - I being
(Note that w. is being used to denote
~
both a set in parameter space wand the hypothesis eEW. restricting the parameters
~
to it).
The hypothesis w itself is assumed to belong to the family and to form
o
the intersection of its members, that is, w En and w
o
0
= (lIw ..
~
The implication relation w. c w. for a pair of hypotheses from n will be
~ J
referred to by saying w. is a component of W.,
~
J
If the relation is strict,
w. c wj' w. will be said to be a proper component of W.,
~
~
J
i. e.,
Any such strict implica-
tion relation w. c w., where iEI and jEI, may be written i-< j and the set of all
I
I
I
I
I
I
I
I·
I
J
~
such relations existing between pairs of indices from I will be denoted
~.
Note
that i<j is a transitive relation, but is neither reflexive nor symmetric and that
o -c i for all iEI-{o}.
Also note that this is not the only type of implication
relation that may exist between members of a family of hypotheses.
~1'~2
wo :
w :
2
and
~l
~l
~3
are parameters one might consider the intersection hypothesis
= ~2 = ~3
= ~3
If, for example,
and let the family further contain hypotheses WI:
and w :
3
~2
= ~3'
In this example the relation set
f
~l = ~2'
contains only
the relations 0 -< 1, 00( 2, 00(3, whereas implications (wl(\w2nw3) c W or (wpw ) c: w
2
o
3
are not included in
10.
A partial ordering of the hypotheses of Q may be obtained by always assigning a higher rank to a hypothesis w. than to any of its proper components w., i"( j.
~
J
This will assign maximum rank to the intersection hypothesis wand minimum rank
o
I
-4-
to hypotheses which have no proper components in n.
n.
m1n
referred to as minimal and their subfamily denoted
that
n.
m1n
= (w./jEI
. ).
J
m1n
Le., n-n.
m1n
=
The latter hypotheses are
and indexed by I . so
m1n
All other hypotheses are referred to as non-minimal,
(w./iEI-I. ).
1
m1n
n
Corresponding to each hypothesis w. of
1
let there be a real valued sta-
tistic Z. = Z.(Y) and write the family of these statistics Z = {Z.jiEI}.
1
1
collection
N
(n,~)
1
The
of hypotheses and their corresponding statistics will be called
a testing family provided the distribution of Z., for every iEI, is completely
1
specified under w., i.e., the same for all 8EW..
1
1
If the family n is closed under
intersection the testing family will be called closed.
.n. =
I'"
,..
{wi iEI}, where I
specified under
wo =
If, for any subfamily
,.. is completely
£ I, the joint distribution of all Zi' iEI,
n~Iw., the testing family will be called joint.
1
These two
Consider a one-way ANOVA with w as the hypothesis of equality of
o
all k(>3) means and wi the hypothesis of equality of a pair of means indexed by i,
I~{o}indexing
(~)
hypotheses, only w
o
such pairs.
being non-minimal.
Note that these
(~)
wis are minimal
This family is not closed under inter-
section for it does not contain hypotheses on equality of subsets of h means if
2 <
h < k.
However, with F-ratio statistics (with a common overall estimate of
P
error variance) being used for each hypothesis, a joint testing family {n , ZF}
'"
results.
For the joint distribution of all the pairwise F-ratios is completely
specified for any subset of h (2 $
of the other k-h means.
h$~)
equal means, irrespective of the values
The same obviously also holds for augmented F-ratios,
i.e., F-ratios multiplied by their numerator d.f.s.
EXAMPLE 2. 2 .
I
I
I
I
I
I,
_I:
categories of testing families are neither exclusive nor exhaustive.
EXAMPLE 2.1.
.-I
In the same set-up let wi be a hypothesis of equality for some subset
II
I
I'
I;
J
,
I;
I
I;
I,
_1.
1
j.:
!
I
-5-
I.
I
I
k
S. of k. means, IS indexing all 2 _k_2 such subsets.
1
The minimal subfamily re-
-1
mains as in example 2.1.
If (augmented) F-ratio statistics are used for each of
these hypotheses, anyone of them corresponding to a true hypothesis will have a
central (augmented) F distribution irrespective of any means not involved in that
I
hypothesis.
I
I
EXAMPLE 2.3.
Thus one obtains a closed testing family {nS,zF}.
N
Again, in the same set-up, one may start by ranking the observations
of all samples together (overall ranking) and for each hypothesis use the "between"
sum of squares of these ranks as a non-parametric statistic.
-I
I
The overall sum of
squares is, apart from a constant, the Kruskal-Wallis statistic and its distribution is specified under w.
o
~(~)
But for any subset of
statistic depends not only on the equality of the
~
where the other k-h means are relative to those h.
siders that the ranks available for that subset of
have been assigned to the other k-h samples.
samples, the corresponding
means involved but also on
This is evident when one con-
~
samples depend on what ranks
Thus, such a collection of hypotheses
and rank statistics does not form a testing family.
EXAMPLE 2.4.
Exactly the same argument holds if instead of "between" sums of
squares of ranks one uses any other statistics based on the overall ranking, such
as the range of rank means or the maximum rank mean.
do not provide testing families.
These types of statistics
(The use of such statistics and their distri-
butions have been discussed by Nemenyi [17J).
EXAMPLE 2.5.
An alternative non-parametric approach to the set-up of example 2.2
is to use a separate ranking for the statistic of each hypothesis, including
only the samples relevant to that hypothesis.
One may then use the Kruskal-Wallis
statistic for each hypothesis, and as its distribution is specified under the
~
hypothesis a closed testing family is obtained.
I
-6 -
Since for each w. of
~
n more
than one statistic can be chosen to satisfy
these conditions, more than one testing family can be defined for any given
family of hypotheses.
Consideration of the properties of different Z.s
~
as test
statistics for their respective hypotheses w. will playa part in choosing a Z
~
appropriate to
given
n
n.
~
corresponding to any
is severely restricted by the following requirement of monotonicity.
{n,~}
will be called monotone if, whenever i< j, the
numerical relation
-
~
(2.1)
J
holds a.e. (*), between the corresponding statistics.
~
e.,
If, furthermore, for any
iEI-I . ,and for any sample point y, there exists a proper
m~n
I
1
I
I
_I
component w of wi' Le.,. i<j,. for which (2.1) is an equality, that is, i f
j
I
Z. (y) = max{Z. (y) i < j}
J
~
{n,~} will be called strictly monotone.
Z. (y) need not be the same for each y.
~
monotone it is also monotone.
(2.2)
~.,
Note that the actual j for which Z. (y) =
J
Clearly, if a testing class is strictly
Also, it is readily seen that if, and only if,
(2.2) holds for every iEI-I . , then, for every iEI-I .
m~n
Z. (y)
~
= max{Z J.(y) I i-< j,
m~n
jEI . }
m~n
(2.3)
a. e ..
Hence (2.3) is an alternative definition of strict monotonicity.
clear that in a monotone testing family the statistic for w '
o
It is also
the intersection
hypothesis, is
(2.4)
*
I
1
Z.(y»Z.(y)
non-minimal w., L
I
~
In the present discussion the choice of
A testing family
••I
Almost everywhere with respect to measure
~.
I
Ii
Ii,
I~
I
I
I
••
I
I
I.
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I
I
••
-7-
since Q has been defined to include w itself.
o
EXAMPLE 2.6.
~
In the testing family
{Q~ZF} of example 2.2 implication relations
N
exist between hypotheses on sets of means and hypotheses on proper subsets of
these sets.
Monotonicity holds since the "between" sum of squares, and hence
the augmented F-ratio, for any set is no less than that for any of its subsets
[ 6].
EXAMPLE 2.7.
If in the subset ANOVA set-up of examples 2.2 and 2.6, all sample
sizes were equal, studentized range statistics ZR might be used instead of (augmented) F-ratios.
In that case monotonicity would be
stric~
for the range of
all means always equals the range of some subset of the means.
EXAMPLE 2.8.
In the testing family of example 2.5 it may happen that the
Kruskal-Wallis statistic for a subset exceeds that for a set containing that subset.
For example, let there be three samples of four observations with the follow-
ing ranks:
sample 1 - 8,.9,10,11; sample 2 - 1,2,.6,7; sample 3 - 3,4,5,12.
The
K-W statistic for samples 1 and 2 is 5.33, exceeding that for samples 1, 2 and 3
which is 4.77.
EXAMPLE 2.9.
Hence that testing family is not monotone.
In a contingency table one may consider the family of independence
hypotheses for the entire table as well as for all subtables obtained by omitting or amalgamating rows and/or columns.
If Pearson's chi-square statistic is
used to test each hypothesis it may happen that the statistic for a table is less
than that for a sub-table, though independence in the former implies independence
I in
the latter [7J.
Hence such a testing family would not be monotone.
Monotonicity holds for all testing families using likelihood ratio or
union-intersection statistics, as is shown next .
I
-8--
LEMMA A:
If the statistics of a testing family are all likelihood ratio (LR for
short) the family is monotone (*) .
Let the family be (n,~} and wi' wj hypotheses of n such that i-< j.
Proof:
The
LR statistic for wi is
z.~ = d(supwi f!supWf)
where d is a monotone decreasing function, and the LR statistic Zj for wj is
similarly defined.
Now io<'j is equivalent to w.c:;w. so that sup f < sup f. Thus the ratio
w.
~
J
wi J
argument of d in Z. is no greater than that of d in Z .. As d is decreasing,
J
~
Z. > Z., as was to be proved.
~
-
J
EXAMPLE 2.10.
In the ANOVA testing family (nS,ZF}
example 2.2 the F-ratios are
N
LR statistics.
EXAMPLE 2.11.
In the contingency table independence testing set-up described
hood ratio statistics instead of Pearson's chi-squares [7J.
A testing family is strictly monotone if and only if its statistics
are related by Roy's Union-Intersection (UI for short) principle [20J.
Proof:
I
I
I
I
I
I
_I
Lemma A confirms the monotonicity noted in example 2.6.
in example 2.9 a monotone testing family may be obtained by using the log 1ike1i-
LEMMA B:
.-I
Roy used the UI principle to construct a test for w. from tests of all
~
its minimal components w., i< j, jE:I . .
J
m~n
If the latter are tested with critical
regions (y/Z.>O, Roy [9, 20J defined the critical region for w~, iE:I-I . , as
J
L
m~n
(*) Essentially equivalent Lemmas have been proved before by Aitchison [2J and
Knight [l1J.
I
I
I
I
I
I
I
••I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
-9-
u(.)
... I
J ~ -< J, JE
.
}(yjZ.
.
m~n
J
> 0, or equivalently as (yJmax(Z.ji-<
j, jEI . } > O.
.
J
m~n
J
Extending this definition to all possible values of
defined the UI statistic for w. as max(Z.ji-< j, jEI .}.
~
.
J
J
m~n
S,
Roy essentially
Use of this definition
for every iEI is equivalent to using the statistics (2.3) which satisfy strict
monotonicity, as was to be proved.
EXAMPLE 2.12.
For the ANOVA set-up of 2.7 one notes that the studentized range
ZR for any subset of means is the maximum of the student t statistics times ~2,
i.e., studentized ranges? for all pairwise comparisons of means from that subset.
Thus, the studentized range statistics generate UI relations for the testing
family (nS,ZR} as was expected by Lemma B from the fact that this family is
",
strictly monotone.
Since the UI relation is one of hypotheses and statistics it is preferable
not to speak of UI statistics, but of UI related testing families.
3.
Simultaneous Test Procedures
For a testing family (n,~} and a critical value
Sa
Simultaneous Test
Procedure (STP) is defined as the family of tests of all W.En which reject any
~
Wi' iEI, if Zi
>
S,
be denoted (n,~,s}.
using the same constant
S for
all Zi
of~.
Such an STP will
The probability
(3.1)
of falsely rejecting the intersection hypothesis is referred to as the level of
the STP.
If Z is a continuous variable the same STP may alternatively be deo
fined in terms of its testing family
being then determined by (3.1).
(n,z} and level
IV
~?
the critical value
S
I
-10-
The term simultaneous is used to indicate that no order or sequence is
imposed on the tests of all the hypotheses of
reference to one another.
n
and that they are made without
It is shown below that this cannot lead to incoherent
decisions.
Note that W is rejected by the STP if and only if Zo >
o
by a significance test of significance
level~.
hypothesis is therefore always part of the STP.
~,
exactly as
This test of the intersection
If the testing family contains
only a single hypothesis W and its statistic Zo' the STP
o
to the test of hypothesis w by means of statistic Z
o
0
{n,~,~}
is equivalent
at critical value
~.
An essential requirement that any procedure must satisfy is that no hypothesis should be "accepted" i f any hypothesis implied by it is rejected.
words, if i"" j then w. must always be rejected if w. is.
This requirement will
J
1
In other
be called coherence, (*) and an STP will be called coherent if its decisions always
satisfy this.
THEOREM 1:
{n,~}
Proof:
{n,~}
based on testing family {n,,} are coherent if and only if
All STPS
is monotone.
.-I
I
I
I
I
I
I
_I
I
I'
Consider any hypotheses w. and w. of n for which i-< j.
J
1
Monotonicityof
postulates that Zi > Zj ~ whereas coherence of {Q,~,~} for any given ~
~) :
requires that (y/Zi >
(Y/Zj >
t) ~(t)
since these events determine the
11
I,
j
.~
rejection of w. and W., respectively
1
J
Now, the statements "Z. > Z. a. e." and "(yj Z. > 1')
1
all
S"
are clearly equivalent.
to coherence of all STPs
-
J --
1
~
:>
-
(yj Z. > 1') a. e. for
J
~
Hence monotonicity of testing family is equivalent
based on that family, as was to be proved.
(*) In an earlier paper [6J this properly was called transitivity.
The terminology
has been changed because of the different connotations of the earlier term.
(t)
For set containment relations AcB and Ac B the a.e. qualifications means that
AnD may be non-empty but of measure zer~. Similarly, A=B a. e. means both Af'\ B
and .A ('\ B may be non-empty but are of measure zero.
-
I'
I
1
••
I
I
-11-
I.
I
I
I
I
I
I
I
COROLLARY 1:
If the statistics of a testing family are all LR, all STP s based
on that family are coherent.
Proof:
This follows from Lemma A and Theorem 1.
EXAMPLE 3.1.
In the subset ANOVA set-up
{nS,~F} of examples 2.2, 2.6, 2.10 one
may test equality of the h means of each subset by means of the (augmented)
ratio against sF, the upper
F~
~ point of the (augmented) F distribution with k-1
and n degrees of freedom, where n is the d.f. for the variance estimate.
-e
-e
This
will give an STP with coherent decisions; set S. of k. means will be rejected if
~
-~
any subset of its means is rejected [6J.
EXAMPLE 3.2.
The non-parametric analogue of the above, using the Kruska1-Wa11is
statistics of example 2.5, is not an STP since its testing family is not monotone.
As was pointed out in example 2.8, the Kruska1-Wa11is statistic for a set may be
I
I
I
I
I
I
I
I·
I
less than the same statistic for a subset contained in the set.
If the common
critical value happens to be between these two statistics I values the subset would
be rejected whilst the set was accepted.
This would lead to the incoherence of
asserting that all means of a set are equal whilst some of them differ from each
other.
Hence, such a procedure would be incoherent.
(See further remarks in
Section 6, below).
EXAMPLE 3.3.
and write
C!~
-~
In the one-way ANOVA set-up denote the expectations
o:
~1
(~1'~2"
"'~k)
C!€V(Ct) for a contrast in the expectations where V(Ct) is the
-~
k-vector space orthogonal to the vector 1 t
W
~I =
= 11 2 = ... = 11 k
and wj :
contrasts such that c! € V(Ct).
-J
=
(1,1, ... ,1).
c~~ = ~
Now consider
c. 11 = 0, j € IC_{o} ranging over all
h=l J h h
Clearly, w = n~. as nullity of all contrasts
o
IL; J
-J-
is equivalent to equality of all expectations.
I
-12-
Let the sample means be written ~
S2
and the sample sizes E1'~2"",Ek'
, = (x1,x2,
- - ... ,xk~
-
the variance estimate
Then LR statistics are, for wo' the augment-
ed F-ratio
I·
and for wj ' j
I
I
C
€
I-{o}, the squared Student t
1-
It is readily seen that this provides a joint testing family
{nC,~F}.
An ~ level STP is obtained by choosing ~F as in example 3.1.
This STP
consists of testing the over all wand each contrast w. by checking if ZF > ~F
o
and
Z~ > ~F,
J
1.
By Corollary 1 this STP {n C,~F,
respectively.
0
~F}
is known to be
coherent in the sense that rejection of any contrast hypothesis always entails
rejection of the over-all equality hypothesis.
It will be noted that this STP consists of the significance tests of
Scheffe's method (example 5.1 [25], [6]).
In a coherent STP the level
~
may be regarded as an experimentwise level,
as follows from the next Theorem.
THEOREM 2:
The probability that a coherent STP {n,@,t} of level
one true w. of n is
1.
.-I
~
if w is true; it is at most
0
~
~
rejects at least
irrespective of the truth of
w provided {n,Z} is either closed or joint.
o
The probability of rejecting any particular true wi of n is no more than
the above probability.
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
-13 -
Proof:
By Theorem 1, (n,~} must be monotone, so that by (2.4) Zo = max{Zi1iEI}.
Hence rejection of w , event (Z
o
Under w
>
0
~),
is equivalent to rejection of any W.,
1.
the probability of the former event
o
is~,
(3.1),
hence so is the probability of the latter, proving the first statement.
.... =
Next, let n
intersection
I......
=I,
{wi iE I}, I
wo = n-I w.
1.
be the subfamily of true hypotheses whose
is necessarily also true.
occurs whenever max{Z./iEI} >
1.
~
Then rejection of a true w.
1.
and the required probability is P... (max{Z.jiEI} >
w
1.
o
If it can be shown that
...
P", (max{Z./ iEI} >~)
W
1.
o
=
P (max{Z.liEI} > ~)
w
1.
(3.2)
o
then the second statement of the Theorem follows, for the latter probability is
(max{Z./iEI} >~) = P
clearly no greater than P
W
If {n,z}
is closed
rv
W
1.
o
W
En,
0
(Z
0
0
>~) = ~.
as n is closed under intersection.
max{z.liEi}, the statistic for
w,
wo'
holds or not, and so (3.2) holds. (*)
1.
irrespective of whether
W
o
0
Hence
has its distribution completely specified under
If {n,~} is joint, the joint distribution of all Z., iEI, is specified
1.
under
w,o
irrespective of the truth of w.
0
Hence the same holds also for
max{Z.liEI} and (3.2) follows as before.
1.
The last statement in the Theorem is obvious.
Coherence prevents the contradiction of rejecting a hypothesis without
also rejecting all other hypotheses implying it.
It does not, however, preclude
the dissonance of rejecting a hypothesis whilst not rejecting any other hypotheses
implied by it.
(*)
Such dissonances, though not desirable, are sometimes allowed when
A similar proof for one-way ANOVA was given in [6] and a general proof for
closed testing families in [11] by Knight. As a result of the requirement
of closure under intersection Knight's form of the Theorem cannot be applied
directly to some important classes of procedures, as, for instance, Scheff{' s
(example 3.3, above). Knight surmounts this difficulty by applying his Lemma
to an extended family which includes all intersections of sets from n.
V·
I
.I
-14-
the alternative to rejection of a hypothesis is taken to be non-rejection rather
than acceptance.
They are familiar to users of tests of significance, as these
merely reject a hypothesis without indicating where it fails.
often when STPs
jected.
They occur less
are used, as these may indicate some components to be also re-
However, the use of STPs
does not eliminate all dissonances:
some STPs
may reject hypotheses without rejecting any of their proper components - such
STPs
will be called non-consonant.
are coherent as well as consonant.
One would generally prefer procedures which
In a later section non-consonant procedures
will be compared in terms of the likelihood of avoiding dissonances.
Note that consonance of an STP can equivalently be defined as the requirement that any wi of n be rejected only if some minimal component of wi is rejected.
In the subset ANOVA F-ratio
EXAMPLE 3.4.
hypotheses W.:
J
~. = ~J'
Ja,
ZF
o
t
j3
jEI~in'
STP(nS,~F~F} of example 3.1 the minimal
are those of pairwise equality of means.
Since
=
and
a. e.
> ")
for any JE
. IS
~
min'
I
I
I
I
I
I
I
_I
I
I
I
I"
and
d oes not necessar il y 1mp
. 1y (ZF.J
( see [7J) , (Z F0 > ")
~
e
In
0
th er
words, dissonances may occur in rejecting over-all equality of means without
I~
I~
rejecting the equality of any pair of means [6J.
EXAMPLE 3.5.
The contrast ANOVA F-ratio STP(n C,~F, ~F} of example 3.3 is consonant
as well as coherent.
To see
thi~note
that the maximal squared t statistic for a
I
-.i
'I
I
-15-
I.
I
I
I
I
I
I
I
Ie
I
I
I
I
I
contrast is equal to the over-all augmented F-ratio T i.e., ZF
o
= max{Z~lj€IC,
}
J
m1n
where I C, = IC_{o}, (This is established by differentiating the Z~ expression
m1.n
J
in example 3.3 over all c, and setting the derivative equal to zero [6J.
See
-J
Hence, (Z F > ~ F) = UIC (Z.F > ~ F ~ so that whenever over all w is
o
min J
0
rejected so is w for some contrast.
j
also [26J).
EXAMPLE 3.6.
{QP,~F,~F} would consist
In the pairwise ANOVA set-up the F-ratio STP
.
of tests be10ng1.ng to both
SFF
CFF
of example 3.1 and {n ,~,S } of example 3.3,
{Q,~,S j
P
S
since the overall w is the same in n , nand n
C
o
and the
othe~
that is; those on pairwise comparisons, are included in both n
THEOREM 3:
All STPS
S
P
hypotheses of n ,
and in n
C
based on testing family {Q,Z} are coherent and consonant
if and only if {n,Z} is strictly monotone.
Proof:
STP
{n,~,~}
is coherent and consonant, by definition, if, for every i€I,
= U{'..i,'
. I
} (y Iz, > 0
1. J,. J€ min
J
For coherence requires the r.h.s. event to imply the
requires that the 1.h.s. imply the r.h.s ..
All STPs
and consonant if, and only if, (3.3) holds for all
~,
(3.3)
a. e.
1.h.s~
whereas consonance
based on
{Q,~}
are coherent
but this is equivalent to
(2.3), the definition of strict monotonicity of that testing family.
COROLLARY 2:
If, and only if,. a testing family is UI related, all STPs
based
on that family are coherent and consonant .
•
Proof:
This follows from Lemma B and Theorem 3.
Corollary 2 points out the correspondence between UI statistics and con-
~
sonant STPs.
Roy has used the UI principle to induce a test of the intersection
I
-16-
hypothesis from the tests of its components, whereas its equivalent is used here
to resolve the test decision on the intersection hypothesis into decisions on the
components.
The UI test of
can be used as part of an STP.
W
o
.-I
Rejection by this
test is followed in the STP by further tests to resolve which components are to
be rejected, whereas non-rejection obviates the need for further testing.
EXAMPLE 3.7.
In the subset ANOVA set-up for equal sample sizes E1
say,. one might use studentized range statistics as in example 2.7.
= E2 = ... = Ek = ~
Thus, hypo-
= 11 i2 = ... = lli . of equality of a11!i means of subset Si' say,
i1
k
would be tested with statistic
~
z~ = max{jx. - x. I .fn/sl i =f iQES-r}'
thesis wi:
11
~
~~
~13
~
I-'.L
Taking sR as the upper ~ point of the studentized range distribution for k variables
with n
e
d.f., and rejecting any wi if
(Z~ > sR~ one obtains an STP {nS,~R,sR}.
As
pointed out in example 2.12 this testing family {nS,~R} is UI related and hence
By Corollary 2 {nS,~R, sR} is coherent and consonant.
is strictly monotone.
EXAMPLE 3.8.
If the decisions of the STP of example 3.7 were restricted to the
overall hypothesis and all pairwise comparisons (as in example 3.6) this would
not effect the coherence and consonance properties and one would obtain an STP
{ H("\P,~R,!,R}
~
..
cons~st~ng
0
f t1-.l~e
.
'f'~cance tests
s~gn~
0f
Tuk eyts met h 0 d
0
f a 11 owances
[ 27].
Coherence and consonance are relations between test decisions on hypotheses which imply one another, Le., between some w. and w, of n for which i-<j.
J
~
As was noticed in section 2, above, other implication relations may also obtain
within the family n.
The STP properties of coherence and consonance do not relate
to such other relations and indeed STP decisions may violate them.
Thus, in a
family of hypotheses on equality of sets of parameters from among 11 ,11 , ... ,11 ,
1 2
k
an STP may accept ]11
I
I
I
I
I
I
= 11 2
as well as ]11
= 113
but reject 11
2
= 113
even though it
I
I
I
I
I
I
I
••
I
I.
I
I
I
I
I
I
I
-17-
is implied jointly by the two accepted hypotheses.
Lehman [14, 15J has considered
procedures which preserve a wider class of implication relations and has termed
them compatible.
STPS, even consonant ones, are not generally compatible in that
sense.
4.
Resolution of STPS
The purpose of multiple comparisons in general, and STPs
in particular,
is to provide resolution of significant test decisions on overall hypotheses into
significant decisions on components, as far down as minimal components.
Thus,
for family n = {w.!ieI} rejection of intersection w is to be resolved into re1
.
0
jection of proper componentsw., ieI, down to minimal hypotheses w., jeI . .
1
J
m1n
The extent of resolution of a procedure must be defined in terms of the
likelihood of rejecting minimal proper components provided the overall hypothesis
is rejected.
If two procedures each test overall hypothesis w at level
o
~,
the
one will be said to be no less resolvent than the other if it always rejects any
I
I
I
I
I
I
I
Ie
I
minimal hypothesis rejected by the other, and will be said to be strictly more
resolvent if it sometimes rejects when the other does not.
sider two STPs {n,~,~} and {n*,~*,~*}, such that Wo
n = {w.1
. c 1*.
1 ieI} and n* = {w.j
1 ieI*} and I m1n
In other words, con-
= nIWi = nI*w i ;
where
Then the former STP is no less
resolvent than the latter if
=
w (z*0 >
P
~*)
(4.1a)
o
and for all jeI .
m1n
(z.
J
> ")
:;,
~
-
(z~
J
>
t*)
~
a. e.
It is strictly more resolvent of the containment in (4.1b) is proper.
(4.1b)
It is
I
-18-
clear that no meaningful comparisons can be made between STPs unless they test
the same hypotheses, at least to the extent that their intersections are the same
and the minimal hypotheses of one of the families are all included in the other
family.
EXAMPLE 4.1.
In the pairwise ANOVA set-up with equal sample sizes the studentized
P R R
range STP {n,~,~ } (Tukey's method-example 3.8) will reject any pairwise comparison
of means which is rejected by the F-ratio STP {nC,~F,~F} (Scheffe1s method-example
3.5) provided both STPs are of the same level [9J.
Note that the family of hypo-
theses of the latter STP is wider than that of the former as it also includes all
contrasts, but that all hypotheses of the former are included in the latter
family.
Resolution comparisons are made between STPs of the same level.
An
The procedure will be said to be no less
(more) parsimonious if it gives these decisions at no more (less) cost in rejection
of other, non-minimal, hypotheses, and specifically of the overall hypothesis.
Thus {n,~,~} is no less parsimonious than {n*,~*,~, where n
for all jeI.
m1.n
for all ieI
and for w
o
(Zj > ~)
(Z. >
1.
I
I
I
I
I
I
_I
alternative kind of comparison can be made between STPs which give identical
decisions on all minimal components.
.-I
~)
G
-
=
~.,
(Z*J' > !:*)
(Z": >
1.
~*)
=n* and
_a.
e.,
P (Z >~) < P (Z* >
w
o w0 0
o
~*).
W
o
= w~,
if
(4.2a)
(4.2b)
(4.2c)
If the inequality in (4.2c) is strict,{n,~,~} is strictly more parsimonious than
I
I
I
I
I
II
••
I
I
I.
I
I
I
I
I
I
I
-19-
A possible measure of resolution is the ratio of the rejection probabilities of minimal hypotheses to that of the overall hypothesis.
overall hypothesis w , if all minimal hypotheses have the same probability of
o
being rejected, i.e., if
0, = P (Z. >
1
w J
s>
j
for all j€I . , resolution may be gauged by the ratio aI/a'
m~n
Clearly, if one
STP is more resolvent than another, the former will have a higher 0,1/0, ratio
than the latter.
However, a higher aI/a ratio merely means more probable resolu-
tion, not necessarily strictly more resolution in the sense of (4.lb).
EXAMPLE 4.2
-e
Under the
Consider an eight mean ANOVA with 40 d.f. for error [6].
level studentized range STP {n
whereas the 101. F-ratio STP {n
S
,!F,4. 10}h.as
S
F
,~,
The 101.
a resolution ratio of 0,/0,=0.00'/0.100=0.06
13.09},noted in example 4.1 to be less resolvent,
has a ratio of only aI/a? 0. 001/0. 100=0.01.
A more general measure of resolution should also take into account the
chances of rejecting non-minimal component hypotheses.
If ranks can be assigned
meaningfully to hypotheses, resolution should depend on the entire sequence
0,1: 0,2: ... : o,k -1 = 0,
where 0,
q
is the probability of falsely rejecting a hypothesis of rank S and
is the maximum rank (as in a
hypothesis.
~
~-l
mean ANOVA), i.e., the rank of the intersection
Examples of such sequences of probabilities are given in [6] for
ANOVA STPs and in [7] for contingency table STPs.
To determine which STPs are more resolvent or more parsimonious than
which others, one is led to consider the following relation between monotone testing families.
W
o
=
w~
{n,~}
will be said to be narrower than {n*,~*} if
and if there exists a function g such that
n
=n* and
I
-20-
g is continuous monotone increasing,
Z~
for every jE I .
m~n
J
=
(4.3a)
~.,
g(Z.)
J
(4.3b)
(4.3c)
for every iEI-I .
m~n
and
p(Z* > g(Z
o
0
»
o.
>
(4.3d)
A stronger condition which implies (4.3d) is
Z* > g(Z )
o
(4.3d')
a. e.
0
If (4.3d') holds {n,~} will be said to be strictly narrower than {n*,~*}.
EXAMPLE 4.3.
Again, in the subset ANOVA set-up for equally sized samples, compare
the studentized range testing family {nS,~R} with the augmented F-ratio testing
family
S
{nS,~F}. The hypothesis family n and intersection wo : ~l = ~2 =... = ~k
S
Any minimal hypothesis JEI .
are the same for both.
of expectations w.:
J
is a pairwise comparison
m~n
~.
Ja,
-
~.
J(3
= 0, for which
Z~ = (Z~)2/2 (example 2.12).
J
J
Since
R
Z can assume only positive values the function ZF = (ZR)2/2 satisfies (4.3a)
and (4.3b).
Also, it has been shown ([9J, see also example 4.10, below) that
for all non-minimal hypotheses w. on k.(>2) expectations
~
that (4.3c) and (4.3d') also hold.
-~
This shows
Z~~ > (Z~)2/2
~., so
~
{nS,~R} to be narrower than
{nS,?:.F}.
EXAMPLE 4.4.
For contrast hypotheses w.:
the studentized range statistic as
J
c~~
-J-
=
k
~ c'h~h
h=l J
=0
.-I
Tukey has generalized
I
I
I
I
I
I
_I
I
I
I
I'
I
I
I.
••
I
I
I.
I
I
I
I
I
I
I
I_
I
I
•
-21-
and shown that ZR
o
= max{Z~lc.€v(ct)}
J -J
[28J section 3.6, [16J pp. 44-46).
...
Hence {oC,ZR} is UI related, just as
is (example 3.5 and Corollary 2).
Thus, in the contrast ANOVA set-up for equally sized samples a generalized
studentized range STP {OC,~R,~R} is an alternative to the (augmented) F-ratio
C F F
STP {O ,Z
.... ,~ } of example 3.5 .
Let g(x)
=
x 2 /2 for x > O.
This is a function which satisfies (4.3a),
(4.3b) for all pairwise contrasts and (4.3c) and (4.3d) for the non-minimal
However, for minimal hypotheses on
hypothesis w :
o
contrasts involving more than two expectations (4.3b) does not hold.
Hence
{OC,ZR}
is not narrower than {nc,ZF}, nor vice versa.
N
~
Also compare testing families {OP,!R} of example 2.1 and {nC,~F}
EXAMPLE 4.5.
of example 3.3.
P
C
Clearly, 0 c: 0 and
W
o
is the same for both.
Furthermore, as
P
in example 4.3, for all j€I .
m~n
and
~.,
W
o
being the only non-minimal hypothesis of 0
P
C
(and 0).
Hence (4.3a,b,c,d)
hold so that {OP,ZR} is found to be narrower than {OC,ZF}.
~
~
Given the narrowness relation between testing families the following two
theorems show the corresponding relations between STPs based on these families.
THEOREM 4:
If
(n,~}
is narrower than
(O*,~*}t
such that {O,@,t} is no less parsimonious than
then for every t* there exists t
{O*,~*,t*}.
Moreover, the former
STP is strictly more parsimonious than the latter for some values of t*t i.e.,
those for which
I
-22-
.1
Pw (Z*0 > t>*
> g(Z 0 )) > 0,
~
-
(4.4)
o
where g satisfies (4.3a,b,c,d).
Proof:
In view of narrowness conditions, (4.3a,b,c,d) hold for some function g.
For any given s* choose
S so
for all jEI.
m~n
that s*
=
g(~).
(Y/Z*J.(Y) > ~*)
=
Then it follows from (4.3b) that
(yjZJ.(Y) > s)
~.,
and from (4.3c) that
~.,
establishing (4.2a) and (4.2b).
Pw (Z*0 > s*)
o
from which (4.2c) follows
From (4.3c) one further obtains
(Z > s) + P
= Pw
ow
o
(Z* >
0
0
~*
that (4.2c) is a strict inequality -- more parsimony
this holds at least for some values of
s*
_I
>
g(Z ))
0
and no lesser parsimony is proved.
It also follows
if (4.4) holds.
That
is ensured by (4.3d), and the theorem
is proved.
EXAMPLE 4. 6.
In comparing the subset ANOVA studentized range STP
narrowness established in example 4.3 it follows from Theorem 4 that the pairwise decisions of both STPs are identical but for all hypotheses on three or more
means the F-ratio STP will reject whenever the studentized ratio STP does and more
often.
In particular, this will result in a higher level for the F-ratio STP
-- see Table 1 in [6J.
I
I
I
I
I
I
I
I
I
I
I
I
I
I
·1
I
I
I.
I
I
I,
I
I
I
I
Ie
I
I
I
I
I
I
I
I·
I
-23-
THEOREM 5:
If
{n,~}
is narrower than
{n*,~*}
and if
(n,~,t}
and {n*,z*,t*} are
of the same level, then the former STP is no less resolvent than the latter.
Moreover, it is strictly more resolvent for some values of
t,
i.e., those for
which
P (Z~ > g(~) ~ g(Zo»
wo
> 0,
(4.5)
where g satisfies (4.3a,b,c,d).
Proof:
all
According to the proof of Theorem 4 P
W
S with
o
strict inequality if (4.5) holds.
(Z* >
0
g(~»
>
P (Z
W
0
>~)
for
o
Hence, if the levels of the two
STPs are to be equal, i.e., if P (Z* > t*) = P (Z > t) (condition (4.la»,
.
w
o w0 0
o
one must have t* > g(t), with strict inequality conditional on (4.5). Now,
minimal hypotheses w., jEI . , will be rejected by (n*,~*,t*} when (Z*J' > t*).
J
m~n
.But, in view of (4.3a)
(Z.
J
the containment being proper if (4.5) holds.
(n,~,t}
and the resolvence of
holds.
>
V,
Thus (4.lb) is established generally
shown to be always no less, and strictly whenever (4.5)
The latter occurs for at least some values of
t,
as argued in Theorem 4
for (4.4).
EXAMPLE 4.8.
R}
Again, comparing the pairwise ANOVA studentized range STP {nP,zR,t
IV
with the contrast ANOVA F-ratio STP {n
to ensure equal levels.
C F F
,~
,t }, the critical values may be chosen
In view of the narrowness comparison of the testing
families in example 4.5, it follows from Theorem 5 that (nP,~R,tR} is more resolvent than (n
C
F
,~,t
F
},
as stated in example 4.1.
I
-24-
EXAMPLE 4.9.
Numerical calculations have shown ([25J, [26J ch. 3) that neither
Tukeyts nor Scheff~ts method is more resolvent for all contrasts (i.e., narrower
confidence bounds - see section 5, below).
This accords with the finding in
example 4.4 that neither of the corresponding testing families is narrower than
the other.
.'I
I
,I
It is evident from these Theorems that narrower testing families have
the advantages of providing more
and more
(~)
~eorem4).
('t)
resolvent STPs of the same level (Theorem 5)
parsimonious STPs for the same decisions on minimal hypotheses
In practical applications many multiple comparisons techniques have
been criticised precisely for insufficient resolution or excessively frequent
overall rejections for given minimal decisions.
Hence, the use of narrower test-
ing families may help to provide more practically useful techniques.
I
I
I
The next Lemma gives a sufficient condition for one testing family to be
narrower than another.
In the following Corollary this condition is applied to
show when some STPs will be more resolvent than others.
LEMMA C:
Let
{n'M}
be a strictly monotone testing family and
tone testing family such that
-
n en*.- n.
m~n
condition for {n,~} to be narrower than
c
-
n*.
m~n
{n*'M*}
and w
--- 0
{n*,~*}
= w*.
0
be a mono-
A sufficient
is the existence of a function g
satisfying (4.3a,b) and also
P
W
(Z*
0
o
T g(Z 0 »
> o.
(4.3e)
If (4.3e) is replaced by
z*o
::I:. g(Z )
7
0
(4.3e t )
one obtains a sufficient condition for strict narrowness.
(t)
"Providing more resolvent and more parsimonious STPs" must be understood as in
Theorems 4 and 5 as strictly more for some critical values and no less for all
other critical values.
I
I
I
I
I
I
I
.,
I
,
I
-25-
I.
Proof:
I
(4.3e') implies (4.3d ) and thus strict narrowness.
'
I
I
I
I
I
If (4.3c) holds,
(4.3e) implies (4.3d) and thus narrowness, whereas
establish (4.3c).
Now, for any iEI-I . , strict monotonicity of {n,z}
gives, by (2.3), that
N
~n
Z.~ = max{Z.li
-< j, jEIm~n
. }
J
Z~ > max(Z~li
-
~
J
-< j, jEI*. }
,.
I
a. e ..
m~n
Under the conditions of this Lemma
-<
m
.. ax(Z*j j i
j, j€I*. } > max(Z~ Ii ..( j, jE I . }
m~n
J
m~n
= max(g(Z.)ji
J
=
I
I
I
~.,
whereas monotonicity of {n*,z*} requires, by (2.1) that
I
Ie
I
.1
I
I
It therefore suffices to
g(max(Z.1 i
J
~ j, jEI . }
-<
m~n
j, jEI . }),
m~n
so that, for all iE I-Im~n
. ,
Z~
~
which is condition
EXAMPLE 4.10.
->
g(Z.)
~
a. e.,
(4.3c).
In the subset ANOVA set-up for equal sample sizes the studentized
range testing family {nS,zF}
,.. was noted (example 2.7) to be strictly monotone and
the augmented F-ratio testing family (nS,zF}
.... to be monotone (example 2.6).
The
family of hypotheses is the same in both cases and the monotonic increasing relation
Z~ = (Z~)2/2 holds for all minimal (pairwise) hypotheses (example 4.3).
J
J
other hypothesis w. on k.(>2) means, the statistic
~
~
For any
I
.-I
-26-
k.
~
-
• 2/ s 2
~ n. (X. -X.)
~h ~h ~
h=l
where
k.
k.
~
~
xi
h=l
~
n.
~h
=
~
n. x.
~h ~h
h=l
,
and the statistic
(Z~)2/2 = max{/x i -xi l.[n/s.[2/i ,iA€S.}
~
a
~
a l-' ~
are equal only if k.-2 of the means x.
-~
~h
are exactly equal to the average of the
Hence Z~ = (Z~)2/2 a. e. ,
other 2 means, clearly an event of probability zero.
~
and all the conditions of Lemma C are seen to obtain.
-
~
Thus the unproven state-
ment in example 4.3 is established and the proof that {nS,!R} is narrower than
{nS, ~F} is complete.
COROLLARY 3:
rt.
m~n
Let {n,Z} and
{n*,~*}
be monotone testing families such that rtc.rt*,
Then each of the following are sufficient conditions that {rtz~} provide more
parsimonious and more resolvent STPs than {rt*,~*},
(II)
i f {rt, ~} is UI related and g also satisfies (4.3e) or (4.3e );
'
i f both {rt, k} and {rt*z#*} are UI related and
P (max{Z~1 j€I*. } > max{Z'l:/ j€I . } > o.
w
J
m~n
J
m~n
.
(4.6)
o
Proof:
By Lemma B, both (I) and (II) require
{n,~}
to be strictly monotone.
Hence, by Lemma C, condition (I) is sufficient for
narrower than {n*,z*}.
""
{n,~}
to be
The rest follows from Theorems 4 and 5.
Under (II), both testing families are strictly monotone so that,
Z* = max{Z'l:/j€I*. } and Z = max{Zjlj€I .}.
o
J
m~n
0
m~n
Now by (4.3a,b)
_I
I
crt*. and w = w* and let there exist a function g satisfying (4.3a,b).
- m~n - - 0
0
-
(I)
I
I
I
I
I
I
~.,
I
I
I
I
I
I
••
I
I
I.
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I
I
,.
I
-27-
g(Z ) = max{g(Z.)!jEI . } = max{Z*jljEI i }, so that (4.6) of (II) is equivalent
o
J
mln
mn
to (4.3d) and this implies (4.3e).
The
rest of the argument is as for (I),
above.
EXAMPLE 4.11.
The conclusions of example 4.6 regarding equal sample size ANOVA
STPs for subsets would ·a1so follow from Corollary 3(1) upon noting the existence
of a g function satisfying (4.3a,b,e) as in example 4.10, and the fact that the
studentized range testing family is UI related (example 2.12).
EXAMPLE 4.12.
Similarly, the conclusions of example 4.8 regarding pairwise and
contrastwise STPs follow from Corollary 3(11).
Note first, by example 4.5, that
a function x 2 /2 exists which .satisfies (4.3a,.b).
and {n
C F
,~
Next, note that both {n
} are UI related (examples 3.8 and 3.5, respectively).
P
'rwZR}
Finally, con-
dition II is established since (4.6) holds, i.e.,
~.,
for otherwise a number of contrasts in means ~ have to be zero, an event of
probability zero.
EXAMPLE 4. 13 .
where w :
w. :
o
J
k
wo
. th
.
1.S
e 'lntersect1.on
0
f n,
D an d
w.,
J
=
nw.
j=2 J
. 2 , ... , k are mlnlma.
. . 1
J=
be the absolute value of Student's t for
j=2, ... , k.
w.,
J
so that
Z~J
= +
Let ZD., J=
. 2, ... , k,
J
JZ~J =
(Z~)/.[2
J
D
Further, let Z~ = max{z~ I j=2, ... , k} so that {n , ~D} is UI related.
An ~ level STP {nD,~D,sD} may be obtained by choosing sD from the tables provided by Dunnett [5].
I
-28-
.-I
To compare Dunnettts technique with Tukeyts (example 3.8) note that
~D
H
C
D. c:
nP, nml.n
~p.
'b
ml.n
and
W
o
is the intersection of both.
The two testing
families are UI re1ated,so it remains only to check whether (4.6) holds.
is so since the event
with positive
This
D
max{Z~1
jEl. } > max{ ,[2Z / j=2, ... , k} may obviously occur
J
ml.n
j
probabi1~ty
(but less than one) under w .
o
It then follows from Corollary 3(II) that if ~D =J2~R the STP {nD,~D, ~D}
will have identical decision on w '
j
more parsimonious.
j=2, ... ,k, as STP
{nP,~R,~R}, but it will be
On the other hand, if ~D and ~R are chosen so that both STPs
are of the same level, the former will be more resolvent than the latter.
An analogous result was proved by Krishnaiah for the related confidence
statements ([13J and example 5.9, below).
Corollary 3 allows parsimony and resolution comparisons of different
STPs
with equivalent statistics for minimal hypotheses.
I
I
I
I
I
I
Condition I of the
Corollary may be used to compare STPs testing the same family of hypotheses (as
in example 4.11) in which case it shows those with UI related testing classes to
be preferable to any others.
Condition II of the Corollary maybe used to compare
STPs all of which have UI related testing classes.
(examples 4.12" 4.13).
In
that case the most preferable STP is shown to be the one whose'subfami1y of minimal hypotheses is contained in that of the other STPs.
This is so unless the
the containment is trivial in the sense that the maximum of the STPs
statistics
for the contained subfamily equals that for a cointaining subfamily with probabi1ity one.
The importance of using the UI principle with as restricted a family
of hypotheses as possible was already stressed by Roy and Srivastava [24J.
Krishnaiah has demonstrated a confidence bound analogue, of condition II (see
Coro1lary3 t (II) below) but has not pointed out the need for non triviality of
containment ([13J, Theorem 6.4).
I
I
I
I
I
I
I
••
I
I
I.
I
I
I
I
I
I
I
Ie
I
I
.1
I
I
I
I
,.e
I
-29-
5.
Simultaneous confidence statements
If the hypotheses of a family relate to parametric functions, simultaneous
confidence statements may be made regarding the values of all those functions.
The correspondence between such statements for certain functions and STPS
for
the hypotheses regarding particular values of these functions allows the extension
of the foregoing properties of STPs
statements.
to corresponding properties of these confidence
A number of additional definitions will be introduced to clarify this
correspondence.
Consider a family
being denoted
that A.EA..
~
.L\.
= (~.liEI)
¢
~
of parametric functions> the range of
~.(8)
A typical value of
$.
~
will be denoted A.> it being understood
~
~
Then
~
Wi(A.)
~
= (el~.(8) = A.}
~
(5.1)
~
is a hypothesis regarding the value of ~.(8), and the class (wi(A.)jA.EA.} of such
~
~
~
~
hypotheses forms a disjoint partition of the parameter space w.
For each value
A.,
1
let Z.(Y;A.) be a statistic whose distribution is the
~
~
same for all 8EW.(A.)
and is independent of the value of A..
1
~
~
collection of all such statistics with A.EA., iEl.
~
Write
Z for
N
the
The collections of parametric
~
functions ~.1 and corresponding statistics Z.(Y;A.)
will be denoted [¢,~J
and
~
1
.referred to as an estimation family.
For any given
probability~.
1
let
t be such
that
P
W.
~
(>.) (Z . (Y; A.)
1\.
~
~
~
<0
-
= P8 (Z . (Y; ~ . (8» < 0 = 1-0.1-•,
~
~
( 5 •2 )
-
this probability clearly being independent of the value of A. or 8.
1
Then the sub-
set of A.
~
(5.3)
I
-30-
l-~..
is a confidence region for $.(e) with co-efficient
~
~
The confidence statement
$.(e)EA.(Y;S) is equivalent to the statement eE~.(Y;S), with
~
~
~
~.(Y;s)
~
being a set in w.
=
-<
(e/z.(Y;$.(e»
.
~
~
(5.4)
S}
Either of these statements is understood in the sense of
acceptance of hypothesis W.(A.) for any value A.EA.(Y;S) by a test of level
~
~
~
~
~.,
~
The family of statements
{ $.(8)EA.(Y;oliEI}
~
~
=
(5.5)
{8E@).(Y;O/iEI}
~
based on estimation family [~,~J and critical value S is referred to as a Simultaneous
Confidence Statement (SCS) and denoted [~,~,sJ.
are true if and only if 8Efl
I
one of the confidence regions
i.
Clearly, all statements of [~,~,sJ
~i(Y;S)' i.e., if the true parameters 8 are in each
~.(Y;S),
~
iEI.
The probability that this event occurs,
e.,
(5.6)
is referred to as the joint confidence co-efficient of [~,~,sJ.
By introducing
(5.7)
It is important to distinguish a joint confidence region for several
parametric functions from a simultaneous confidence statement.
The former is a
confidence region in the sense of (5.3), where $. is a vector valued parametric
~
function and is equivalent to a single region in parameter space-(5.4).
The
latter consists of a family of such regions, each one relating to either a scalar
~
I
I
I
I
I
I
_I
I
I
I
(5.4) this may be written
or a vector $ ..
••I
Note that Roy and Bose [22J have restricted consideration to the
I
I
I
I
••
I
I
I.
I
I
I
I
I
I
I
-31-
case where each
~.
~
was scalar valued and specifically to the case where each A.(Y;S)
~
could be expressed as an interval on the real line.
EXAMPLE 5.1.
In the one-way ANOVA set-up a simultaneous confidence statement on
all contrasts
c~u_ c!€V(C')~
-JI;;,? -J
in expectations is
where x indicates the observations,
SF=(k-1)F(k_1'n
)1-~
, e
and
with N = diag(n1,n2""'~)' is a confidence interval for contrast ~j~'
Furthermore,
since C'E:. generates all such contrasts
Ie
I
I
I
I
I
I
I
.I
is a single joint confidence region for all contrasts in expectations.
Since
this region is the intersection of all the A (~sF) regions in the space of conc.
-J
trasts C'E:. (and in that of the parameters E:.,cr 2 ) the joint region and the simultaneous
statement have, by (5.6), the same confidence co-efficient
1~.
This was first pointed out by Scheff: [25] (see also example 3.3 for the
related STP on null contrasts).
An estimation family is said to be related if the. functions of
transitive relation
t
For i
<j
J
~
the relation
transitive, i
-<
have a
which is neither symmetric nor reflexive, such that i f i.,( j
then W.(A.) implies W.(A.) for a particular value A., and
1.
~
J
! defines
J
0 ~
i for all i€I-{o}.
a function r .. such that A. = r .. (A.).
j and j..( g require i
J~
-<
J
J~
~
Since ( is
g so that i f w.(A.) C:w.(A.)Cw (A )
~
~
J J
g g
I
-32-
Ag
= rgi(A i ) = rgj(A j ) = rgj(rji(Ai»~
on these functions.
= rgj(r ji )
which imposes the relation r gi
From these definitions it follows that for i
< j
(5.8)
~j
The functions
indexed by jeI i '
mn
for which there exists no
~ge~
such that j
~
g will be
The adjective minimal will be applied to these functions and
all other concepts depending on these functions, as, for instance, the confidence
regions {A.(y;~)lj€I .}.
J
SCSs
m~n
Similarly, the adjective related will be applied to
based on related estimation families.
EXAMPLE 5.2.
In the subset ANOVA set-up one may be concerned with estimation of
all contrasts in the expectations of each subset.
S
For each subset S., ieI , of
~
k. expectations let C. be a k.x(k.-l) matrix such that C!U generates all contrasts
-~
~
of S4'
...
~
J
Hence, if C!ll
the functions
~.(ll)
= C.~
~ ~
J
~I::.
It is well known that whenever S. c: S. there exists a B.. such that
= B~J~.C!.~
C~
~
~-
= A.
-~
~
J~
this implies that
C~u=A.
JI::. -J
are defined for each ieI
S
where Aj
-
= B~.A..
J~-~
and a relation
Thus,
~.(~)
= B~.~.(~)
J
J~ ~
exists whenever S. c S. .An estimation family for these contrast sets will thus
J
~
be related.
.-I
I
I
I
I
I,
I
_I,
I:
I
1
1
A simultaneous
l~
confidence statement on all these linear sets of con-
trasts is
{C ~!l!eA.~ (~t
F
) I ie I
S
}
where, with the same definitions as before,
F
A.(x.~)
=
~
-'
I' -
-1 C.) -1 C!(x- l ) < t F s2}
{C!ll (X-ll),C.(C!N
_ _
_ lI::.
_
~_
~
~
~
~
I:
I;
Ii
Ij
_1.
I'
!
1
.
...~
!
I
I.
I
I
I
I
i
3
3
Ie
J
-33 -
is the confidence region for
C!~
The corresponding STP for hypotheses of null contrast sets was given in
examples 3.1 and 3.4.
EXAMPLE 5.3.
In the same set-up as that of example 5.2 Roy and Gnanadesikan [23]
have proposed putting confidence bounds on non-centrality parameter
S
for each i€I .
It is readily checked that if SiC: Sj' then for any Ai'
does not generally imply a particular single value A. for
J
A.=O then also A.=O).
J
~
~.(~),
J -
~i(a)=Ai
(except that if
Hence these non-centrality parameter functions do not
allow of a related estimation family.
Defining statistics
-
y, = x1C.[C!N
-
~
~
~
-1
C.] -1 6!x~
~-
' k an can f'd
for each ~€IS, the Roy- Gnana des~
~ ence s t a t emen t '
~s
L
-I
RG
{~. (a)€A,
~
"1
using the augmented F-ratio.
~
~
F,
I
S
(~~) ~€I },
with the confidence regions
J
]
-,
The sense in which this SCS is non-related is illustrated in example 5.5, below,
For a related estimation family [~,Z]
consider, for any Aa €A,
the family
~
a
of hypotheses
]
Ij
(5.9)
and the family of statistics
I
-34-
~A
o
= (Zi(Y;Ai)/A i
= r.1.0 (A 0 )}
•
(5.10)
It is easily checked that { is an implication relation for n
and that (n '~A }
A0 0 A
0
satisfies the requirements of a testing family (section 2, above). Hence
(n
Ao
,t}
'~A
is an STP of level
0
(5.11)
This STP accepts W1..(A i ), where A1.'
=
r.1.0 (A 0 ), if Z.(Y;A.)
< t.
1.
1.-
of W.(A.) by STP (n, ,Z, ,t}, where A.
1. 1.
1\
"'/\
1.
o
0
= r.1.0 (A 0 ),
Thus, acceptance
is determined by the same event
as inclusion of the parametric value Ai in SCS [~,j,t], i.e., Ai€Ai(Y;t) as in
This brings out the correspondence between the STPs
(5.3) .
(~ '~A ,t} for all
o
0
Ao €A 0 and the related SCS [~,Z,~].
tV
provided Zo (Y;A
. 0)
=
The experimentwise error rate of (n '~A ,t} is,
Ao
0
max(Z.(Y;A.)IA.
=
r.
(A
)}
(see
(5.16) below),
1.
1.
1.
1.0 0 .
a,= PW (..../\ )(max(Z.(Y;A.)/A.
> t)
1. .
1.
1. = r.1.0 (A)}
0
o
=
0
P8(max(Z.(Y;~.(8»li€I}
>
1.
1.
t)
which is seen to be l-C, for joint confidence co-efficient C.
Implication relations have been defined for a related estimation family;
hence the terms coherence and consonance may also be defined for a related SCS.
For non-related SCSs
these terms have no clear meaning.
Coherence was defined
in section 3 as the requirement that i f w. (A.)c:: W.(A.), acceptance of W.(A.) would
1. 1.
J J
1. 1.
entail acceptance of w. (A ).
J j
Consonance was defined as the requirement that w. (A.)
1. 1.
be accepted if all W.(A.) were accepted for which W.(A.)CW.(A.).
JJ
1.1.
JJ
related SCS [~,~,t] is said to be coherent if, when A.
J
ensures A.EA.(y;t).
J
AiEAi(y;t).
J
= r J1.
.. (A.)
1.
Correspondingly,
then
A.€A.(y;~)
1.
1.
It is said to be consonant i f A.€A.(y;S)VA.=r .. (A.) ensures
J
J
J
J1.
Thus [~,~,U is coherent and consonant when the statements
and A.EA.(y;t) for all j such that A.
JJ
J
= rj.(A.)
1.1.
1.
A.EA.(y;~)
1.
1.
must occur together for all iEI.
.
--I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
-.
I
I
••I
I
I
I
I
I
I
-35-
In terms of the
(5.12)
and the consonance requirement is that for all i€I
(5.13)
Thus [~,Z,~Jis
coherent and consonant if
I\l
(5.14)
for every i€ 1..
In view of the analogous definitions of coherence and consonance for
Ie
I
I
I
I
I
I
I
respectively.
.•
I
set formulation the coherence requirement is that for
all i€I
STPs
'
e
and SCSs , it is immediately seen that a SCS is coherent and/or consonant,
if and only if everyone of the corresponding STPs
is coherent and/or consonant,
,n
Finally, to complete the analogy between [~,j, ~J and UtA '~A
A €A, the estimation family [~,ZJ will be called monotone if
o
0
o
for all
0
-
Z.(y;A.) > Zj(y;A.)
~
~
-
(5.14)
a.e.
J
whenever A. = r .. (A.), and strictly monotone if
J
J~
~
a.e.
for all iE1.
(5.15)
In particular, for any monotone [~,jJ
Zo (Y;A 0 )
= max{Z.(Y;A.)IA. = r.
~
~
~
~o
(A 0 )}.
(5.16)
These definitions correspond, for any inA '~A } to (2.1), (2.2) and (2.4), so that
o
0
I
.-I
-36-
\
(strict) monotenicity of [~,~J corresponds to the same property of {~ '~A } for
o
each Ao EA.
0
0
Also, estimation family [~,jJ will be said to be UI related if test·w
ing family {~'~A } is UI related for each Ao '
o
0
With these analogous definitions and properties of STPs
and SCSs
clear that the Theorems and Corollaries that have been proved for STPs
mutatis mutandis, to SCSs.
COROLLARY 1':
All SCSs
will apply,
They will be stated below without further proof; the
number of each statement being the same as for STPs
THEOREM 1':
it is
except for a prime.
based on [~zZJ are coherent if and only if [~zZJ is monotone.
If Z.(Y;A ) is a LR statistic for W.(A.)
for all Z.(Y;A.)E~,
then
l.
l.
l.
l.
i
--l.
based on [~,ZJ are coherent.
all SCSs
THEOREM 21:
The joint confidence co-efficient of coherent [~ziztJ is C
15.11). The confidence co-efficient of any
THEOREM 3 :
'
All SCSs
~.
l.
is at least
=
l~L for
Q
l~.
based on [~,ZJ are coherent and consonant if and only if
[~,ZJ is strictly monotone.
COROLLARY 21:
All SCSs
based on [~,ZJ are coherenx and consonant if, and only if,
[~z~J is UI related.
EXAMPLE 5.4.
To show the monotonicity of the estimation family of example 5.2 and
the coherence of the resulting SCSs
one may argue as follows.
It follows from
a well known theorem (applying 1£.1.1 of [19J with c l = xIC!), that
-
l.
max
CleV(Ct)
-
l.
where the r.h.s. is
Zi(Y;Cr~),
the statistic for wi in example 5.2.
Now, hypotheses
of
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
-.
I
I
I.
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I
I
Ie
I
-37-
W.OH): C!jl
~.
~
~-
= A.
have implication relations W.(A.)C:: W.(AJ for all Si::::;lSj
-~
~
provided these A.S, iEI
S
-~
J
~
J
are such that there exists a jl
'"'"0
for which C!jl
~ '"'"0
= A"~'iEIS.
-~
Further, S.= SJ' is equivalent to V(C!) ':::) V(C!) so that
~
~
J
max
C'EV(C!)
1.
,
~
e.,
which proves monotonicity.
Coherence of the SCS of example 5.2 follows by a similar argument.
Let
Si;:) Sj' let ~i be a value ofel~ and write ~ for any value such that ~i = Ci~'
Now ~EAi(~~) is equivalent to saying that Zi(~Ci~) ~~. Hence, by the inequality
above, also
A.
~
Z.(x:C~lJ.
J -
= C~u,the
J~
J-O
) < "',which is equivalent to saying that A.EA.(x."'),where
- ~
-J J - ~
value corresponding to A.
~
= C!u.
~~
Inclusion of A. in the SCS
~
implies inclusion of A., establishing coherence.
-J
EXAMPLE 5.5.
In the Roy-Gnanadesikan set-up of example 5.3 one notes, arguing as
in example 5.4, that i f S.:=> S. then for anyjl. cI>.(jl) > cI>.(ll) and Y. >Y ..
~
J
..... ~- - JI::.
~J
for any given
may occur.
~
Now,
the random event
In that case
set of expectations
~
cI>.(~)
~
F but
E A.RG (~~)
~
cI>.(~) /
J
the noncentrality parameter
. A.(~~ F).
cl>i(~)
may be included in the
Roy-Gnandesikan SCS whereas· the other non-centrality parameter
also that in the event that
Thus for a given
J
cI>.(~)
J
is not.
Note
I
-38-
the converse would occur,
~.(a)
J
being included whereas
~.(a)
.-I
was excluded.
~
Evidently there is no coherence in the inclusions within this SCS.
EXAMPLE 5.6.
sample sizes are equal.
may define
I
I
I
I
I
I
An alternative approach may be taken in the subset ANOVA set-up if
~~(~)
For any subset Si of
k
~i expectations~. ,~.
1
as the set of (2 ) differences
~.(a)
~
The distribution of
Z~(~A.) under
~
-~
~2
, ..
,~.
ia
i • For any
13
a
I-'
one may define statistic
S
values A. of the differences in set
r
~i -~iA'
~l
~
= {al~iS(a) = A.}
-~
W.(A.)
~ -~
one
~k.
~
is that of a studentized
range of k. means and -e
n d.firrespective of the value of A..
,-~
_~
Hence [~S,iRJ
is
N
an estimation family.
_I
Defining, for any i, j such that S. ::;, S., r .. as the func tion picking out
J
~
the subset
S.::;,S ..
~
i\.
-J
J
=
~~(a) from the set ~~(a) of differences, ~~(a)
J
-~
J
~
Now, if
r .. (A.).
J~
J~
S
~.(a)
~
= -~
A.,
clearly
S
~.(a)
J
= r J..~ (i\.),
-~
= r .. (~~(a»
J~
~
whenever
so that W(A.)CW.(i\.) if
-~
I
I
I'
J -J
Hence the estimation family [~S,~RJ is related, and so is the SCS.
~
The resulting SCS of confidence co-efficient l-a is
where sR is the upper a point of the
k
{~~(a)€A~(~~R)li€IS}
mean studentized range distribution with
n d.f. and
-e
Note, in particular, that for Sj consisting of just two expectations
and the confidence region becomes the interval
S
J -
~.(~) =~. -~.
Ja
J
,
13
I
I
I
I
e
I
I
I
I.
I
I
·1
I
I
I
I
Ie
-39-
on the value of the difference A.
J
Ja,
-~
..
J~
This type of SCS is due to Tukey
[53J, being referred to as the method of allowances.
The corresponding STP for
null differences was discussed in example 3.7.
Since
A.
~J
S
S
1 -
J -
4>.(~)~4>.(~)
= r J1
.. (A.).
-1
R
R
for S.::::>S. it is clear that Z.(X.A.) > Z.(X.A.) provided
1
J
1 - ....1
-
J --J
Moreover,. it is readily apparent that
r .. (A . ) , S.c S. }
J1 -1
J
1
so that [~S,ZRJ
is strictly monotone.
tV
S -R R
Coherence and consonance of SCS [¢ ,~ ,s J, which follows from Theorem 3',
may be checked
as follows.
From the above relation between Z~(X.A.) for S. and
1
--1
1
R
the statistics Z.(X.A.) for all pairs of expectations S.(cS.), it follows that
J - -J
.
J
1
(Z~(~A.)
< sR)is equivalent to(Z~(~A.) < sR, where A. = r .. (A.), V pairs S.(cS.)).
1
-1 J
-J -J
J1 -1
J
1
Thus the statement A.EA~(~sR) is equivalent to the statement that A.EA~(x.sR)
-1
where A.
I
I
I
I
I
I
I
I·
I
=~.
-J
= r J1.. (A.),
-1
-J
1
J-
for all pairs S.(cS.), and this establishes coherence and
J
1
consonance.
Resolution may be defined and compared for SCSs
it was for STPs
in section 4, above.
analogously to the way
[¢,~,~J
However, a comparison of SCSs
and
[¢*,~*,~*J has meaning only if the hypotheses on the functions ~ are the same as
those on some of the functions of ¢*.
4>
o
E~
Thus, for example, for each value A of
o
there must exist another value A*, say, of
0
4>*E~*
such that W (A )
0
0
=
W*(A*).
0
0
If that is so and if I . c 1* then the conditions
m1n (5.17a)
equally for all A EA , and
o
0
~.,
(5.17b)
I
-40-
N
define [<p,b~J to be no less resolvent than [<p*,~*, ~*J. If the containment in
(5.17b) is proper the former SCS is said to be strictly more resolvent than the
latter.
Thus, if two 8CSs
which can be compared have the same joint confidence
co-efficient, but the minimal confidence regions of the one are contained in the
corresponding regions of the other, the former is said to be more resolvent.
Again, let estimation families [<p,~J and [<p*,Z*J have related hypotheses,
and for each 7\ €A and the corresponding 7\* such that w (7\ ) = w*(7\*), let
00
0
00
00
~ O~* also hold.
07\
o
Then [<p,~,~J will be said to be no less parsimonious than
0
Aj(Y;~) = A*J.(Y;~*)
for all j€I.
m~n
and for
W
o
(7\ ) = w*(7\*)
0
0
0
~.,
(5.18b)
(5.18c)
_I
Again, if the latter inequality is strict the former SCS is said to be strictly
more parsimonious than the latter.
In other words, for equal minimal confidence
regions the more parsimonious SCS has all non-minimal confidence regions contained
in the corresponding regions of the less parsimonious SCS, and its joint confidence
co-efficient is larger.
Parsimony must here be understood in the sense of in-
creasing joint confidence, and that is equivalent to reducing the experimentwise
error rate.
Further, estimation families with related hypotheses may allow narrowness
comparisons.
Thus [<p,iJ is said to be narrower than [<p*,!*J if, for each 7\0 and
the corresponding 7\~ (i.e., wo (7\o) = w~(7\~», {~'~7\ } is narrower than {n*7\*'~~ }.
o
0
o
The requirements for this are that for each 7\
o
(4.3a,b,c,d) for the corresponding hypotheses.
0
there exist a function 8r...
I
I
I
I
I
I
(5.18a)
C > C*.
.-I
satisfying
I
I
I
I
I:
I
I
o
This function mayor may not be
• •i
;
I:
I
I.
/1
I
I
I
I
I
I
Ie
I
I
I
I
I
I
I
,.
I
-41-
the same for all A EA.
o
Thus, narrowness of estimation families is defined simply
0
as narrowness of the testing families which they generate.
Again, the following Theorems and Corollaries may be stated in view of
these further analogies between properties of STPs
THEOREM 4':
and SCSs .
If [¢,ZJ is narrower than [¢*,f*J, then for every t* there exists
such that [¢,Z,tJ is no less parsimonious than [¢*.Z*,t*J.
t
Moreover, the former
SCS is strictly more parsimonious than the latter for some values of t*, i.e.,
those for which
Pw (A )(Z~(Y;Ao) > ~* ~ ~ (Zo(Y;A o ») > 0,
o
where
~
~/\
o
0
0
satisfies (4.3a,b,c,d) for all A0 EA'.
0
THEOREM 5':
If [¢.ZJ is narrower than [~*,Z*J and if [¢,i,tJ and [¢*,Z*.t*J have
the same co-efficient, then the former SCS is no less resolvent than the latter.
Moreover, it is strictly more resolvent for some values of t, i.e., those for which
Pw (A )(Z~(Y;Ao) > ~ (~) > ~ (Zo(Y;A o
o
where
~
-/\
o
0
0
»)
> 0
0
satisfies (4.3a t b t c,d) for all A0 EA0 .
Clearly, narrower estimation families have the advantage of providing more
resolvent SCSs
SCSs
with the same joint confidence co-efficient and more parsimonious
for identical minimal confidence regions.
EXAMPLE 5.7.
In a subset ANOVA set-up with equal sample sizes, compare the estimation
families [¢S,ZRJ of example 5.6 and [¢S,ZF
J of example 5.2.
N
~
The relation
I
-42-
F
Z
R
R
F
R
g(Z ) = (Z )2/2 holds for all Z.(~'A,) and z.(~'Aj) i f S. is a pair.
=
J
J
J
J
As
in example 4.3, above, so also here Z~(~'A.) > [ZR(~'A.)J2/2 if S. contains three
1 -1-1
1
or more expectations and these inequalities are strict ~ .•
Hence [~S,ZRJ is
'"
S -F
.
narrower than [~ ,Z J and it follows from Theorems 4' and 5' that for all subsets
'"
the SCSs due to Tukey are more parsimonious or more resolvent than the corresponding subset SCSs
EXAMPLE 5.8.
based on Scheffe's approach.
The comparison in example 5.7 between Tukey and Scheffe confidence
•
•
statements for subsets of expectations also follows, by v1rtue
of Corollary 3,
condition (I), from the UI and LR relations of the respective estimation families,
Let [~,ZJ and [~*.Z*J be monotone estimation families such that
COROLLARY 3':
~ ~~,
~
.
c ~*.
and w ('A )
m1n --- 0 0
m1n -
AoEA o a function
~o
= w*('A*)
0
0
for all A EA
0
0
and let there exist for each
.
satisfying (4.3a,b) for the corresponding hypotheses.
Then each
of the following are sufficient conditions that [~,iJ provide more p~rsimonious and
more resolvent SCSs than [~*,2J,
(I)
i f [~z%l is UI related and ~o also satisfy (4.3e) or (4.3e') for
0
(II)
P
W
o
if both [~,ZJ and [~*,i*J are UI related and
(. Jmax{Z*j(Y;r j (A »ljEI*. } > max{Z~(Y;r. (A »ljEI . }) > 0
1\ r o o
m1n
J
JO
0
m1n
I
I
I
I
I
I
_I
I
1
each A ,
--
--I
(5.19)
0
for all Ao EA 0 .
IJ
I:
I
I:
I:;
I!i
This Corollary covers some results of Krishnaiah for ANOVA and MANOVA
set-ups [13J.
He uses UI related families throughout and his Theorems 6.2, 6.3
and 6.4 establish that for an identi ty function g and finite I . c 1*. , the
m1n
m1n
I
-.:
I
I
-43-
I.
I
I
I
I
I
I
I
minimal confidence regions of [¢,~,~J will be properly contained in those of [¢*,~*,~*J
where both SCSs
have the same co-efficient.
Apart from cases where (S.19) might
be zero his conditions are seen to correspond to (II) of Corollary 3', and his
containment conclusion corresponds, again apart from possible sets of probability
zero, to the greater resolution conclusion of Corollary 3'.
He does not explicitly
discuss anything corresponding to parsimony.
The following are two examples of Krishnaiah's results:
EXAMPLE S.9.
Dunnett's simultaneous
1~
confidence bounds on comparisons of the
expectations under (~-l) treatments with that under a control [sJ are more resolvent
than Tukey's simultaneous
these k expectations [27J.
1~
confidence bounds on all pairwise comparisons of
(Krishnaiah, [13J, Corollary 6.3).
The corresponding comparison of
EXAMPLE S.10.
Simultaneous SANOVA
1~
ST~
was given in example 4.13.
confidence bounds on a finite subset of
./
I
I
I
.
contrasts in k expectations are more resolvent than Scheffe-type s1mu1taneous
confidence bounds on all contrasts
6.
(Krishnaiah [13J, Theorem 6.2).
Other methods of multiple comparisons
The previous discussion relates almost entirely to STPs
and SCSs
are based on monotone families, and thus provide coherent decisions.
I
1-~
which
Some examples
have been mentioned of procedures for non-monotone families in which case the
decisions are not always coherent and the properties in the Theorems do not apply.
I
In particular, we may mention the use of the Kruska1-Wa11is statistic for non-
-
parametric ANOVA (example 3.2) proposed by Nemenyi [17J (see also the Miller's
discussion [16J, section 4.6).
•
One would presume that incoherences, though
possible with this technique, are pretty rare, especially with large samples .
~
In that case such a technique might be said to be asymptotically coherent.
may warrant further investigation.
This
I
-44-
Other techniques have a step-wise procedure built in which ensures coherence
but destroys simultaneity.
Thus, .Newman [18J and Keuls [lOJ, in the equal sample
size ANOVA, decide on each subset of means according to whether the studentized
~
range for that subset or any other set containing it is
tribution of its range..
Duncan
[3~
significant in the dis-
proceeds in a more intricate way, choosing
a different percentage point for each one of the ranges.
He also extended the
method to F-statistics [4J.
In these methods the decision on a particular w. depends not only on the
~
statistic Z. but on all Z. such that j
J
~
< i.
Clearly, this does not provide a test-
ing family in the sense of section 2 and the STP results do not apply.
These
methods have the disadvantage.of making the decision on w. depend in part on
~
statistics Z. whose distribution depends not only on w. but also on W.-W. which
J
~
is irrelevant to W.,
~
~
On the other hand, these methods offer better resolution
in the sense of section 4 -- than
workers.
J
ST~
and are therefore preferred by some
(For a detailed discussion see [6J, section 9)
Acknowledgements:
I am indebted to my colleagues in Jerusalem and Chapel Hill
with whom I have discussed these techniques.
I am particularly happy to acknowledge
the help of J. Putter who critically read an early version of the paper and that
of W. J. Hall who made many useful suggestions in the last revisions of this paper.
AITCHISON, J. (1964).
Confidence-region tests.
J. R. Statist. Soc. ~ ~
462-476.
[2J
AITCHISON, J. (1965).
Likelihood-ratio and confidence region tests.
J. R. Statist. Soc .
.!! ll,
245-250.
I
I
I
I
I
I
_I
I
I
I
I"
Ie
REFERENCES
[lJ
••I
I
I
••
I
I
I.
I
I
I
I
I
I
I
-45-
[3J
~
[4J
Virginia Journal of Science
(New Series) No.3, 171-189
DUNCAN, D. B. (1955).
Multiple range and multiple F-tests.
Biometrics
1lJ
164-176.
[5J
DUNNETT, Charles W. (1955).
A multiple comparison procedure for comparing
several treatments with a control.
J. Amer. Statist. Assoc.
~
1096 -1121.
[6J
GABRIEL, K. R. (1964).
A procedure for testing the homogeneity of all sets
of means in analysis of variance.
[7J
GABRIEL, K. R. (1966).
I
I
I
I
I
I
I
[9J
Biometrics
~
459-477.
Simultaneous test procedures for multiple comparisons
on categorical data.
Ie
I
A significance test for differences between ranked
treatments in an analysis of variance.
[8J
Ie
DUNCAN, D. B. (1951).
J. Amer. Statist. Assoc.
~
1081-1096.
GABRIEL, K. R. (1967). Simultaneous test procedures in multivariate analysis of
variance.
University of North Carolina, Institute of Statistics, Mimeo series.
GABRIEL, K. R. (1967).
On the relation between union intersection and
likelihood ratio rests.
To appear in S. N. Roy memorial volume
(R. C. Bose, ed.).
[10J
KEULS, M. (1952).
The use of the 'studentized range' in connection with an
analysis of variance.
[llJ
KNIGHT, William (1965).
~
[12J
Euphytica
b
112 -122.
A lemma for multiple inference.
Ann. Math. Statist.
1873 -1874.
KRISHNAIAH, P. R. (1964).
Multiple comparison tests in multivariate case.
U.S.A.F. Office of Aerospace Research, ARL 64-124.
[13J
KRlSHNAIAH, P. R. (1965).
On the simultaneous Anova and Manova tests.
Ann. lnst. Statist. Math. 11j 35-53.
[14J
LEHMANN, E. L. (1957).
A theory of some multiple decision problems, I.
Ann. Math. Statist.
~
1-25.
-46-
[ 17l
[ 18]
I
••I
I
I
I
I
I
I
_I
I
I
I
I
II
I
••
I
I
I.
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I
I
Ie
-47-
[27]
WKEY, J. W. (1951).
Quick and dirty methods in statistics.
Simple analyses for standard designs.
Part II.
Proceedings Fifth Annual
Convention, American Society for Quality Control, 189-197.
[28J
WKEY, J. W. (1953).
manuscript ..
The Problem of Multiple Comparisons.
Unpublished