March 8, 1988
A Simple Approach to Inference in Random Coefficient Models
Marcia Gumpertz and Sastry G. Pantula
Department of Statistics
North Carolina State University
Raleigh, NC 27695-8203
Key Words and Phrases: Repeated measures regression; Asymptotic inference;
Estimated generalized least squares; Growth curve.
ABSTRACT
Random coefficient regression models have been used to analyze crosssectional and longitudinal data in economics and growth curve data from
biological and agricultural experiments.
In the literature several estimators,
including the ordinary least squares (OLS) and the estimated generalized least
squares (EGLS), have been considered for estimating the parameters of the mean
model.
Based on the asymptotic properties of the EGLS estimators, test
statistics have been proposed for testing linear hypotheses involving the
parameters of the mean model.
An alternative estimator, the simple mean of the
individual regression coefficients, provides estimation and hypothesis testing
procedures that are simple to compute and simple to teach.
The large sample
properties of this simple estimator are shown to be similar to that of the EGLS
estimator.
The performance of the proposed estimator is compared with that of
the existing estimators by Monte Carlo simulation.
1.
INTRODUCTION
Frequently in biological, medical, agricultural and clinical studies
several measurements are taken on the same experimental unit over time with the
objective of fitting a response curve to the data.
Such studies are called
growth curve, repeated measure or longitudinal studies.
In many bio-medica1
and agricultural experiments the number of experimental units is large and the
number of repeated measurements on each unit is small.
On the other hand, some
economic investigations and meteorological experiments involve a small number of
units observed over a long period of time.
Several models for analyzing such
data exist in the literature; the models usually differ in their covariance
structures.
See Harville (1977) and Jennrich and Schluchter (1986) for a
review of the models and of approaches for estimating parameters.
Recently,
there seems to be a renewed interest in analyzing repeated measures data using
Random Coefficient Regression (RCR) models.
In this article we present a brief description of the RCR models and
some of the existing results for these models.
We present a simple approach
that is not difficult to discuss in a course on linear models.
estimation procedure with two of the existing methods.
Section 2 contains the
assumptions of the model and a brief review of the literature.
present the properties of the simple estimator.
the estimators is presented in Section 4.
which includes some possible extensions.
We compare our
In section 3 we
A Monte Carlo comparison of
Finally, we conclude with a summary
Page 2
2.
RCR MODEL
Suppose that the t observations on the ith of n experimental units are
described by the model
,
,
i=1,2, ... , n
y. = X.f3. + e.
"
(2.1)
,
where y. = (y. ,y. , ... , y. )' is a tx1 vector of observations on the response
t
"1 2
,
,
,
variable, X. is a txk matrix of observations on k explanatory variables, 13. is
,
a kx1 vector of coefficients unique to the ith experimental unit and e. is a
tx1 vector of errors.
Each experimental unit and its response curve is
considered to be selected from a larger population of response curves, thus the
,
regression coefficient vectors 13., i=1,2, ... ,n may be viewed as random drawings
from some k-variate population and hence (2.1) is called a RCR model.
In this
paper we discuss the estimation and testing of such models under the following
assumptions:
,
(i) the e. vectors are independent multivariate normal variables
with mean zero and covariance matrix a 2 I
t
i
(ii) the f3
i
vectors are independent
multivariate normal variables with mean 13 and nonsingular covariance matrix
Eo1Jf3 ; (iii) the vectors 13., and e.J are independent for all i and j; (iv) the X.,
matrices are fixed and of full row rank for each i; (v) min(n,t) > k and
(vi) there exists an M <
~
such that the elements of
M in absolute value for all i and t.
restrictive.
,,
t(X~X.)
-1
are less than
The assumption (vi) is not very
It is satisfied for the models that include polynomials in time
and stationary exogeneous variables.
Several authors, including Rao (1965), Swamy (1971), Hsiao (1975), Harville
(1977), Laird and Ware (1982), Jennrich and Schluchter (1986) and Carter and
Yang (1986) have considered the estimation and testing for the RCR models.
We
summarize the results of Carter and Yang (1986) since they consider the large
sample distribution of the estimated generalized least squares (EGLS) estimator
Page 3
as nand/or t tend to infinity.
For the sake of simplicity, we have assumed
that equal number of repeated measurements are taken on all experimental units
and that the variance of the error vector e. does not depend on i.
1
However,
similar results exist for more general cases and will be discussed in the
summary.
Consider the least squares estimators
b. = (X'.X.)-1 X'.y.
i=1,2, ... , n
11111
of
Note that the b. 's are
computed for each individual experimental unit.
~
independent and normally distributed with mean
-1
Wi =
~
(2.2)
E~~ +
2-1
a (XiXi)
•
~
1
and variance
Therefore, the best (linear) unbiased estimator of
is the generalized least squares (GLS) estimator.
Swamy (1971) showed that
n
n
= ( E w.)-'( E N.b.)
~GLS
i=1
i=1
1
1
(2.3)
1
that is, the GLS estimator is the "weighted" least squares (average) estimator
of b. where the weights are the inverse variance-covariance matrices of b ..
1
1
Under the normality assumption,
of
~
(provided
E~~
and a
2
~GLS
is also the maximum likelihood estimator
are known).
The elements of
E~~
and a
2
are seldom
known and hence we consider the estimated GLS (EGLS) estimator
n
~EGLS
where
=( E
i=1
n
1
A
W.)
1
(E
.
1=
1
W.b.)
1
1
(2.4)
Page 4
A_1
A2-1
Wi = E ~~ + a (X Xi)
i
= Sbb
-1 2 n
A
E (X~X.)-
- n a
i =1
n
= (n-1) -1 E (b. i =1'
= [n(t-k)]
-1 n
1
'
b) (b.
- b)
I
,
E
[y~y.
i=1
"
-
,
b~X~y.]
",
and
-1 n
b = nEb.
. 1 '
,=
Carter and Yang (1986) suggested inference procedures based on the large
sample distribution of the estimator
below.
~EGLS.
Their results are summarized
(They suggested a slightly different estimator of
E~~
in the case
E~~
is not nonnegative definite.)
Result 2.1:
(vi).
Consider the model given in (2.1) with the assumptions (i) through
Consider the statistic
2
T
o:
for testing H
rows.
(a)
= [L
L~
= ~o '
A
-1
-1
L I]
A
[L ~EGLS - ~o] ,
(2.5)
where L is a qXk matrix of q linearly independent
Then,
for a fixed nand t tending to infinity:
(n-q)q
(b)
n
A
~EGLS - ~O] I [L( i:1 Wi)
-1
(n-1)
-1 2
T
is (asymptotically) distributed as F(q,n-q),
for a fixed t and n tending to infinity:
T2 is (asymptotically) distributed as chi-square with q degrees
of freedom,
and
Page 5
(c)
2
T is approximately
for the case where nt is large and q=1:
distributed as F(1,v) where
II
=L
I
,
and
Proof:
See Carter and Yang (1986).
[]
Carter and Yang (1986) proved part (b) of the above result by observing
that the distribution of
that of
~EGLS
~EGLS
is asymptotically (as n
~~)
equivalent to
To prove part (a), they observed that the distribution of
~GLS'
is asymptotically (as t
II
13
~~)
equivalent to that of
= n -1
(2.6)
which is also asymptotically equivalent to I3
GLS
as t
Finally, when
~~.
nt is large, Satterthwaite's approximation was used to approximate the
2
distribution of T.
In the next section we present inference procedures based
on the large sample distribution of the simple estimator
3.
A SIMPLE APPROACH
It is well known that the GLS estimator
unbiased estimator of
estimator
~EGLS
~
b.
~GLS
is the best (linear)
and that (under some regularity conditions) the EGLS
is asymptotically (as n
~~)
equivalent to the GLS estimator.
However, in small samples, the distribution of
~EGLS
may be far from being
Page 6
normal.
It is also argued that the estimator
~EGLS
may even be worse than
the ordinary least squares (OLS) estimator,
~OLS
because
~EGLS
n
n
= ( E X'.X.)-'( E X'.X.b.)
i=1
1
i=1
1
n
1
=
1
1
~
and variance
1 [E.n 1X '. X. I: ooX ~ X. + a 2 E.n 1X !X . ] (E.n 1X '. X . )-1 •
1=
1 1
,.,,., 1 1
1=
1 1
1=
1 1
Thus, to compute either the EGLS estimate
covariance matrices of
elements of
(3.1 )
1
It is easy to see that the OLS estimator,
is normally distributed with mean
(E. 1X '. X. ) -
1
depends on the estimated variance-covariance matrix which may
introduce additional variability.
~OLS'
1
I:~~
and a
~EGLS
2
and
~OLS'
~EGLS
or to compute the variance
it is necessary to estimate the
We now present the properties of the simple estimator
b, which does not require the estimation of
I:~~
2
and a .
Note that the GLS, EGLS and OLS estimators are weighted averages of the
individual least squares estimators b .•
1
b
The estimator
-1 n
= nEb.
i=1
(3.2)
1
is the simple average of the individual least squares estimators.
In the
special case where the model matrix X. is the same (=A say) for all
1
individuals, then the GLS, EGLS and OLS estimates coincide with the estimator
b.
The estimator b is normally distributed with mean
Var(b)
~
and variance
(3.3)
Page 7
Note that
= E[(n-1)-1
= (n -1 ) -1 E[
n
E (b.- b)(b.- b)'l
i =1'
~
(b. -
. l'
,=
,
~)( b ,. - ~)
I
n (b
-
-
~)( b - ~) .]
= (n-1)-1[ ~ var(b.) - n Var(b))
. 1
'
,=
= (n_1)-1[n 2 var(b) - n Var(bG
=n
var(b)
-
-1
Therefore, a simple unbiased estimator for var(b) is n Sbb
That is, the sample variance (covariance matrix) divided by n is an
unbiased estimator for the variance of the sample mean even though the variances
(of b.)
are not
homogeneous.
7
Consider the statistic
(3.4)
for testing H :
o
A*2
Notice that T
L~
= ~o'
where L is a qxk matrix of linearly independent rows.
2
is the Hotelling's T statistic one would compute if the
,
,
variances of the b.'s were equal (i.e., if the X.'s were the same for all
individuals).
A*2
Before we establish that the statistic T
has similar asymptotic
2
properties as that of the statisticT , we will make a few remarks.
Remark 3.1:
,
Recall that the estimators b. are independently and normally
distributed with mean
~
-1
and variance Wi
= E~~
2-1
+ a (XiXi)
Under the
assumption (vi), the elements of the matrices t(X~X.)-1 are uniformly (over i)
,,
Page 8
bounded.
Therefore, the matrices (X~X.)-1, i=1, ... ,n, converge uniformly
,,
(over i) to zero as t tends to infinity.
,
b. =
,
~.
Also, note that
,
+ Z.
(3.5)
where
i=1, ... ,n.
Since var(Z.) = a2(x~x.)-1 converges to zero (uniformly in i) as t tends to
,
,,
,
tending-
infinity, the difference between b. and
,
~.
tends to zero in probability.
-1 n
Therefore, for n fixed and t
to infinity, b = n E. 1b. and
-1 n
,., = n E., = 1~', are asymptotically equivalent. In fact, since
-
,= ,
-0
var(Z)
and
where J is a matrix with all elements equal to 1, we have
(3.6)
Hence b is also asymptotically (as t
~ m)
equivalent to
~GLS
and
~EGLS.
(See also Hsiao (1975) for similar comments.)
It is important to note here that the OLS estimator
necessarily asymptotically equivalent to b.
however, is not
For example, suppose
where B is a fixed kxk positive definite matrix.
satisfied.
~OLS'
In this example, the OLS estimator
itB
Then the assumption (vi) is
~OLS
and hence ~OLS is not asymptotically equivalent to
,,=
X~X.
is (Eni=1'.)-1 Eni=1'. b i
b = n-1
n
E.,= 1b ,..
Page 9
Remark 3.2:
For a fixed t and n tending to infinity, the estimator b may not
be asymptotically equivalent to
estimator.
~GlS
and hence may not be an efficient
However, we know that the exact distribution of
b is normal and
hence the (exact) distribution of
is chi-square with q degrees of freedom, where l is a qXk matrix of rank q.
A*2
We now present the asymptotic distribution of the T statistic as n
and/or t tends to infinity.
Result 3.1:
(vi).
Consider the model given in (2.1) with the assumptions (i) through
Consider the test statistic r*2 defined in (3.4) based on the
estimator b. Then,
(a)
(b)
for a fixed nand t tending to infinity:
_1
-1 A*2
(n-q)q (n-1)
T
is (asymptotically) distributed as F(q,n-q),
A*2
for a fixed t and n tending to infinity: T
is (asymptotically)
distributed as chi-square with q degrees of freedom,
and
(c)
A*2
for the case where nt is large and g=1: T
is approximately
distributed as F(1,v* ) where
v * = g- 1 [i'I:
~~
i + n
-1 2 n
a E
1
i'(X'.X.)- i]
2
i=1' ,
g
and
L = i'
Proof:
.
See Appendix.
()
Page 10
With the exception of v * , the Satterthwaite's approximation for the degrees
of freedom, the asymptotic distributions of T2 and r*2 are identical.
A*2
advantage of T
2
over T
The
is that it is simple to compute and is simple to
Note that, as in the case of T2 , the degrees of freedom v * (a) tends
explain.
to (n-1) as t tends to infinity and (b) tends to infinity as n tends to
Also, the degrees of freedom v * is always greater than or equal to
infinity.
(n-1) and hence the approximation in (c) serves as a compromise between the F
and chi-square approximations.
To summarize, we have seen that asymptotically (as t ~GLS' ~EGLS
(as n -
~),
and b are equivalent and are efficient.
the estimators
~EGLS
and
~GLS
~),
the estimators
Also, asymptotically
are equivalent and are efficient.
However, for a fixed t and n large b may not be as efficient as
~GLS
and
hence the tests based on b may not be as powerful as the tests based on
~GLS'
The distribution of b is exactly normal for all nand t, whereas the exact
distribution of
~EGLS
is unknown.
A small Monte Carlo study was conducted to
compare the performance of the test statistics based on b and ~EGLS'
the study, the test statistics based on
~OLS
were also included.)
(In
The
results of the study are summarized in the next section.
4.
MONTE CARLO SIMULATION
Consider the model
Yij = ~Oi + ~1iXij + e ij , i=1, ... ,n
j=1, .•. ,t
where
~i
=
(~Oi' ~1i)'
are
NID(O,E~~);
xij's are independent N(O,9) random
variables if i is even and N(O,4) if i is odd; e ,..J 's are NID(O,4);
and {e .. } are independent and
'J
(~.},
,
{x ,..J }
Page 11
E~~
4
4
=[
:]
The values for nand t are taken to be 5, 10 and 50 to represent small,
moderate and large samples.
A set of 250 x.
0
'J
values were generated once for
all and the same values of x. 0' i=1, ... ,n; j=1, ... ,t, were used in all of the
'J
replications. For each pair of values of nand t, 100 Monte Carlo replications
were used.
In each replication, independent ~o 's and e .. 's were generated.
,
'J
2 A*2
2
Test statistics (T , T and TOLS ) based on ~EGLS' b and ~OLS for
testing the hypotheses (i) HO:
and (iv) H :
O
~O
=
~1
~1
= 0, (ii) HO:
= 1 were computed.
~O
=
~1
~1
= 0; (iii) HO:
=
The number of times the test
statistics rejected the hypotheses are summarized in Tables 1 and 2.
From the asymptotic results in section 3 we would expect that the EGLS
estimator
is large.
~EGLS
and the simple estimator b perform equally well when t
However, we do not expect the ordinary least squares estimator to do
as well as b when t is large.
For t=50 this expectation was borne out.
At
all values of n the probability of rejecting a true hypothesis (using the Fapproximation) was 9% or less for all three statistics, but the power for
rejecting either of the false hypotheses was always greater for
than for
~OLS.
identical.
Furthermore the rejection rates for
A look at the true variances of
~GLS'
~EGLS
b and
~EGLS
and b
and b were
~OLS
revealed
that the relative efficiency of b was almost 100% for both the intercept and
the slope parameters, whereas for
~OLS
it was only 67% for the intercept
parameter and 89% for the slope parameter in the case when n=5 and t=50.
smaller t, the efficiency of
b was always close to 100%.
~OLS
was even worse.
For
However, the efficiency of
Similar values for the relative efficiencies of
the estimators were observed when n=5 and n=50.
Page 12
Table 1.
Comparison of the Levels of Test Criteria:
The Number of Times
a 0.05 Level Test Criterion Rejects the Hypothesis (out of 100 replications).
(i )
n
t
5
5
10
50
10
5
10
50
50
5
10
50
Estimator
F
1, n-1
(i i )
HO: 13 1 = 0
2
X1
F1 ,v*
F 2,n-2
2
X2
11
10
4
10
0
26
32
19
2
11
1
24
30
8
29
29
18
EGLS
BBAR
OLS
6
15
16
14
EGLS
BBAR
OLS
9
9
8
18
18
17
9
9
EGLS
BBAR
OLS
5
5
5
12
12
12
5
5
EGLS
BBAR
OLS
3
3
5
5
5
3
3
EGLS
BBAR
OLS
7
7
9
7
7
8
9
2
HO: 13 0 = 13 = 0
1
9
5
3
2
2
9
6
10
8
4
4
5
4
5
4
3
3
EGLS
BBAR
OLS
8
9
8
7
8
6
7
7
7
8
8
EGLS
BBAR
OLS
1
1
2
2
1
1
2
3
0
1
1
EGLS
BBAR
OLS
4
4
4
4
4
EGLS
BBAR
OLS
4
10
3
5
5
2
4
4
2
2
1
14
10
11
6
16
16
6
13
13
5
10
9
9
2
2
1
2
2
4
Page 13
Table 2.
Comparison of the Powers of Test Criteria:
The Number of Times
a 0.05 Level Test Criterion Rejects the Hypothesis (out of 100 repl ications).
(;)
1 n-1
2
X1
EGLS
BBAR
OLS
13
13
13
10
EGLS
BBAR
OLS
50
(i i )
H : ~O
O
= ~1 =
F ,
2 n-2
2
X2
24
25
23
13
15
5
10
3
34
43
22
12
12
12
18
18
23
12
13
8
16
4
34
41
27
EGLS
BBAR
OLS
13
13
11
27
27
26
13
13
11
11
7
48
49
31
5
EGLS
BBAR
OLS
13
12
10
17
16
15
14
13
15
17
5
26
28
16
10
EGLS
BBAR
OLS
18
19
16
27
27
23
18
19
19
19
6
33
34
21
50
EGLS
BBAR
OLS
15
15
14
20
20
20
15
15
18
18
9
35
35
24
5
EGLS
BBAR
OLS
69
67
51
70
69
54
70
67
79
80
52
82
84
57
10
EGLS
BBAR
OLS
70
70
56
70
71
60
70
70
86
85
51
87
88
53
50
EGLS
BBAR
OLS
67
67
56
69
69
58
67
67
83
83
72
88
88
76
t
5
5
50
=1
F1 ,V*
n
10
HO: ~1
Estimator
F
I
Page 14
As n approaches infinity for fixed t we would expect
powerful than
b.
~EGLS
to be more
As it turned out, for n=50 the rejection rates for ~EGLS
and b (using the x
rejection rate for
2
approximation) were nearly indistinguishable.
~OLS
The
ranged from 14 to 39 percent lower than that of the
other two estimators.
For small sample sizes none of the estimators was very powerful.
contrary to our expectation, the performance of
estimator b may have been more powerful than
was reasonable.
~EGLS
~EGLS
~EGLS'
~O
=
~1
The
~o
in rejecting HO:
but by the same token, b rejected the true hypotheses, H :
O
often than
However,
=
~1
=1,
= 0, more
One problem that other authors (e.g., Jennrich &
Schluchter (1986), Carter and Yang (1986)) have noted is that, with small sample
sizes,
E~~
is often not a positive definite matrix.
In our simulation this
occurred 34% of the time for n=t=5, but for moderate sample sizes, (n=t=10)
this was no longer a problem.
(If
E~~
is not positive definite, the modified
estimator suggested by Carter and Yang (1986) was used.)
,
In our simulation even though the X. matrices were different for different
1
individuals, the weight matrices W. turned out to be close to n- 1.
,
be one of the reasons why the tests based on b and
~EGLS
This may
had very similar
power for all sample sizes.
5.
SUMMARY
In random coefficient regression models several estimators for
the literature.
~
exist in
Carter and Yang (1986) derived the asymptotic distribution of
the estimated generalized least squares estimator as either n, the number of
experimental units, tends to infinity and/or as t, the number of repeated
measurements on each unit, tends to infinity.
based on the EGLS estimator.
They proposed test statistics
-
The simple average b = n
-1
n
E.,= 1b., of the
Page 15
regression estimates from each unit has not received much attention in the
literature.
The main contribution of the paper is to show that inferences can
be made, without much difficulty, using the simple estimator b.
b, similar to those derived by Carter and Yang
results for the estimator
(1986) for
~EGLS'
Asymptotic
are derived.
Also, the results of a small Monte Carlo
study indicate that it is reasonable to use b for inferences on
~.
It is important to emphasize the simplicity of the estimator b, the test
statistics based on b and their asymptotic properties.
is not as simple to compute.
computation of
~EGLS
Also, the estimator
E~~
may need to be adjusted so that
The estimator
~EGLS
that enters the
E~~
is positive
A
definite.
estimator
We are, however, not suggesting that
~EGLS
~EGLS
be ignored.
The
may perform very well for several model matrices (especially
when n is large).
,
Our results extend to the case where unequal number (r., say) of
measurements are made on different individuals.
In this case, part (a) of
,
Result 3.1 should be modified to say "for a fixed n and minimum (r.) tending to
infinity."
2
,
o.
,
Also, when minimum (r.) is large, the Result 3.1 (a) holds even if
= variance
(e .. ) is not the same for different experimental units
'J
2
,
(provided one uses s., regression mean square error for the regression of ith
individual, to estimate o~).
,
Wh~n n is large, Result 3.1 (b) holds even if
o~ ~
0
of.
Our results can also be extended to the case where the errors e. . are
2
for all i, provided we assume that for all i, o~ ~
0
2
for some finite
'J
For example, suppose for each i, {e .. : j=1, ... ,t} is a
'J
correlated over time.
stationary time series with variance covariance matrix of e., given by Eee .
-1
-
is easy to see that n Sbb is still an unbiased estimator of var(b).
Under
It
Page 16
some regularity conditions (similar to those given in Section 9.1 of Fuller
on X" Loo and L one can obtain the asymptotic results for the
,
~~
ee
test statistic based on band Sbb. The proofs, however, are not included for
(1976»
the sake of brevity.
ACKNOWLEDGEMENTS
The work of S. G. Pantula was partially supported by the National Science
Foundation.
REFERENCES
Carter, R. L. and Yang, M. C. K. (1986). "Large Sample Inference in Random
Coefficient Regression Models," Communications in Statistics - Theory and
Methods, 15(8), 2507-2525.
Fuller, W. A. (1976). Introduction to Statistical Time Series, New York:
John Wiley and Sons.
Harville, D. A. (1977). "Maximum Likelihood Approaches to Variance Component
Estimation and to Related Problems," Journal of the American Statistical
Association, 72, 320-340.
Hsiao, C. (1975). "Some Estimation Methods for a Random Coefficient Model,"
Econometrica, 43, 305-325.
Jennrich, R. I. and Schluchter, M. D. (1986). "Unbalanced Repeated-Measures Models
with Structured Covariance Matrices," Biometrics, 42, 805-820.
Laird, N. M. and Ware, J. H. (1982).
Data," Biometrics, 38, 963-974.
"Random-Effects Models for Longitudinal
Rao, C. R. (1965). "The Theory of Least Squares When the Parameters are
Stochastic and its Applications to the Analysis of Growth Curves,"
Biometrika, 52, 447-458.
Swamy, P. A. V. B. (1971). Statistical Inference in Random Coefficient
Regression Models. Berlin: Springer-Verlag.
Page 17
APPENDIX
In the appendix, we outline the proof of Result 3.1.
(a)
n fixed and t tends to infinity:
From Remark 3.1, we know that
(A.1 )
r*2 = n(m~ - ~)LI[LSbbL,]-1L(m~ -~)
and hence the statistic
+
1 2
0p(t- / ).
A1so , reca 11 ,
Sbb = (n-1)
= (n-1 )
-1
-1
n
E (b. -b)(b.- b)
i =1
n
E
i=1
1
1
[~.
+ Z. - II
~
1
1
I
Z][~.
-
1
= S/3~ + SZZ + S/3Z + S'
/3Z
+ Z. -II-Z]'
1
~
(A. 2)
where
= (n-1)
-1
n
E (c. - c)(d. - d)',
i =1
and Z. is defined in Remark 3.1.
1
1
1
Since~.
1
variables with means
~
and Z. are independent normal random
1
and 0 respectively,
E[S/3Z] = 0
and the variance of the
var[(n-1)
-1 n
E
i=1
(/3.
1,~
(~,m)th
element of
)Z. ] = (n-1)
/3,~
1,m
- m
S~Z
is
-2 n
E
i=1
Page 18
Therefore,
S~z
(A. 3)
Now,
Szz =
(n_1)-1[~
Z.Z.'-
nIl']
=
i =1 ' ,
, = 0 p (t- 1/ 2 )
and Z
since from Remark 1 we know that Z.
Therefore, from (A.2) and (A.3), we have
=5
1 2
~~
+ 0 (t- /
P
(A.4)
)
Combining (A.1) and (A.4), we get under H :
O
A*2
T
L~
= ~o'
2
- Tm = 0 p (t -1/2)
where
n( ..~- ~)' L' [L S~~L']
-1
L(m~ -~)
.
2
2
Now, the result (a) follows because T has the Hotelling's T distribution
m
with (n-1) degrees of freedom.
(b)
t fixed and n tends to infinity:
From Remark 3.2, we know that the exact distribution of T*2 is chi*2
The difference between T
square with q degrees of freedom.
the matrix n var(b) is replaced by it's unbiased estimator Sbb.
show that Sbb is consistent (as n
Slutsky's Theorem.
~
A*2
and T
is that
If we can
m), then the result (b) will follow from
Page 19
From (A.2) and (A.3) we have,
Now
=
n
_1
(n-1)
Z.Z ~ - nZZ ']
[E
1
i=1
n
E
= n _1
i =1
= n _1
+ n
Z.Z~
1 1
1
-1-1
(n-1)
n
Z.Z~ -
E
1
i=1
1
(n-1)-1 n
ii'
n
1 1
E Z.Z~ + 0 (n- t- )
i=1 ' 1
P
(A.5 )
(A. 6)
Now, since
are iid N(O,
~i's
E~~)
variables, we have
(A.7)
Also, since Z.'s are independent N(O, a2(X~x.)-1) variables, we have
,,
,
n
_1 n
E Z.Z.'] = ni=1 ' ,
E[n
1 2
1
E (X '. X. )0
. 1
,=
'
(A.S)
1
and
Var[n
_1
~'
n
Z.Z~~]
E
i =1
"
(A.9)
for any arbitrary
Sbb
vector~.
-1 2 n
=E
+
=n
var(b)
~/3
n
0
+
Therefore,
E
(X~X.)
i =1
"
0 (n- 1/ 2 )
p
-1
+
0 (n
P
-1/2
)
Page 20
and the result (b) follows.
(c)
nt large and g=1:
Consider the t-statistic
for testing the hypothesis H :
O
= AO'
l'~
has a standard normal distribution.
~*
T
= T*
We know that the variable
To show that
[l'n var(b)l] 1/2(l'S lf1/ 2
bb
is (approximately) distributed as Student's t-distribution with v * degrees of
--1
freedom, we need to show that v * [l' n var(b)
l]
l'Sbbl is (approximately) a
chi-square random variable with v * degrees of freedom and is (asymptotically,
when nt is large) independent of l'b.
From (A.6), (A.S) and (A.9) we have
=5
~~
= S~~
+ 02
n
n- 1 E (X~X.)-1 +
i=1
+
1
~2 -1 n
1
n
E (X~X.). 1 1 1
1=
0
where ~2 is defined in Section 2.
x2 (n-1)
1
Note that (n-1)(l'S~~l)(lIE~~l)-1 is a
random variable and (nt-nk)~2/02 is a
x2 (nt-nk)
random variable.
Therefore, Sbb is the sum of independent scalar multiples of chi-square random
variables.
Ignoring the terms of order (nt)-1/2 and using Satterthwaite's
--1
approximation, we have that v * [l'n var(b)l]
l'Sbbl is approximately
distributed as chi-square with v* degrees of freedom.
Page 21
Now, to show the (asymptotic) independence of T* and Sbb' note that
b
= .~
+
Z is
independent of S~~ since ~i's are NID(~,E~~) and are
independent of {Z.}.
1
Also, for each i, the least squares estimator b. is
1
independent of the residual sums of squares
-2
and a
are independent.
y~y.
1
1
-
b~X~y.
1
1
1
and hence b
-*
Therefore, for nt large, the distribution of T
can be approximated by Student's t-distribution with v* degrees of freedom.