BIOMATHEMATICS TRAINING PROGRAM
THE ANALYSIS OF CROSS-SECTIONAL TIME SERIES DATA
by
Joao Gi1berto Correa da Silva
Institute of Statistics
Mimeograph Series No. 1011
May 1975 -- Raleigh
ABSTRACT
DA SILVA, JOAO GILBERTO CORREA. The Analysis of Cross-Sectional Time
•
Series Data.
(Under the direction of A. RONALD GALLANT and M. M•
GOODMAN.)
This study is concerned with the estimation of linear relationships from cross-sectional time series data.
The subject has been
extensively discussed in the econometric literature.
,,
The diversity of
the approaches proposed in the literature stems both from the
different sets of assumptions and the different estimation procedures
adopted.
Most of the alternative approaches are based on simpler
assumptions than the more realistic assumptions used here.
The
variance component approaches ignore the possibility of serial·
correlation in the time,direction.
The seemingly unrelated regressions
approaches aSSume q specific first order autoregressive error structure
and treat cross-sectional unit effects as fixed rather than random.
Two models are proposed to fit alternative situations.
Model A
aSSumes that the linear relationship is affected by a random
disturbance with three random components, namely
where
u it • a i + b t + e it '
is a
is a time invariant cross-sectional unit effect,
cross-sectional unit invariant time effect, and the third component,
e
it
' varies with both cross-sectional unit and time.
For the first
two error components the usual variance component assumptions are
adopted.
The assumptions for the third component take into account
the possibility of serial correlation in the time direction.
Model B
specifies the same set of assumptions except that the time invariant
cross-sectional unit effects are considered
fixed~
The serial corre lation among the disturbances within crosssectional unit is taken into account by the specification that the
residual terms
e
it
are a realization of a finite moving average
process.
The covariance matrix of the vector of observations has a complicated structure which
makes
the study of the asymptotic
properties of the two step generalized least squares estimator with the
covariance matrix estimated directly intractable.
This problem has
been overcome by substituting an approximating matriX, following
Hannan (1963).
This approximating matrix is a circular symmetric
matrix which is used in place of the covariance matrix of the
stationary moving average process,
It has been shown by Hannan (1963)
that in regression problems with stationary errors, under mild
conditions, the generalized least squares estimator with the approximating covariance matrix is an asymptotically efficient estimator for
the regression coefficients.
The proposed estimator for the regression coefficients is
essentially a Hannan's type estimator with the variance and covariance
components replaced by the corresponding estimators ob~tined by a
method suggested by Seely (1970).
The two step generalized least
squares estimator is shown to be unbiased and to have the same
asymptotic multivariate normal distribution as Hannan's estimator.
THE ANALYSIS OF CROSS-SECTIONAL
TIME SERIES DATA.
by
JOAn GILBERTa CORREA DA SILVA
A th~sis submitted to the Graduate Faculty of
North Carolina State University at Raleigh
in partial fulfillment of the
requirements for the Degree of
Doctor of Philosophy
DEPARTMENT OF STATISTICS
RALEIGH
1 9 7 5
APPROVED BY:
Co-Chairman of Advisory Committee
Co-Chairman of Advisory Committee
ii
BIOGRAPHY
Joao Gilberto Correa da Silva waS born April 18, 1938, in
Pelotas, Brazil.
He received his elementary and secondary education
in his hqme town, graduating from Colegio Municipal Pelotense in
1957.
He received the degree of Engenheiro-Agronomo from Escola de
Agronomia "Eliseu Maciel", Pelotas, in December, 1961.
He then worked
for the next three months as an agronomist in Cia. de Cigarros Souza
Cruz, Santa Cruz do SuI, Brazil.
time jobs:
In April, 1962, he started two part-
One at the Instituto de Pesquisas e Experimentacao
Agropecuarias do SuI, Pelotas, Brazil, as consultant statistician,
position held until August, 1970;
the other as Assistant Professor of
Mathematics at Faculdade de Agronomia "Eliseu Maciel", now integrating
the Universidade Federal de Pelotas.
In August, 1970, he started a full-
time appointment at Universidade Federal de Pelotas, and immediately
was granted
a leave for graduate studies.
He then entered the Graduate
School of North Carolina State University at Raleigh, where he
received the Master of Statistics degree in 1972.
He is now returning
to his duty at Universidade Federal de Pelotas.
The author married Miss Mercedes Santos Mascarenhas in 1964, and
th~have
three children:
Marta, born May 7, 1965; Marcus, born
April 25, 1973; and Marcia, born January 23, 1975.
iii
ACKNOWLEDGEMENTS
The author is grateful to Professor A. Ronald Gallant for
suggesting the topic for this dissertatiun and providing a
stimulating environment for the development of the research.
His
continuous interest and guidance are sincerely appreciated.
Appreciation is also extended to the other members of the
Advisory Committee, Professors Major M. Goodman, Francis G.
Giesbrecht, Thomas M. Gerig and Robert- f:\ilber
for the ir con-
strllctive criticism and suggestions.
Part of the research was performed TNhile the author was on leave
from the Universi.dade Federal de Pelataa, Brazil.
The financial
support was provided through a scholarship granted by Coordenacao do
Aperfeicoamento de PessoaI. de Nivel Superior (CAPES), Ministry of
Education, Brazil.
Both institutions are gratefully acknowledged.
The author expresses his speci.al gratitude to his wife, Mercedes,
and his children, for their tolerance, comfort and sacrifice during the
course of this study.
iv
TABLE OF CONTENTS
Page
L
INTRODUCTION • • • •
1
2.
REVIEW OF LITERATURE • .
5
2.1
2.2
Historical Notes
Proposed Approaches
2.2.1
2.2.2
2.2.3
2.2.4
3.
MODELS AND ASSUMPTIONS
3.1
3.2
3.3
4.
Rationale
Model A •
Model B •
ESTIMATION FOR MODEL A •
4.1
4.2
5.
Dummy Variables Model
Random Error Components Model
Mixed Model For Regression • • • .
Models With Correlated Error Structure •
Estimation When The Covariance Matrix is Known
Estimation When The Covariance Matrix is Unknown
PROPERTIES OF THE ESTIMATORS FOR MODEL A • •
5.1
5.2
Unbiasedness • • • •
Asymptotic Distribution •
5
7
8
13
25
29
41
41
46
51
53
53
62
69
69
72
6.
ESTIMATION FOR MODEL B • • • •
97
7.
COMPUTATIONAL CONS IDERATIONS .
105
8.
NUMERICAL ILLUSTRATION .
109
9.
ANALYS IS WITH UNBAIANCED DATA
129
10. SUMMARY AND CONCLUSIONS
134
11. LIST OF REFERENCES •
137
12. APPEND IX • • • • • •
143
12.1 Some Concepts and Results On Stoc~astic Convergence.
12.2 Summary of Some Special Concepts and Results of
Matrix Theory • • • . . • . • • . . • • • • • • ••
144
149
v
TABLE OF CONTENTS
(Continued)
Page
12.2.1
12.2.2
12.3
Inequalities • • • • • • •
Direct Product of Matrices
Proofs of Lemmas 5.1 and 5.2 • • •
0
149
150
155
1.
INTRODUCTIO~
Research in some substantive fields requires the analysis of data
collected at successive points in time a.'1.d across classifications,
forming what is usually termed
cross~sectional
time series data.
This
kind of data is very common in economic research, where observations
(~,&.,
are collected on units
during a period of time,
families, firms, geographic areas)
It occurs also in other fields of research,
such as in animal and plant research, when observations are taken on
individuals or groups of individuals at consecutive time points.
How-
ever, the relevant previous work whi.ch deals with the analysis of such
data is connected with econometric studies.
Given a sample of observations on
cross-sectional units at
A
T
time points, we postulate that the observed value of the dependent
variable on the
i
th
unit at time
t
can be expressed in the form:
p
~
k=l
(k = 1, •
0"
( 1.1)
. k+ u .
,
~t
It
t = l , ••• ,T
i = l , •• o,A
value of the
8 kx
where
x
I
is the corresponding
itk
~ independent (explanatory) variable,
p)
are unknown parameters and
u
it
13
k
is a remai.nder term
which represents the divergence between the observed value of the
dependent variable and the value which would be provided by the
postulated linear relationship if the parameters were known,
It is
assumed that the explanatory variables are nc.nstochastic and that the
AT x 1
vectors (x
llk
, •• x
1Tk
x
21k
o,
"X
AT
)
,
k
=:
1, ••• , p , are
linearly independent,
The problem considered in this thesL:: is the estimation of the
set of regression coefficients of (1 .. 1) under sui.table assumptions
2
fur: the remainder term
discussed in the econometr.i.c litec.'atore,
have been proposed, differing in LL.0
Ma:'.y di6tin.ct approaches
~:;p('ci.ficati.cn
of the remainder
In most cases the errDr t:crm is assumed to have a three
term
component structure:
+
a.;
.L
where
a
i
b
t
+
e"
~t
(1. 2)
,
represents a time invaria".t individual
r~ffect,
e
represents an individual invariant t tme effec [" and
it
b
t
re.prt2s'onts
an overall residual effect, varying over both individuals and time
periods,
In the main, the literature may he classified by these
f ·t~ons
·
f te
h
a
f our spec ~o~ca
(a)
. d'
?' :
e r ('1L 0-)
rema~n
dummy variables model, which aSSumes the remainder
termS
a
i
and
bt
to be constant and
(eit1
to
be a set of uncorrelated random variables with
common variance;
(b)
random error components mode 1, when
and
(eitt
(a.1
,
~
are aSSumed to be three uncorre1ated
sets of uncorrelated random variables with common
variances;
( c)
mixed model for regression, which assumes
be constant, and
(b
1 and
. t
a.
~
to
[e>l.t 1
correlated sets of uncorrelated random variables
with Common vari.ances;
3
(d)
models with
autoregn~.-i3iv,-, ~"1. (DC
ECl'UC.tC.r:c,
which
account for possible serial correlation of the first
order autoregressive type amo'lg the di:;tu:r:bances
of a same individual over time.
In this thesis we adopt two specifications which are extensions
of (b) and (c),
The correct specificati.:.m in a given instance
depends on whether the data iE asst.med to be a sample or a cenSU8 of
the cross-sectional units.
Equations (].I) under
Lh228
alternative
specifications are called, respecr.ively, MDdcl A and Model B.
The
extensions allow for possi.bility of serial correlation among the error
components
e
it
for a same indi'.:iduaL
It is generally recognized
that this is a more realistic assumption for economic time series data.
So, it is expected that the proposed models will be appropriate to
many situations of pooling of cross section and time seri.es data.
A review of the previous work on the analysis of cross-sectional
time series data is the subject of Chapceor 2.
A detailed description
of the assumptions for both the proposed models i.s presented in
Chapter 3.
Chapter 4 defines the estimation procedure for M·)del A.
the procedure is an adaptati.on of a
techni.qut:~
Briefly,
proposed by Hannan
(1963) for the estimation of regression equation parameLe:cs with
errors generated by a stationary p:t.·OCe,,'S.
Hannan's technique, which
approximates the covari.ance matric< by a ci.:ccula.r symmecric matrix,
provides asymptotically effi.cient 8stimiltors for che regression coefficient.s.
The proposed regression::o"-';t fi.cient estimators are two
4
step generalized least squares
Estimato~'8
with the covariance matrix
estimated by a method developed by Seely (1910b).
In Chapter 5 we show that the pro,'::edure pruvides unbi.ased
estimators for the regression
c()ef[icL~~~1X.3)
with the same asymptotic
properties as the Hannan's estimators.
Chapter 6 extends the theoretical
Model B.
r~8ulc3
of Chapters 4 and 5 to
Computational problems are considered in Chapter 7 with
special emphasis on methods of redad.ng the
computations.
s~o:ragerequired
i.n the
Chapter 8 illustrates the c3timation tech'lique by a
numerical example.
Chapter 9 extends the procedures proposed in
Chapters 4 and 6 to the situation of unbalanced data.
summary of the results contained in th,e main text.
Cb,apter 10 i.s a
Chapter 11 is a
list of the literature referred throughout the dissertation.
Finally,
Chapter 12 presents a summary of knmvfl concepts and results which
are used in thl? dissertati.on, and proofs of Some tedi.ous results used
in Chapters 5 and 6.
5
2.
REVIEW OF LlTERA'l'IJRE
2.1
H~stori2Ql
NotSE
-------,,---~-~--
In the past, it waS commOD.
pra(ti;~L
tc 03t.i.mate economic
relationships from data consisting of a single time series or a single
croSS section.
section data.
The time series
!_
a.,
gep~;raUy
'were aggregated cross
pi.one(~ring
Schultz (1938) in hi.s
study of
statistical laws of demand confines hi.:::; empirical work to aggregate
time series samples.
As might be expected, estimates obtaieed from data of two very
different sources - croSS section and time series - are usually
different.
It was argued that some paramc,:tExs
:3~tIOtlld
be estimated from
time series while others, by their characteris tics> should be
estimated from cross section_
Marschak (1939) suggests for the first
time a method to combine cross section ::tnd time series information in
demand studies.
Marschak (1943) state;; tLat
lI
poo ling" is the answer
to the discussion as to whether crOGS section or time series methods
of demand analysis are prefe!'able.
He th.en suggests a nine-step
procedure which pools cross section and time <;(',rie8 data, and
Il
u s es as
mu.ch information as possible to estim1te the i.ndividual demand
function".
Tobin (1950)
co~'tends
that
tilE::
consistent with both kinds of obscr·,,-atim.l.8.
dema:ld tU!1.ction sho':jld be
He as:oert2 that there are
botl:1. economic and statistical reasons fe'r basing quantitat.ive demand
analysis on a combinatinn of time series an.d cruss section data.
The
economic reason is the very nature and qilali.t.y of aggregate economic
data _
Statistically, wi.dening ':he scope of tr!£ observations on wl,ich
6
~:,tatL:OC:.i.cal
demand al'.alys:is
based
:]',3
the possibilities
i.:li:ICa'3E"S
of
rejscU.n.g false hypoth'=,sE:s an.dLmpLU'JE'8 t:he esti.mates of the parameters
of demand functions.
A13<) tne e;;ti.mat'L':,D of paramt:ters from ti.me
s':C1ies (; 2cuunters many s tac:::':, tLcal pi!...falls, sp\;;cia lly multih~s:"
coU.ineari.cy) a han.dicap 'Nf"dch appears
The"e were reasop.s which led
sect~ons.
t;)
seve.re in cross sections.
t.r1ct'eaEt:::d exploitation of cross
However, cross section data are, by themselves, in-
suffici';;;:it to tear: 'hypothesLs c,)!1c.ecning demand or to estimate all the
parameters in a demand fuw::tiGn.
SOn¥2 of tJ.:e rc,levant variables
change only with time and dIe Lc effect::; cannut be evaluated without
appeal to time series of agg:t'e.gates.
This waH the motivation for
Tobi.n (1950) to propose a procedu:re which estimates a parametric conL~om crU2S
straint involving income t,lasti.cit.ies
section data and
estimates the demand rel.ationship from aggrsgat:e time series data
subject to this constraint.
A simi.lar approach is adopted by Wold
(1953) and Stone (1954) also in demand sel.dies.
estimate from cross
:~ectior.
data tLe
They suggest to
cCJ~fficier1tB
aDDt 11er
change from one cross-sectional unit to
of vari.ables which
(income elasticity in
their case) and thsn, substituting thiE estimace into the structural
equation, to estimate the r.emaining cueffi:'Lents frcm time series
data.
However, doubts were still
compa-r.ab.ility of e.stimate"
aggregat'~,
:raLs~,d
about the suitabi.lity and
from diffE.:ru,t. k.LnrL, of data, mi.cro or
cross section or time ,:::erit"s.
'Tbf~
fle:"d for more adequate
beha\'::'uJ::al models ar.d the ir:cre.asing ava.;.labilit:' ut more detailed
cross SEction data
challcng,~d
thE; :1.ngenn1.r:y of many investigators to
7
develop be tter ways uf ut.iiL!..bg t.l
the ava.Uab18 time serie2 . .:Jata.
i.c~
.l.:ltc,r:'l1ldtion in cO!J.junction with
Kuh (1959) pre:.:.ent"
an assessment of
the differences of e:3cLmate:3 dccived f.:>::om ClOSS section and time
f;eries data,
ar~d
r:h"" effects .)£
agg:c~gat::.oc,.
He points out that one
rea:30n the €3timate;:: differ l,s t. 1lat .:r028 secclOCJ.3 typically reflect
lung run adjustment.3 \iii.':-!'ereab timE' ser L::-c cend to ref lee t short run
reactj,U!1s.
Also the bias.;;:'! fCJm excluded variables can be strikingly
predicl::;'OUS in oC'.e cop.cext based
another
conU~xt
O~l
bcha',:h:<c r81at:i.ons estimated in
is highly quesci/)nablt:,.
pos.i.ti')~l
L'Jat under::-tandi.':'.g t:he
CTCJS8
Lion requi.·reE :38quS.'ltia 1 observatiun;" on. the same individuals
3E:C
Ln, a n.umber of different time
88s8nU.al to che full
t:~m8
Kuh insists upon the pro-
fi6ries implications of a single
peri.~){js.
unde:'C~:;tanding
A rectangular array of data is
of the ;systematic interrelations
between thE different type::" of 8ampll2's.
Observations on individuals
at only one or a few points cf time will often be structurally incomplete because the observations on a given cross section are likely
to bE:, affected by pi'.iol' ob8ervativn~".
More impo·.ctantly,
from time serie,:; rC.3:iduals abc;:jt r,'he <::::'08,.,
as the most efficLmt: way uf 8i'timati':lg
8ectio~1
inferences
behaviur can be
E:1~on(jmi.crf;lacion,::;hip8.
Many
8
p0c'U.t',g c,f
secti"x'. ::l,">.d tIme setic:,"" data.
Cr.UdS
,..." a.
wbe-re
<'
~l.
,~:~divi.dual
a "reprf.;sents an
:l
indivi.dJal~
Mo;;t of t:hem, hmv8ver,
effect which is constanL throu.gh
at a given time point hut vgr:i.cs bu:ough timE) and
is a residual effect which differs among i.ndividual units both at a
point in time and through tLme.
The differences among the proposed
approaCL8S origi.nate frLlm bo:.h the diffm'ence,3 of basic assumpticI:,2
abuut the terms
used.
()f
the remaInder (2.1.) and th,e mE:th::>ds of estimatLon
.f-Jc;"18ver, muG t
of the p:t'oposed pn);;edu:L8s are '/ariations and
sGmet:imes combipations cf
~~ome
basis on8S .We w'i.11 discuss thesE' bash:
approaches and point out t'he distinctivE charactE,:ciatic3 of
alternati.'Je approaChes\A7DdTl rele\J::mt.
Th(~
fClr
t~'e
dummy var:i.able3 appr(j'.:lcb 3pscifi.es
t"':~ :La t
:cema inds r te rm (2.1) cd the
a
i
a rd.
i"'" 1
a
.1.
follue,ii.!J_g aS3umpU,ofl
ionsh i p (1.1):
b a r e fix€d paramef;e-cs
t~
A
I;
t~i.e
and
'1
2::
c~l
b
t
- 0
f'~:l(,}"
that
9
E
are random
it
varj~b18~
with
2
'" a e
dnd
Lf..L
~
j
t,8 '" 1.,
o.
0,
Tj
a.n.d by John.;:;on (1964) ia Li'.';;;
of d.ummy va:ciab.1.8;"
tL)
~8Limation
combine
CCOl'3S
uf
COdt
functiuns.
:sec li.on and time
.h:om uJ[lce.rn abol1.t pc.e.sib1.r:: bias in the
estimaLF~s
3t;ri.€S
The
Lese
data stemS
of causal parameters
Ihty alh'W th.e elimination. of the effect:.3 of diffen:nces among cr03S
'~SLU.ons
and ti.me petied:., fr'-;ffi the E:.f-tima te oF. L'he residual vari.ance
aDd the estimates of the l.'e.gor:ess:i.iJD cc'e fficients
0
Hoch (1962), con-
cernF,d ",ddt tbe e.stimat1.un. cf Cubb-Dc'uglas prud'lcti.,..:m function pararrK-:Lp;,Cs.
pO:l..c.ts out the £ollc"Allng adva:-J.t::ages of r.h.e dummy variables
mer:1:'od:
U)
Unbiased estima
U~2
of d'E r2gn,SS:l.of), coeftLC.:..ent:8
caE be obtained;
firm
(;("r:",'3ta:~.U~
te('!:-:.ni.cs,l ef ficicncy of
and 'year con.SCan::.:':; Cdn bt? ,)bt:ai.ned o
fi:Cill8,
whtlL dUft.'·csc,ccs
dmc:"g )'Sd-C' cOrtdtanL':
0
j,G
reflect weather differenced (in agriculture) and changes in proMU!ldlak (1961.), in an empirical sLudy of production
ductivity.
functions, uses a simplLfica tion of Assumption I which inc ludes dummy
variables only for firm effects as an. attempt to avoi.d manageme.nr. b Ia:',
He observes, however, that the estimate of managem8nt may contain
effects of other fixed inputs, such as climate, type of soil, etc.
In matrix notation, tb.e equation;: (L1) can be written as:
:L -
Xa
+ u
where
§. ::: (S 1 ••• S ) I
.
P
It i.s assumed that thE, first column of
X
is a vector of orw •
Then, (2.2) can be written in the form:
( !{
\
where
lAT
X :::: 0ATXl)
is an
and
AT x 1
a::::
\lector who,se elements are all
(S 1@'(1)>
f
•
An alternative. form is:
1
.~
0 •. ,'
11
(2.4)
where
Ct
== t'l
x
+ XI§.(l) ,
== Xl
~TX' , and
Xl
is an
1 x (p-l)
vector whose elements are the averages of the elements in the
corresponding columns of
Xl •
The dummy variables model can be fitted by the usual regression
techni.ques.
However, when the only concern is the estimation of the
regress i.on coefficients
Ii, a specia 1 procedure can be used.
Under Assumption I, the best (minimum variance) linear unbiased
estimator (BLUE) of
is the ordinary least squares (DLS) estimator,
Ct
(2.5)
Now, following Wallace and Hussain (1969), Swamy and Arora (1972),
and others, one can sweep out the constants
and
a.
~
b
for a given
t
sample by the covariance transformation represented by the matrix:
(2.6)
where
I
A
is the identity matrix of size
matrix whose elements are a l I I , and
We note that
Q
@
is an idempotent matrix:
A,
J
A
is an
A x A
denotes Kronecker product.
Q == QI
and
Q2 == Q •
Thus, the transformation matrix is singular with rank equal to
tr Q == (A-l)(T-l) •
Applying the covariance transformation to the equations in the
form (2.4), we obtain:
Q.l
= Q~~
== Qxi3..(I)
+ ~(l) + Q~
+
Q~
(2.7)
12
where the second equality follows from the fact that
transformation operating on the vector
a
i
and
b
t
u
QI
AT
= Q.
The
"sweeps out" the components
, so that (2.7) is equivalent to
(2.8)
where
!::. = (ell ••• elI' £21 ••• e
E(Q.~)
can be easily seen that
Q~
covariance matrix of
AT
), •
=Q
According to Assumption I, it
2
Var(Qe) = cr Q •
e
and
is singular.
Consequently the usual Ai.tken's
generalized least squares (GLS) estimator of
Since the covari.ance matrix of
Q§..
The
f2..(1)
does not exist.
is not scalar, application of the
~(l)
OIS procedure might lead to an i.nefficient eS timator of
•
Wallace and Hussain (1969) apply OL8 to (2.8) without providing proper
Swamy and Arora (1972)
justification.
justify the procedure by using
a result of Mitra and Rao (1968) that an unrestricted Aitken estimator
based on any generalized inverse of
Q
i.s a BLUE of
linear subspace spanned by the columns of
Qx
li(l) , since the
is contained in the
linear subspace spanned by the columns of the covariance matrix of the
disturbance vector
Q~
This shows that
is a generalized inverse of itself.
estimator of
a
~.(
Q
Ji(l)
l)DV
Since
Q
is an idempotent matriX,
based on a generalized inverse of
QQQ = Q
Then, a
Q
GLS
is
= (X' QX ) -lX' Qv •
(2.9)
""-
/.
The esti.mator
l2..(l)DV
is the same as the estimator obtained by
h
applying 01.,8 to equation (2.8).
is a BLUE of
.§.( 1) •
If Assumption I holds, then
Its covariance matrix is
li(l)DV
13
2.2.2
Random Error Components Model
Maddala (1971) comments that one common argument that is made
against the use of the dummy variables technique is that it eliminates
a major portion of the variation among both the explained and the
explanatory variables if the between uni.t and between time period
variation is large.
In some cases there is also a loss in a
substantial number of degrees of freedom.
Balestra and Nerlove
(1966) point out that the use of dummy variables wastes degrees of
freedom since one is not really i.nterested in the value of their
coefficients but only in the coefficients of the explanatory
variables.
Added to these there is the problem that rarely it is
possible to give meaningful interpretation of the dummy variables.
An alternative approach treats the components
a
and
i
b
t
of
the remainder (2.1) of the relationship (1.1) as random variables.
It
speci.fies the following assumption:
Assumption II.
The termS
a.
,
1.
b
t
and
~t
i
j
t
== s ,
= s, = 0
uncorrelated
are random variables such that
it
E(a.a.) == 0'2
E(e.)=O
t
e
1
=0
otherwise
(i,'j = 1, ••• , A ;
a
J
otherwise
ai' b
t, s =
if
i = j,
E( e. te . )
l
JS
t
1,
and
e
.... , T)
it
= 0
otherwise;
= 0' e2-if
...
are mutually
•
This assumptiun characterizes the random error components model,
which has been considered by Wallace and Hussain (1969), Nerlove
(1971b), :Maddala (1971), Swamy and Arora (1972), and Fuller and
Battese (1974).
Kuh (1959) uses a variation of the random error
14
components model \,;Ihi.ch omits the time effect
specify that the unit effects
uncorrelat"',d.
a.
1.
b
t
and does not
and the residual effects
He observe:3, however,
are
that the remainder' term should,
for the sake of completent-)ss, include time constant effects.
Balestra
and Nerlove (1966) and Nerlove (1967, 1971a) adopt similar error
structure itl mode.ls that include lagged values of the dependent
HO~7e\ler,
variable as regr8SS01's.
they aSSume that the individual
effects and the residual effects are uncorrelated.
Wit}:l
the
b
t
A
(t
the random error components model we estimate, instead of
parameters
= 1,
approach,
variables
.'"
T)
a.
l
(i
= 1,
1.
and
and the
T
parameters
which are estimated by the dummy variables
just two parame ters:
a.
" ' , A)
b
t
•
the variances of the random
As far as the estimation of the
regression parameters is c.oncerned, this procedure amounts to
extracting some information on the components of
Ii
from the between
unit and between time period vad.ation of the dependent and independent
vari.ables.
Maddala (1971) says that t.his is a strong argument in favor
of the use of error components models in pooling cross section and
time series data, si.nce that. source of information is often completely
eliminated wi.th the use of the dummy variables technique.
He argues
that we can also rationalize this procedure of treating the unit
effect.s and time effect.s as random by observing that these effects
too, lil<8 the overall residual., represent our ignorance of the true
situation, and there is no reason to treat one source of ignorance as
random and the other as fixed.
15
Accordi.ng to Assumption II, the covariance matrix of the error
vector, whi.ch we denote by
V, can be written ae;
(2.10)
Under the assumptions, the OL8 estimator,
unbiased for
1i.
R~
"'"OL3
= (X'X)-lX'v"'-
is
However, except i.n very special circumstances,
considered by Rao (1967), this is an inefficient estimator of
the variance components
cr
2
a
and
2
cr e
~.
If
were known, the BLUE of
would be provided by the GLS estimator:
(2.11)
°th
WI.'
0
covar~ance
assumed that
u
ma tXl-X
0
var(R."c)
= (X'V-lx)-l
t=v
Provided it is
has a multivariate normal distribution, (2.11) is
also the maximum likelihood (M£.,) estimator of
~.
Under quite un-
restrictive conditions, this estimator is
(i)
(ii)
(iii)
(iv)
function of a sufficient statistics;
conSistent;
asymptotically normal, and
efficient.
Wallace and Hussain (1969) show that with the model in the form
(2.4) the GIS estimator (2.11) reduces to the equations:
.,
Q'VC
::::: -AT":"
I' viAl'
and
"~(l)VC =
(X 'V-\) -IX 'V-1y •
-
(2.12)
16
Unfortunately, however, in practical situations the covariance
matrix is unknown.
The question is then whether it may be possible to
derive estimators having som8 or. all of those desirable properties of
the GLS ur. the ML estimators.
An obvious suggestion is to use an
Estimator of the form (2.11) wi.th some convenient estimator of the
covariance matrix as substitute for
V.
have been proposed in this direction.
Several distinct approaches
Most of them are justified by
unbiasedness and a:,ymptoci.c properties.
If the errors
tIit
observablE:~,
were
the analysis of variance
estimators would be
- -'I1
1
-2
a a '" T(A-I)
A
L:
i=1
'I
-2
1
a b '" A(T-1) L:
t:::l
1
("T
u.
(1A
u
1
~.
AT
·t
AT
1
)
u.
2
~.
11
)2
- T1 a-2e
11
)
2
1 -2
-a
A
e
whe:ce the dot in the subscript indicates the summation taken over it.
In matrix notation, we have
a-2 e -
~
1
,-~-....,...
(A-I) (T-l)
,u_' Q_u
(2.13)
(2.14)
(2.15)
17
where
Q
1.3 the matrix defi.ned in (2.6).
Graybill and Hultquist
(1961) prOIlE that, under normality, these estimators are best (minimum
variance) quadratic unbiased estimators (BQUEs).
However,
(1969)
u
~mgge8t
is usually unobservable.
Wallace
and Hussain
to replace it by observed residuals obtained by OIS
applied directly to (2.2), i.~., by
u =
Replacing
u
. (X' X). -L"'-J' 2d.
[I - X._
.l\.
in equations (2.13) - (2.15) by its predictor
u = M.u
we obtai.n
(2.16)
(2.17)
Wallace and Hussai.n show that, if the residuals
distributed and the matrices
X'X/AT
finite posit.ive definite matri.ces as
every possible way, then the
eS
and
u
X'qx/AT
A and
it
are normally
both converge to
T tend to infinity in
timators (2.16) - (2.18) are
asymptoti.cally unbiased and consistent.
Tl:ey use the same assumptions
to prove that the estimator
(2.19)
18
,....
V
where
is the matrIx defined i.n (2 10) wi.th the vari.ance components
0
substi.tuted by the correspundhlg estimators g i.ven in (2.16) - (2 18),
0
·h as
th
.. e
f .C' ·l·j
.
• .. ow~ng
.
1/ :
propert].e[.:;e~
~(l)WH and the GIB estimator
(1)
~(l)VC ' given in (2.12),
have the same multivariate normal limiting distribution.
(ii)
The expectation of
to.r
(i.ii.)
CO~.1Vtrge t,) the same set of moments as
A
t;.,(l)WH
th08~
and the covari.ance matrix
;i:l
~(l)WR
of the limiting discribuLion.
The estimator
§'O)WH
is asymptotically more efficient
~(l)OLS.
than the alB estimator
Indeed, the OLS
esti.mators have unbounded asymptotic variances.
Wallace and Hussain (1969) also show that under the assumptions pre"
li(1)vc.:
viously referred, the GLS estimator
fie 1)DV
estimator
a.
.L
and
b
t
and the covariance
' given in (2.9), which regards the error components
as fixed for computational purposes, are asymptotically
equivalent i.n terms of their first and second moments.
the properties of the estimator
covariance estimator
Ji(l)WH
12n(l)DV
i:l, )
~·tl
:l
\An
Hence, from
it can be concluded that the
and the Wallace-Hussain estimator
have the same limiting covari.ance matrix.
Amemiya (1971) suggests that
11
be predicted by residuals
obtained by applying OlS to the mode 1 with dummy variables,
f":;;
u ""
·1/
1.. -
Q,DV~_T
l.~.,
by
- X§.( l)DV
_-:. All these a::;ymptotic results could be derived without: the normality
assumption. However, as Wallace and Hussain observe, it would
then be necessary to ma.ke a larger number of lesser assumptions
about the moments of the error distributi.on.
19
whet'.::
§.O)DV are the covariance estimators given in (2.5)
and
O'nv
and (2.9); hence
~
11
["T·
l"J
-I
-AT - AT 'AT - X(X 'QXJ X 'Ql..L
-
~2
aa
Let us denote by
and
~2
the estimators of
ab
2
O'b ,n:spectively, obtained hy replacing
(2.13) - (2.15).
(2.18) with
by
M,u
1"~
2
aa
and
in equations
Then, these estimators have tbe expressions (2.16)
replaced by
M
u
2
ae
M
l
•
Amemyia (1971) shows that
has the same asymptotic multivariate normal distribution
a-2 )
b
•
Moreover, he states that
di.fferent asymptotic distribution and hen.ce its asymptoti.c covariance
by a positive definite matrix.
He
then argues that it is advisable to use esti.mators of the variance
components based on
rather U',an
, and
the \:."stimators
R::l2
ab
Mu.
Amemyia shows also that
have thE same asymptotic distri-
bution as the M:L estimators.
Swamy and Arora (1972) show that, under the set of assumpti.ons
adopted by Wallace and Hussain, there are an infinit.e number of
estimators which have the same asymptoti.c distribution as the GLS
estimator.
estimator oE
They also show chat it is not possible to choose an
@.(l)
on the basis of asymptotic effi.ciency without
bringing in small sample considerations.
They develop an alternative
estimator from t1:>.e class of asymptc',tically efficient estimators and
show that this estimator is 1e8s effi.cient tJ:-lan the OIS estimator if
20
the sample siz.es
small.
2
T
and thE true values of
(J
2
a
and
are
It is also less efficient than the covariance estimator i.f the
sample SiZ8S
(J b
a.Q.d
A
and
A
I
are
small and the true values of
(J
2
a
and
are large.
Nerlove (l97lb) derives the characteristi.c roots and vectors
associated with the covari.ance matrix of the di.sturbances in the random
error components modeL
From the knowledge of tb.e characteristic
roots and vectors of the covariance matrix, whi.ch Nerlove denotes by
(J
2
n,
the spectral decomposition of
[2
or
V,
or any power of them,
can be deri.ved; ~"&.,
,...2
V = ('"'
e
,...2
+
I(J
2
a
+
A_2)
nub .M
00
2,
(2.20)
+ ('"' e
+ 'I(J
).M
a·
l
where
.M
M.,
.I."
Using the spectral decompositi.on form of the covariance matrix in the
express ion of the GI,') estimator of
.@.., NerlovE derives a number of
interesting results, snell as the rationale behind the greater
21
efficiency of the GLS eslimatc)I's in c.omparisonwith OLS and covariance
estimators.
Ner10ve also suggests that GIS estimators can generally
be found more simply computationally by first transforming Y...
and
X
to
= [2 -1/2Y...
and
respectively, and then estimating
by the OLS regression of
@.
Y...·k
on
Henderson (1971) calls attention to a simple, direct method of
obtaining the inverse of the covariance matrix
V.
He also derives
interesting comparisons between the GLS procedure and the covariance
method (which regards
purposes).
a.
and
1.
b
as fixed for computational
t
He first writes the model in the form of a general mixed
model,
x. ::
~
+
Zv
+
e
matrix of zeros and ones.
Henderson Shows that the GLS estimator of
A
§.
is the· so 1ution
XiX
§.VC
to
n"
X'Z
I~IZ
- jI
i tl.VC
=
Z'X
l
Z'Z + D-
v
-l.
lz,~
(2.21)
22
where
Lo
On the other hand, if
v
is regarded as fixed, the GIS normal
equations have the same tor-m as equations (2.21) with
from the matrix on the left-hand side.
with
a e2/a a2
2
and
2
aela b
a"-/a 2
')
to infinity.
v
D
-1
-1
omitted
is diagonal
l,n the diagonal, the GLS esti.mator becomes
the covariance e.stimator as
other extreme,
Then, since
D
e
a
and
a 2/a 2
e b
not inc luded in the mode 1,
tend to zero.
2
2
a e /a a
At the
and
tend
Henderson argues that, unless either of these limiting
cases is a close approximat.ion of the true situation, it seems
appropriate to estimate the variance components and use GLS with these
estimates as substitutes for the true unknown variance components.
Henderson remarks that the use of estimators of variance components
based on least squares resi.duals on the grounds of having desirable
asymptotic properties is not a very convincing one.
He argues that
these should be regarded as minimum acceptable properties, and that
one should try also to consider small sample properties.
He then
suggests that the use of the fitting of constants technique might
yield better estimates.
Fuller and Battese (1974) suggest an alternative GLS estimator
with estimated covariance matrix.
Their results are more general than
those appearing in. the previous ly referred literature in that
23
( i)
they explicitly consider the cases where
cr
(it)
2
b
cr
2
a
and
are equal to zero;
they do not require that the matrices
X'QX/AT
X 'X/AT
and
have positive definite limits.
Thus, for example, their theory is applicable if the relationship
(1.1) contains a time trend as one of the explanatory variables.
Fuller and Batt.ese estimate the variance components
and
cr
2
by the fitting of constants method suggested by Henderson
e
(1971).
They show that the fitting of constants procedure provides
the following unbiased estimators:
(A-l)(T-1) - rank(X'M
12
X)
(2.22)
(2.23)
(2.24)
where
M
M.
2
,
MI.
and
Ml2
are the same as in (2.20),
24
and
A+
denotes a generalized inverse of
A
,,':"1.
'J/
:::~
Qo
Fuller and Battese
observe that t.he variance componer'.t estimat.ors (2.23) and (2.24) are
not guaranteed t.o be non-n8gative and suggest that, in practice,
negative values should be replaced by zero for the estimation of the
regression coefficient parameters.
Fuller and Battese show that the estimator
A
V
where
(2.24)
is the matrix defined in (2.10) with the estimators (2.22) -
as substitutes for the corresponding variance components, has
the following properties:
(1)
aFB
is unbiased for
l2.,
(a)
u
the errors
if
it
are symmetrically
distributed about zero and have
finite fourth moments, and
(b)
(ii)
~FB
the expectation of
exists.
has the same asymptotic di.stribution as the
GLS estimator
-~'V"
a
~
A
(0- e2 )-1
.,
1
and
b
and
t
T
\~
it
, prdvided the error components
are normally distributed and
are strictly increasing functions of
AT •
'!:../ We
have corrected an erre'r in Fuller and Battese I s paper in the
expressions of cr~
(?~ and
at .
25
2.2.3
Mix•.od Model For ReKression
Hussain (1969) propO:3es a model which is a hybrid of the dummy
variables model and the random error components model.
He observes
th.at in some econome.tric studies a relationship of the form (1.1) is
postulated and that data are available for a finite number of cross
sections over a very large number of time periods.
He argues that in
suell si.tuations it may be assumed that the cross-sectional effect
is constant and the time effect
b
is random.
t
a.
~
To illustrate the
situation Hussain refers to data of income, employment, import and
export, number of commercial enterprises, etc., which are available
for each of the two wings of Pakistan, East Pakistan and West
Paki.stan,which can be used to study relati.onships between various
economic variables.
One may, for example, be interested. in the effect
on income of variations in public investment and private investment.
Hussain asserts that, as in such a case the variables of interest show
significant variation from one wing to another, and the number of time
periods for which data are available is very large, one may aSSume that
the cross-sectional effects are constant and the time effects are
random.
In a rde r to re view the thea ry
0
f Hus sa in's mode 1) we ma ke the
following assumption for the components of the remainder term (2.1) of
the relationship (1.1):
Assumptiqn .111.
The term
a.
1.
is a fLKed parameter such that
A
L: a.~ = 0
i=l
26
b
and
t
are uncorre1ated random variables such that
e.
J.t
= E(e,J.t )
E(b)
t
2
E(e, e. ) =cr
J.t JS
e
E(b b )
t s
0
::
if
i
t
j
::;
::
:2
crb
s
=:
t = s
if
::
::
0 otherwise
0 otherwise
(i, j
= 1,
o •• ,
A; t.s :: 1, ••• , T) •
Hussain reparameterizes the mode 1 in the form (2.4) by writing
=
a~'(
i
Qi
+
i.
a.,
J.
= 1.,
••• , A
Then, if we let
= bt +
Wit
e
it
t = 1, " ' , T) , the equations (2.4) can be written
(1 "'" 1, ••• , A;
in t.he form
Ca i ';; +w
= Z.§.
C:: diag(1r ••• ~)
where
w
=:
and
(2.25)
+::!..
(W
n ... wlT
w ••• w )
Z1
AT
::; (R I ,a"'(')
•Q.
~(1)--
(a)
X'X/I
v
AT x A
Z :: (XC)
I
matrix,
is an
AI x (p+A-1)
matrix
The following additional assumptions are made:
converges to a finite positive definite
matrix as
(b)
is an
p < T/2
I -+ 00,
the error components
distributed for
i
b
t
+
and
= 1, .oo,
1
e
A
A ~ 2 ;
and
it
and
are normally
t
= 1,
••• , T •
From the assllmptions i.t follows that the covariance matrix of the
error term
w
is
The GIS estimators of the parameters
!l
and
~-(l)
are given by
27
(2.26)
and
.12'(1)
= [XiX
- r-\'(1A
~JJT)Xf\Xi
- T-\i(IA@JT)].l·
(2.27)
ThE:' Esti.mator
can be obtained by a covariance transformation
I
represented by the matrix
CDnstant.s
i
".. 1,
•.
AT
fA
0,
- r-
1
CIA [~.)T)
These OIS esti.mators are unbiased
0
and consistent but g£nerally i.nefficienL
and
, which sweeps out the
2
cr e ' were known, tt-!.e BLUEs of
If the variance components ,
and
fi( 1)
would be pro-
vided by the GIS procedure, which yi.e lds
and
n
fi( l)MM
= [X 'X
-
I.T
X' (I
A
(ill J 1JX.
2
.
cr b
-1
.~-_._.-~- X' (J
I )X] ·[X'
2
A_ 2
A l":J T .
cr e + "'" b
fX\
(2 28)
0
However, si.nce the variance components are usually unknown, the GLS
procedure is not operationaL
Hussain suggests a two-stage
estimation procedure, whos\.: fLest stage consi.sts in constructing the
esti.mators
=2
cr
and
e
(2.29)
28
=2
O'e
+ Aa=2b
=:
1
1.@"'1.
--:.~
virl.A
~'(JA x I I T) - A'r~, .lA'.,T' T-p .01.0.
(2.30)
where
Q
is the matrix defined in, (.2.6).
Hussain shows that, under
the assumptions, the esti.mators (2.29) and (2.30) are unbiased and
consi.stent.
again.sr
F
HI:
He then suggests a test for the hypothesis
2
O'b > 0
distribution with
In case
H
O
ba:3ed on the ratio
T-p
and
(~2e
2
+ ,A;b ) /;2
(A-l)(T-l) - p+]
e
H :
O
0'
2
b
=0
which has an
degrees of freedom.
is true, the OL'3 estimators (2.26) and (2.27) are BLUEs
and hence we do not have to look for any other set of estimators.
For the case
-ff' 0
is rejected Hussain proposes the following estimators:
and
'a'O;k=l,-l C '(
Xa
)
-H
~ - ~(l)H
is obtained by substituting the estimators
and
for the c\Jrresponding parametric express ions in equati.ons (2.28).
Hussain (1969) proves that these estimators were unbiased, consistent,
asymptotically normal and asymptotically more efficient than the
corresponding 01,5 estimators.
29
Tbe assumption. made concerni.ng thE:' dummy variables model imply
a spheric form for the covariance stcucture, while those made for the
random error c0mponents model and the mixed model amount to assumptions
of very specific forms of serLal correlation.
These assumptions are
open to criticism.
Nerlov.:: (1.97la), referring to the random error components model,
says that one may argue that it 1.5 simply an approximation to a more
realisti.c model which allows rather pr'-l'.1ounced serial dependence among
disturbances for the same indivi.dual, but only a negligible amount of
dependence among distrubances for different individuals.
He asserts
that to achieve a good approximation, it may be necessary to allow,
for example, Some negative serial correlation among the disturbances
8
lt
for the same individual to counter balance the exceptional rigid
formulation involving the random effect
a. , assumed to persist for
1
all time periods over which a given individual is observed.
Balestra
and Nerlove (1966) remark that the assumption that the random
compone n ts
a.
1.
are independent among themselves for di.fferent
individuals may be very dubious o
As an example, they mention
that~
if individuals are geugraphic areas with arbitrarily drawn boundaries,
a8
Ln thei.r case, it cannot be expected this assumpti.on to be well
satisfied.
Nevercheless, they adopt it.
KmeEta (1971.) 8uggestE an alternative approach to the
SpEC ification of the behavi.ur (»f the disturbances that combines the
assumptions frequently made about cross-sectional observations with
those that are usually made when dealing \\lith time series.
As for the
30
cross-sectional obsEocvatlc'Es ... for <:xample, observations on individual
househo l.ds at a poi.nt of time - it i.s frequently assumed that the
regression disturbances are mutually independent but heteroscedastic.
Concerning the time series data, Kmenta says that one usually
suspects that the di.strubances are autoregressive though not
necuisacily heteroscedas tic.
Kmenta argues that, when deal ing with
pooled cross section and time series observations, we may combine
th~se
assumptions and adopt a cross-sectionally heteroscedastic and
timewise autoregressive. model.
Essentially, the approaches tlla t have been used to take into
account possible correlations
a~)ng
the disturbances, including the
one suggested by Kmenta (1971), are adaptations of the method of
seemingly unrelated regressions, originally developed by Zellner
(1962), or of its extensions.
Brehm and SaVing (1964), in a demand
study with states a6 cross-sectional units, use Zellner's technique to
account for the expected positive autocorrelation of the disturbance
termS for a gi.ven. state.
Blai.r and Kraft (1974) use an adaptation of
an extension of Zellner's approach in a model that includes dummy
variables for the three years of study to allow for the possibility
of technological progress.
In Zellner's original formul.ation and in itB further extensions,
the "seemi:o.gly unrelated regressi.ons!l are assumed to have distinct
sets of regression coefficients.
In the discussion that follows, we
adapt the models and the estimation procedures to the aituation of
pooU.ng of cross section and ti.me seri.es data, where the set of
31
regres;.,iIJI1 coefficienU, is the same for all regressions.
The proofs
follow along the 3ame lines as the ones for seemingly unrelated
regress .ions.
To put the equationl (2.2)
related regressions, we write
them in the form
i
X•.,
where
(
.L
Yn"' Y iT
) i
It is assumed that
1,
'00,
A.
X
1.
E(u.)
'~l.
= 1,
= (x
::'~il"
~
0
~
(2.31)
••• , A ,
and
x)
'~iT
i
an
E(u.u~)
d
~i
= a.
=
(
un" .u iT
,IT'
i,j
~l.-Jl.J
) ,
=
Thus, che. covariance between the disturbances of
.th
observathYDS on the
l"~
to be constant over time.
further assumption that
matrix as
'
in the framework of seemingly un-
T
~
and
,th
r-
cross-sectiona 1 units is assume d
For the asymptotic results we make the
x'x/r
converges to a positive definite
co
According to the
assumption8~
the covariance matrix of the vector
of residuals is
where
is the
AxA
matrix with
.. th
l.J~."
element
(J.,
1.J
•
The
inverse of the covariance matrix is
If
were known, the GLS estimator of the vector of regression
paramEters would be
(2.32)
•
32
This estimator is the BLUE of
£2.,.
F'urther, with the added normality
assumption, it is also the MI. estimator.
However, since
L-
A
is
usually unknown, the estimator (2.32) cannot be used in practice.
estimator for
~
A x A
matrix with
~'DLS
and
is provided by the expression (2.32) with the coL-
variance matrix
replaced by
.. th
~J-
I:
= SA
Gf) IT
,where
R
=
SA
is the
element
~,i..!:..,
is the single equation 018 estimator of
"'"OLS
An
(X'X)-lX'v
~
0
Along the same lines as in Zellner (1962) , it can be shown that the
~,
two-stage GLS estimator
i.o~.
~Z
given by (2.32) with
L-
A
replaced by
SA
,
has the same asymptotic distribution as the GLS estimator (2.32) as
T .... 00.
It can be shown, followi.ng procedurE:\ of Kakwani (1967), that
the estimator
~Z
is unbiased for
.§., provided its expectation
exists and the disturbances have a continuous symmetric distribution.
As we can see, the trouble with the use of this method in pooling
cross section and time series data is that
(a)
it allows for contemporaneous autocorrelations,
but not for serial correlations;
,
33
(b)
U~
assumes a fixed number of
cro~~,,:-8ectional
units
(i.e.) seemingly unrelated regressions).
It is obvious that a simple interchange of indices would partially
solve che first difficu] ty, a llowing for autocorre lation among the
disturbances for a same cross-sectional unit.
This is apparently the
approach adopted by Brehm and Savin.g (196!.).
J-1owever, then the number
of time points is th.e one to be conc:;idered as fixed, so that the
propcn~.i.c3
asymptcltic
are valid just for the sample size increasing
with t.he number of cross-sectional units.
This assumption of fixed
number of time periods is generally very unreasonable in the pooling
of crosS :3ection and time series data.
Parks (1967) presents an extension
ot Zellner's method that
alldws for both contemporaneous and serial correlations.
\)f
u.
the disturbance vector
The elements
in (2.31) are assumed to be generated
~:l
by the stationary first order autoregressive process
i. = 1,
••• , A ,
(2.33)
where
are random vari.ables such trwt
en
E(e. e: .. )
:=
l,t Jd
0'
ij
if
t,s ::::; 1, ••• , T) •
t
::::;
b
,
s-t > ()
E.(l''-'·'l-' u '0 )
J.. J
otherwise
=,.,.v..
1(1.
I.J
S;
0
t,s -= 1,
o
if
(i,j ;; 1,
it is assumed that
a ·nd
~
0
e it )
::::;
0
(i., j
::::;
1,
This assumption implies that
t-s/(·l
B(u. u. ) ::::; a .. p
.-p.p . )
. l.t JS
~ J 1.
1. .J
if
::::;
E(
s-t
and
••
0
)
A ;
B(u. ) ... 0
it
: : ; a .. p ~-t/(l - PiP j)
1J J
0
0
,
T) •
For convenience,
i.s a random vari.able such that
- 1-'"
- p '),
.1. J
and
i,J'::::; 1." •••.. A.
For the
34
asymptoti.c. results we assume that
definite matrix as
T -+
X.'XjT
converges to a positive
so.
According to the error sp',"cification, it follows that the
c()variancE'; matrix of the disturbances
=
If
1J.
11:
has the expression
O'uPn
P
0' 12 12
0' lA PIA
0' 2/21
0'2:/22
P
O'ZA 2A
0'Al PAl
cr A2 A2
(2.34)
p
P
O'AA-AA
where
2
Pj
1
p.
1
1.
p.
= I-p. p .
Pj
T-l
Pj
Pj
T-2
Pj
:I
T-3
P.
(2.35)
,--~_.,
lj
1.
J
2
p.
Pi
T-l
P 1..
T-2
Pi
1.
J
T-3
Pi
If tb.E; covariance matrix were known, the BLUE of the regression coefficient8 would be provi.ded by the GIS estimator given by
expression (2.11) with
U
defined in (2.34) as substitute for
However, the covariance matrix is usually unknown.
V.
A consistent and
asymptotically efficient estimator is provided by the three-step
estimation technique which follows.
parameters
Pi
In the first step we estimate the
of the autoregres,iive process (2.33).
For that we
apply the OLS method to all of the pooled observations and compute
the residuals
The parameters
p,
1.
can be
35
consistently e.stimated
equations
u
:il,
= p
it
1 L,t-
T
.:...
L:
=
PiT
+ ~ ._i t ';
1
ttl is gives
~',
u,
t=2
the bast<3 of the OIB appli.ed to the
OD
U
.it
i,t-1
. . ."
(2.36)
=~._.---"---_._--=--~
L
t=2
_,?
u~
1,t-1
Next, we use these estimate;, of
the
Pi'
.1. "" 1, ... , A , to transform
observations into
AT
(2.37)
where
-,
yit = Yit
- PiYi,t-l
x';','
- pox,
1 k
itk = x';tk
L
1 1,t- ,.
k ;:::; 1,
••• , p
~
-p.ll.
1
.it
1 1,t-
=u,
i
= 1,
000,
A;
t
""
:2,
00",
OL8 to t.hese transformed
T
0
A(T-l)
In tbe second stage we again apply
observations;Y and compute a new
r.
set of residuals
variances
cr..
ij
Then, the contemporaneous variances and co-
are estimated by
1.]
:,;,; '!':
u"~'.
:;
1
~.~-~-
T
L:
T-p-l t=2
(2 38)
0
l/Parks (1961.) suggests a va.riation of this approach that avoids the
108s of one observation for each cross-sectional unit.
36
The estimatc:\rs (2.36) and (2..38) are substituted for the corresponding
parameters in (2.34) and (2.35) t.o provide a consistent estimator
for the covariance matrix.
LJ
Finally, 1.rl the third step, the estimator
(2.39)
i.s cumputed.
Fo llowing along lines completely ana 10gous to those i.n
Parks (1.967),
i.t can be shown that the estimator (2.39) is consistent
@.
for
and, moreover,
the GL:3 estimator as
Ii
"
fl_.ir'
has
T -+ co.
same asymptutic distribution as
tIll:!
The estimator
....
ftp
is also unbiased for
if Kakwani's conditions are satisfied.
The procedure can be simplifi.ed by applyi.ng the GLS technique
to the transformed observa tions (2.37) wi.th the covariance matrix
substituted by
'U~,~
,
the estimated
of the transformed residuals
UA
is the
A x A
fd.~r<,
matrix with
A(T-l) x ACT-I)
where
ij th
·U~'~
covariance matrix
= UK ®1 T_ 1
and
element given in (2.38).
This
alternative procedure yields the modified Parks' estimator
(2.40)
with asymptotic covariancE. matrix
(X"r uU7(
-1.
X~'~)
-1
•
The two esti.mati.on procedures differ because the first one uses
AT
observations whi.le in the later we drop one observation from each
cross-sectional unit berort: using the GIS techniqUE; with estimated
covariance matrix.
same.
Howe.ver, their asymptotic properties are the
37
The approach suggeste.d by Kmenta (1971) - cross-sectionally
heteroscedastic and timewise autoregressive model - is a
specialization of Parks' extension of Zellneris seemingly unrelated
regressions to the pooling of cross section and time series data.
It
aSSumes the absence of contemporaneous autocorrelations but allows
Eor heteroscedasticity of the disturbances of cross-sectional units.
The basic assumptions for the autoregressive model (2.33) become
(i, j
= 1,
o
0
E(u. ) == 0
ll
•
,
E(e:it€js) == a.1.
t,s
A
(i,j
,- 1,
o
Q
i
i.E
0
2
=cr./(l
1
2
P1.)
,
j
I
.
l t JS
= 1.,
000,
1
A ,.
1
t,
1
S
if
i = j
== 0
,
t == s
== 0 otherwise
These assumptions imply
T)
,
= l,
is a random variable such that
iO
-,
2 \ t-s
2
E(u.u.)=a.p.
/(l-p.)
and
lt
otherwise
2
and
E(€it) == 0
E(u
0'"
iO
otherwise
T)
) = 0
if
0
i
=j
=0
We assume that
and
E(u·OU· O) ==
1
(i, j
1,
J
... , A)
0
Then,
the covariance matrix takes the form
2
a I Pl
U
l
where
-
0
o
2
o
0
a 2 PZ
o
o
(2.41)
38
')
T-l
<.,
1
p,
Pi
Pi
p.
1
p.
Pi
2
Pi.
p.
1
P 1..
~
1.
T-2
J.
~
T-3
1
As in the previous case, there are two consistent and asymptotically
efficient procedures wben the covar lance matrix is unknown.
The
procedures are analogolls to the correc;ponding ones described above,
except that now the covariance matrix has the form (2.41) and
expression (2.38) is replaced by
s
2
T
1
---
i
I:
"U":k 2
it
T-p-l t=2
A further specialization is obtained when we assume the
disturbance vectors
u.
-:L
in (2.31) to be generated by the same first
order autoregressive process,
i ..t=...,
when
Pi = p ,
i
= 1,
•.. , A
Then, the covariance matrix of the vector of disturbances has the
form (2.41) with
P.
L
:=
P ,
i
= 1, " ' ,
A , defined by
1
P
P
P
1
P
T-l
T-2
p
=:
l~
T-2
T-I
P
1
39
The esti.mation of the regression. coefficiEnts proceeds in exactly the
same way a2 in the previous case, except that formula (2.36) for
p.
1.
is replaced by
r.
• ....
u.I t u.1.,t-.
1
P =
i=l t=2
A
T
",2
I:
L: u.
1
1:.1 t=2 1., t- .
and the transformed variabLes in (2.37) are obtained by using
instead of
P 1.. •
The procedure which led to the estimator
p
(2.40) in
the previous caSe can now be carried out by a second transformation
of the set of observations.
The set of
A(T-I)
observations
corresponding to (2.37) is transformed into
. (2.42)
where
')
X"k'i'r
Uk
==
X'"k
itk
k
/ S ""'
i
=
1, ••• , p
= E:~:l.t Is.2.1
i = 1,
••• , A
to the set of
t'" 2,
A(T-I)
> •• ,
T.
Then the 01,8 procedure is applied
transformed observations in
the same estimator defined in
(2.42) to yield
(2.40) apprupriate to the present case.
40
The later procedure for the cross-sectionally heteroscedastic
and tlmewise autoregressive model is used by Blair and Kraft (1974).
Zellneris original procedure and all further extensions and
varl.ations of it are theoretically jus ti.fied in termS of unbiasedness
and asympt.oti.c properties.
Other small sample properties are
investigated by Zellner (1963), Zellner and Huang (1962) and Kmenta
and Gilbert (1968, 1970).
As pointed out above, one difficulty in
the application of Zellner's original method of seemingly unrelated
regressions and its extEnsions to the analysis of cross-sectional
time series data is that all of them aSSume fixed number of crosssectional units, so that the asymptotic theory holds for the sample
size increasing only in the time direction.
The objection is that
this treatment of asymptotic properties is not appropriate when crosssectional unit effects should be considered as random.
41
3.
MODELS AND ASSUMPTIONS
3.1
Tht:' random variables
Rationale
-~._-
uit
which disturb the relationship (1.1)
are generally supposed to YE;sult from observational errors and the
omission of numerous indivi.dually unimportant but collectively
significant determining variableswhicb have been omitted from the
analysis.
If the mathematical form of the postulated relati.onship is
not appropriate, the remainder term will also contain systematic
effects due to the inadequacy of the relationship.
When one con-
siders a pure cross section and fits, for example, a simple linear
relati.onship to the data s it is standard practice to aSSume that the
measurement error and the large number of factors whi.ch affect the
i.ndividual s in the sample and the value of the dependent variable
observed for each of them, but which halVe not been explicitly included
as independent variables, may be appropriately summarized by a random
di.sturbance.
Hhen time series data are conside red a similar argument
is made for the i.nclusion of stochastic disturbances in the relation
to be estimated.
However, when numerous i.ndividuals are observed over
time the problem of specifying the st.ochastic nature of the disturbance
becomes conceptually more difficult.
It. is clear in the abstract that
some of the. omitted variables \vill represent factors peculiar to both
thE; ind i,vidual units and the time periods for which observations are
obtained; other variables rEflect i.ndividual differences which tend to
affect the observations for a given i.ndividual in more or less the
same fashLon over more than one, and perhaps all peri.ods of time;
42
still other variables may reflect factors peculiar to specific time
periods but affecting individual unitE more or less equally.
considerations suggest that the remainder term
u
it
These
be decomposed
as
e
where
a.
represents the more or less time invariant individual
:1
effects,
(3.1)
it
b
t
represents the period specific and more or less
e
individual invariant effects, and
it
represents the remaining
effects which are assumed to vary over both i.ndividual units and time
periods.
The fundamental question which should be addressed prior to the
statistical analysis is whether or not to treat
parameters or as random variables.
a.
~
and
b
t
as
If we treat them as parameters, we
mayor may not wish to estimate their value::; explicitly; if we treat
them as random variables, we may, of course, only estimate certain
moments of thei.r distributions.
This question is not a trivial one,
for it makec; a surprising amount of difference in the estimation of the
other parameters depending upon which approach is adopted.
Searle
(1971a) presents an interesting discussion of the characterization of
fixed and random effects.
He suggests that, in endeavoring to decide
whether a set of effects is fixed or random, the context of the data,
the manner in which they were gathered and the environment from which
they came are t.he determining factors.
In considering these points,
the important question is that of inference.
If inferences are going
to be confined to the effects in the model, the effects are considered
43
fixed, when LnfereQces will be made abCl'lt a populatio.n of effects from
"1hi.c~\
those in the data an; considered to be a random sample, then the
effect[; are conside:rE:'d as random.
Regarding i.ndividual unit effects,
butb si.tuati.cns may occur in actual problems.
Instar:ces where
i.ndi\;idual units are a truly random sample from Some real populati.on
ace quite unusual, specially in economi.c st.udies.
However, their
E-,f feets may reasonably be consi.dered random if one i.8 not interested
in comparing the spec Hi.c units, but ra ther in extendi.ng the results
to a larger hypothetical population for which those units may be
thought of as a random sample.
The time points are never randomly
selected, for they are a group of consecutive years, say, over which
the data havE:' been gathE:re.d.
But, in general, their effect.s may be
considered as random, since one would not usually be interested in
Lnferences concerning spec ifi.c. years.
E.~rror
So, two spec ifications of the
structure are of interest:
(A)
a
(B)
a.
L
1.
,
b
1.:S
t
and
eit
CO:l:':; tan t
are random variables;
and
E
it
are random
variables.
Botb specifications are
coesider~d
in this thesis.
To cl-.aracterLz:e the models completely there is still the problem
of specifying the distributional properties of the random variables.
With respect to the unit eEfE;cts and time t:,ffects we adopt the usual
assumptions of the random error components model and the mixed I1lJdel,
respectively for Model (A) and Model (B).
effects
e
lt
However, for the residual
we make assumpti.on which takes i.nto account the
44
possi.bility of serial correlation among the residuals for the same
indi.vidual.
This assumptiun should be more appropriate in most
situations.
In fact, it has been recognized that successive
observations collected on an indiVidual for a period of time are
usually correlated.
Cochrane and Orcutt (1949) in a study of the
estimation of relationships containing autocorrelated errors, point
out that the significant fact in the analysis of functional relationships between time series Ls the autocorrelation of the error terms
and not the autocorrelati.on of the time series themselves.
They
argue that there is strong evidence that the error terms involved in
most formulations of economic relations are highly positively autocorrelated.
They support this view by investigating the sources of
the error terms, which they collect in three main headings:
(1)
errors in the mathematical form of the relationship,
(2)
errors in the measurement of the variables, and
(3)
omission of determining variables.
Stone (1954), following similar lines to argue that with economic
time series the assumption of serial dependence is quite likely to be
justified, says:
"The first of these sources of error is clearly
likely to lead to a systematic component in the
disturbance. The second source will do so in so
far as the errors of measurement represent
systematic (but not constant) errors rather than
random errors. There is reason to think that
systematic errors are important in economic measurements. Finally, the third source is also likely
to lead to a sys tema t i.c component in the dis turbance.
45
This is clear in the case of the (hlitted economic
variables, most of which will show some serial
correlation; indeed, in general a similar serial
correlation to that possessed by the variables included.
It may also happen with other omitted
variables such as rainfall which, even if not
serially correlated themselves, are likely to affect
economic variables through a serially correlated
variabel, such as the water-level of an area. The
anticipated result then follows, since the sum of a
number of serially correlat~d variables is itself
serially correlated," (Stone (1954), page 284)
Thus, a realistic specification for the model should allow for
the possibility of serial correlation of the disturbances for each
individual unit.
Different sorts of serial correlation structures
could be adopted.
It has generally been accepted that a first order
autoregressive error structure is a reasonable approximation for most
economic time series,
The assumption that
autoregressive process for each
i
e
it
is a first order
means that all the useful
(correlational) information about the residual term
entire realization previous to
e.
1., t-
1
e
it
e
it
in the
is contained in the residual
of the previous observation for the same individual unit.
The
appropriateness of such assumption will depend on the problem at hand.
Perhaps the most general useful assumption is that of an infinite
moving average.
This is the general expression for all weakly
stationary processes, including autoregressive processes (!:..a., Fuller
(1971), page 184).
For theoretical convenience we assume for the
residual term
the specification of a finite moving average
process.
As we will see later, this assumption is effectively
equivalent to using a "truncation" estimator of the spectral density
of a general weakly stationary process.
It is expected that this
46
assumption will be a reasonablE approximation for most practical
situations.
Suppose there is available a sample of observations at T time
A
points on each of
cross-sectional units o
It is assumed that the
th
observed value of the dependent variable at the
t,h e
'
l-~
th
time point on
.
'J
be expresse d as:
cross-sect 1.ona"
uUll"
can
"
Yi t "" ~~. tii
i
~--
= 1, H.,
A;
t
+
+ b t + e it
ai
= 1,
• GO, T , whe re
vector of explanatory variables for the
o
cross-sectional unit,
-S
X'
-it
th
=
t-
(X·
lt1
.
oooX·
l
tP
)
time point and
is a
.th
1:-
c
0
=
(3.2)
'
(6 '1· ·.13 p )'
is avec tor of parameters,
is a time invariant cross-sectional unit effect,
sectional unit invariant time effect, and
e
it
b
t
a.
is a cross-
is a residual effect
unaccounted for by the explanatory variables and the specific time
and cross-sectional unit effects
0
If we arrange the observations first by individual units, then
by time periods within i.ndividuals, we. may write the equations (3.2.)
in matrix notation as
:to "" X~" +
where
u
(3.3)
l
47
=
:L
(Yno oY 1T Y21
0
(a •• oa )'
A
l
i.s an
1
~:A.
,
0
0
.YAr ) , ,
X
b:::; (bio •• b )'
T
A x 1
,
=
(~llo
0
o~lT ~2l·
e = (ello .. e
0
12 21
1T
.~AT)'
,
=
a
••• eAT) , ,
vector with all elements equal to
1 , and g.;;)
eJ
denot.es Kronecker product.
With equations (3 >3) we make the following assumptions:
,
t~it
i.n
RP
is
P
lA,
T .~s
i=l,t=l
a sequence
0
f nonstoc.astlc
h·
k nown
RP
whose elements are uniformly bounded in
P x 1
vectors
The rank of
0
X
0
Assumption 2.
§..
is a
a
is a vector of uncorrelated random variables such that
=0
£;.;(a.)
~
P x 1
constant vector of unknown parameters.
(unknown) ,
and
02
cr
> 0 ,
a
i = 1, ••• , A •
Assumption 4.
b
is a vector of uncorre1ated random variables such that
(unknown) ,
and
= (e i l ·· .e lT )'
~i
0
0
+ Q'le t _ I +
U "" Q'aet
where
0
0
Q'O' Q'1 '
0
••
I;t
,
Q'M
b>
0 ,
t
= 1,
••• , T •
is a sample of a realization of a finite moving
average ti.me series of order
e
07
cr
M
«
+ Q'ME: t-M ,
0
T-l)
t
for each
:::;
1,
o
0
•
,
hence
i
T
(i = 1,
are unknown constants such that
0
Q'o
... , A)
:f 0
,
48
Q")
(E:j}j=-CXJ
i.s a whitt' ~loi3e process, i.~
.... ,
bequenc", of ttncorrt':lated random var i.ablEs with
() i.
(vnknuwn) ,
(J
1,
are mutually
... , A)
A
fb ",
[a'}"-l"
,
1. 1-
k ::; 1,
T
and
" tJt=l
uncorCf~lated9
The: random term3 hav'C:- norma] ::l1.strih.utiOllS:
and
and
t
> (\ .
e
Ole Sets of random variables
(i -
E(e: )::; 0
a
a.
?
i - l ' l j .. ' A;
e: t-k ""
N(O,a'-)
a
rv
J.
t
::; 1,
,
••• , T
••• , H •
tht;~
[f
Equatione (3 3) satisfy Assumptio!lS 1 - 6, then
0
and
(3.4)
where
I
is the identity matrix of size
A
and
matTix with all elements equal to
tll
ts~-
with
cov(e·t,e. ) ::;
1.
18
(
'.
in
cc,varia~'1(;e
U:H"~
fonn
"
[T
J
is an
A
is a
T x T
A x A
matrix
element
\ 'Yc I t-s \)
The
,
A
,
if
I t-s \
s:
M
('3.5)
o
H
It-sl
matrix 0.4), which we denote by
>
M
V , can be written
49
rxJJ T) +crb-'(JA,,. (}DI
)
T
'):\?
V,.. ",'-(1
a A -
where
::; IT
whose
off-diagonal elements are
M
+
2:
k;O
k -- r
1., "
••• ". 'M.
and, f o
1.
(k)
,),(3.6)
@r T
A
Y(k)(I
r'T(k)
is a band matrix
and all other elements are
o •
Su according to (3.6), the covariance matrix of the vector of
observations
has the form
:l
M+3
V(~) --=
where
f)
~
,..
... ,
G
(\,) 1.'
(J
o
\,)M+3
) 1
(3.7)
,
a
02
o
\,)2 ""
'\V k ,
02
f)
\,)1 ""
2::
k--=l
(J
(3.8)
b
k "" 3,
••• , M+3
(3.9)
Theorem
If
L.J.
e. "" (e.0l ••• eoT,)1
-I.
variance matri.x
l
'
1.
satisfies Assumption 5, then the co-
Var(~i):; f T
is positive definite.
,with
ts tr~
element defined in (3.5),
50
Proof:
According to Assumption 5,
is a linear transformation of
e.
'-l
random variables from a white noise process, defined as
E
where
is the
T x (T+M)
matrix
&M-1
(\
Q'M
E
:;
c
0
Q'1
Q'a
"
Q'2
Q']
(\
"
Q'o
)
l
"
Q'M
Q'M-l
v
0
(T+M) x 1
assumption that
Q'l
Q'M
where the vacant entries are zeros, and
0
:f
0
Q'o
0
Q'2
0
Q'1
Q'o
= (€ l_M···E: -1 €aE: 1·· ·€T) ,
The theorem follows immediately since the
vector.
Q'o
,§..
0
0
0
,.
is a
= E~-
e.
-~
and
~M
;'
a
implies that
E
is of full
0
rank.
Then, by result (j) of Theorem 12.6, it follows that
positive definite.
According to the same theorem,
that
J
I
A
®.]T
and
A
®IT
in the range of
~.
A
@r T
02
0
cr >
a
is
it is also true
are positive semidefinite matrices.
fact.s together with the assumptions that
that the covariance matrix
I
and
These
imply
V , defined in (3."7), is positive definite
51
3.3
Model B
Model B specifies for the regression equations (3.3) the same
Assumptions 1, 2, 4, and 5 of Model A.
The Assumpti.on 3, 6, and 7 of
Model A change to the following:
Assumption 3a.
A
o
a
is a vector of unknown constants such that
L:
i:::1
o
a. == 0 •
~
Assump!.i.on 6a.
The sets of random variables
i ::: 1,
and
••• ,
A , are unco rre la ted.
Assumption
7~.
The random terms have normal distributions:
02
€t-k'" N(O,O's)'
t::: 1, ••• , T;
If the matrix
X
k::: 0, ... , M •
contains a column of ones, say the one correso
13
ponding to the regression coe fficient
reparameterize the mode 1 by setting
it is convenient to
1
v
c
ai'\:
i
= S1
0
+ a1
i = l , o .• ,
A •
Then, the regression equations (3.3) can be written as
o
::L ::: X [,
where
+ UA ®.£) +
(3.10)
e
o
x
and
and
~(l)
as defined for equations (2.3).
It can be easily seen that the covariance matrix of the vector of
observations
y
under Model B has the form
52
.~
V(y)
(3.1.1)
o
o
V
k
as defined Ln (3.8) and (3.9).
under Assumption
5
V
k .. 2, ••• , M+.3 , are
and
Theorem 3.1 can be used to show that
is a positive defi.nite matrix in the range of
We not.e that the equations (3.10) resemble closely the equations
(3 • .3) of Model A; the difference is that
ai'
omi.ttt::d from the random term, the
X
set of regression coefficients
full rank"
§.,
111dtriX
= 1,
••• , A , is
is replaced by
is replaced by
.s
~.
If
X
X
and the
is of
the regression equations (3.10) under Model B satisfy all
the assumptions of Model A, except that
term.
i
a
is omitted from the random
53
EEl TIMATlON FOR MODEL A
4.
!i.!..:;!..;........~E_stimation
When The Covariance Matrix is Known
Under Assumptions 1 - 6, the BUJE of the vector of parameters
[
of the regression equations (3.3) is the GLS estimator
(4.1)
with cava rianee matrix
With the added Assumption?, the estimator (4.1) is also the ML
v
estima tor of
~.
The asymptotic properties of the GIS estimator (4.1) are
establ ished in the following theorem
(~ •.&.,
Theil (1971), page 398):
Theorem 4.1
Suppose the regression equations (3.3) satisfy Assumptions 1 - 6
and that
XlV-IX/AT
AT -. co •
Then, the GIS estimator
and
\fAT 7
(Ii
-~)
converges to a positive definite matrix
~.
o
AT -. co )
V •
V
is usually unknown.
the GLS estimator (4.1) cannot be used in practical situations.
V
4/ E
-_.&.,
£L,
p-l
Unfortunately, the covariance matrix
if
when
v
is a consistent estimator of
b.as a normal limiting distribution (when
with zero mean and covariance matrix
P
So,
Even
were known, there would be the bothersome problem of inverting
'rhe}",l (197])
...
,
page 2'38
" •
54
V
Ii.
for the computation of
However, this problem can be
circumvented if the cb.aracteristic roots and characteristic vectors
(or, equivalently, the spectral decomposition) of
trten, since
where
For
is positive definite, one can wirte it as
V
Q is the orthonormal matrix whose columns are the normalized
V and
characteristic vectors o:f
characteristi.c roots of
pi = 1\ -1/2.Q I
where
V is known.
AT
as its diagonal elements.
Let
Then, the disturbances of the transformed model
•
Z"k '"' piX
Var(lt'<) = I
V
A is the diagonal matrix with the
.
X~'<
'"'
piX
and
= plu ,
u''''
E (.!::!:.~'<) = 0
and
~ and its variance could be
Hence, the BLUE of
obtained by tbe OIS regress ion of
satisfy
on
y..."<
Altbough the covariance matrix
V
X'k
0
is usually unknown, this
procedure as well as the results to be established for known
will
V
be valuable for the method of estimation to be developed in Section
4.2.
The matrix
V, defined in (3.6), has a complicated structure,
because its characteristic vectors, as well as characteristic roots,
are tunc tions of the parame t.ers
H.
02
0"
a
02
O"b
and
o
y (h) ,
h
= 0,
1,
As a consequence, the asymptotic theory with the unknown co-
vari.ance matrix
V
is intractable.
What we shall do is to follow
Hannan (1963) and substitute an approximati.ng matrix
whose
o •• ,
55
characterLstic roots depend on the unknown parameters and whose
characteristi.c vectors do not.
This matrix is defined by
(4.2)
'j\:
rT
wh8re
fr
;;
is the circClIar symmetric matrix of dimension
QT1\,rQ~
,where
1\'1'
is the diagonal matrix with the
T,
i ..!:..,
th
t-
diagonal element
M
yCh) cos (rr tn/T)
2. I:
h=l
= 2,
t
4,
T ('I even)
••• , T-l (T odd) or
v
d
t
::;
(4.3)
M
yeO) + 2 ~
y(h)cos[rr(t-l)h/'I]
h=l
t ::; 1, 3,
and
Q'
T
is the
'I x 'I
th
ts-
orthonormal matrix with
t
VZTf
••• , T (T odd) or 'I-I ('I even)
element
'" 1
cos[nt(s-l)/'1'] ,
t
'" 2, 4,
or
•••• '1'-2 ('I even)
'I-I ('1' odd)
(4.4)
\jZ7Y
si.n[n(t-I) (s-1) /T] ,
t
::; 3,
5, ••• , 'I-I
(T even) or 'I (Todd)
t
=T
if 'I is even •
56
S
:=
~
1, •••
T •
The fundamental result obtained by Han.nan (1963) is consequence
of particularly relevant properties of the covariance structure of
stationary time series.
following theorem.
The properties manifest themselves in the
Proofs of the theorem are given by Grenander and
Szego (1958), Amemiya and Fuller (196'7) and Fuller (1971).
~rem
4.2
La t
f ,
I
bl2 the covariance matrix of a realization of
'I
observations from a stationary time series with absolutely summab1e
covariance func tion.
Then, for every
e: > 0
there exists a
'I
e:
such that forT> T
E:
where every element of the matrix
E:
;
the matrices
QT
and
1\1'
E
T
has absolute va lue less than
are as defined above.
This theorem establishes that, asymptotically, the matrix
Q
T
defined in (4.4) will diagonalize all covariance matrices associated
with stationary time series with absolutely summable covariance
function.
This tact was the motivation for the result established by
Hannan (1963) which we now state in the following theorem:
Theorem 4.3
Consider the regression equations
vector of
T
components,
vector of parameters,
and
<Ji
is a
r:=
'I x p
v:= (VI". v )
T
PI2.. +
;L ,where
mqtrix,
i
l2.
Y
is a
is :1.ndependent of
is a
p x 1
<Ji
•
57
Suppose,
V
t
,
t
= 1,
... , T , is gener::ited by a stationary process
if
with absolutely summable covariance function and that
susceptible to a "generalized harmonic analysis".
diag(g,'<j)
Then, as
•
T -+00,
6 (£
T
-.l2.)
is
6
Let
T
=
has the same limiting co-
,-v
6 ([ - Q.)
T
(~'r-l~)-lif'r-ly
and b
,~
T ~
T - ,
~
variance matri.x as
where
~
~
is the estimator with the same
=
expression as
.....
~
except that
iii
The assumptions on
(i)
li.m
T~oo
( H)
lim
T~oo
')
ii,
is replaced by
rT
defined above.
amount to t[te followtng, where
T+h ?
2:: ~'~ . .
t=ll.
---- = 1
T
rT
is tbe GLS estimator,
for all
i
and fi.nite
<j)
=
(~l •••.~p) ,
h
(r.
2.:
t=l
t1.
T-h
2::
t=l
exists and is finite for
T
2::
t=1
all i, j, h •
The implicati.on of this result is that in regression problems
with stationary errors the transformation l)f tbe observations by the
matrix
Q
T
yields an approximately tincorrelated set of data.
In this
way the problem is reduced to one in ordinary weighted regression in
which the weights can be estimated from the data.
The weight.s to be
estimated are, in fact, proportiunal to the spectral density of the
process at frequencies
tTt/T
corresponding to the columns of
QT.
58
This fact suggests the terminology "conversion to the frequency
domain" used to describe the procedure by several authors.
The spectral density of a stationary process
rtV
"I
1
CD
t' t=- CD
is the
Fourier series
1
few)
= Zn
CD
~
y(h)cos(w h )
h=- CD
where
y(h)
= Cov(vt,V t +h )
In our case,
y(h)
=0
for
so, the spectral density becomes
f (w)
1
= 2Ti
M
~
y(h)cos(w h )
h=-M
and can be estimated by
1
= 2Ti
M
/.
~
Y (h) cos (w h )
h=-M
where
y(h) ,
h = -M,
... , -1,0,
estimators of the autocovariances
1, ... , M,
are appropriate
(see Anderson (1971), Fuller (1971),
or any of the standard references in time series analysis for details).
Hannan's method has been applied to a variety of regression
problems by Hamon and Hannan (1963).
Duncan and Jones (1966) present
a practical computing technique for the application of the method.
Engle (1974) presents an extended version of Hannan's method of
regression analysis in the frequency domain.
Engle argues that
frequency dl)main regressi.on analysEos have the standard small sample
properties, that they do not require smouthing of the periodogram for
the estimation of the spectrum and are computationally easy to use,
The application of the technique has been extended to distributed
59
lag analysi.s by Hannan (1965, 1967), Amemiya and Fuller (1967),
Fishman (1969), Dhrymes (1971) and Sims (1971).
"k
We next establish the spectral decomposition of
V.
For that
we will use Theorem 12.8 of the Appendix.
Theorem 4.4
j'C
The matrix
defined in (4.2), has spectral decomposition
V
T
~2
(Ae' b + dt) (£1
+ L:
t=2
A
02
L: (TO' a +
i=2
+
T
A
L:
+ L:
t=2 i=2
where
.9.. t
,
= 1,
t
G
dl )(2t
® qt) (.0'1 ®.9..t ) ,
®.9..1) (21. @.9..1)'
dt (2. i ~ .9..t ) (9. i @9..t )
g
•
,
T
,
th
is the t--
defined in (4.4);
A-I
,
(4.5)
column of the matrix
= 2,
i
orthonormal contrasts of dimension
I
A
is any set of
"', A
~
and
t
'
t
= 1,
""
is de fined i.n (4" 3) •
Proof:
---,
ok
The matrix
Nuw,
has characteristic roots
multiplicity
is
V, defined in (4.2), can be wri.tten as
1 "'
~A'='A
•
A-l.
Then,
AlJ 2b
and
(j
,
the later of
The characteristic vector corresponding to
has spectral decomposition
Acr~
T ,
60
02
S im.U ar ly ,
cr J ,
a 1.
=
has spect.ral decomposition
o2( 1 1 )( 1
Tcr a \[T ---T"'\[T
IT
.)'
.
ok
f
Moreover, by definition,
"k
f
where
.9..t ,
hence
51 1
has spectral decomposition
=
... ,
t = 1,
1
\[T
1T
.
T
, is
the
th.
t-
02
cr aJT
Therefore,
column of the matrix
QT
,
":k
+ f T has spectral decomposition
T
+
l:
t=2
dt51t.s.~ •
The theorem follows immediately from these results and Theorem
0
12.8.
Cow l.lary 4.).
In Theorem 4.4, £i '
column of the
A.
A x A
i
matrix
= 1,
Q
A
••• , A , can be chosen to be the
given by (4.4) with
T
.th
1:-
replaced by
With such a choice,
(4.6)
where
AAT
=
diag("l1" '''IT ~'21" <"AT)
is the
AT x AT
diagonal
matrix whose diagonal elements are the charactertstic roots of
namely
'k
V
61
("
02
Acr b +
02
Ao'b
A
it
02
TO'
a
+
.,
d
1
+ dt
i
= l',
t
=
1
i
- 1 ,.
t
=
2,
i
=
.... ,
A',
t
... , A;
t
••
0
T
,
=
02
TO'
a
+ dl
a
2,
i -,2,
t
=
1
V provided by the matrix
The appruximation of. the matrix
evidenced in the following immediate consequence of Theorem
~'(
V is
4.2.
Corollary 4 . )
~
Let
QA
and
V
Q
T
be the matrix defined in (3.4) ,
are as defined above,
of Corollary 4.1.
integer
T
€
and
Then, for every
such that for
T
>
€
A.AT
> 0
be the diagonal matrix
there exists a positive
T
€
where every element of the matrix
E
T
has absolute value less than
Proof;
From the expressions
By premultiplying by
Ql
.'.
QlT(V - V)QAT
T
(3.4) of
V
and
(4.2) of
and postmultiplying by
":k
V
we obtain
QAT' we get
62
The result of the theorem follows immediately [com Theorem 4.2.
Q
Estimation When The Covariance t<latrix
4.2
is Unknown
In this section we develop an estimator of the set of regression
coefficients
known,
[
for the usual case when the covari.ance matrix is un-
The estimator is a two step GLS type estimator,
.i.~.,
it is
provided by the GLS technique with the unknown covariance matrix
replaced by a suitable estimator of
The covariance matrix
V
V.
has the form (3,7),
i .. ~.,
M+3
V =
and
(3.9).,
(4.7)
L
k=l
k
= 1,
So, an estimator of
... , M+3 , are defined in (3.8) and
V
is provided by substituting in (4.7)
estimators for the corresponding variance and Lovariance components
o
k
=
1,
••• , M+3 •
The classical methods for estimating variance components are
summacized by Henderson (1953) who popularized three methods, known as
Henderson's methods I, II and IlL
Searle (1971b) reviews the methods
in use to that date and presents them in matr'ix lOrillulation.
Anderson
(1969, 1970, 1973) derives a general formulation of HI, estimators of
covariance matrices which are linear combi.natL-mE>
assumi.ng normality.
However,
,J[
given matrices,
the method involves the solution of a
63
::iCC 1'['
n,)ldtncar eq,laLi,}n") and
(19/0)
ptC'c'ot"
a
'Jlllii(.:d
1.,,-
tfll,'(1ry
cc'mputationally burdensome.
t.~lillli!tj,'n
[q';Jdcll'atic
()f
Rao
variance
intt'uduclng th,' ]\UNQlJE (mLnlmum norill quadratic unbiased
C"l1lp()neIlL".
c,-,timatLon) principle.
This method 1,a:, been further develope.d in a
:,(",rle;:, u[ papers by Rao (1971a,
19?1b,
1972).
Abuut the same time and
Lndepend(;'lt.Ly, Seely (1969) pre:3ent3 a gUleral
theory of unbiased
e"timatioll\vhcn the choice ,)f ('stimatut'.:; 1.5 restri.cted to finite
dLrnensL,'nal ,'cctcJr spaces, wLth specLal emphasis to quadratic
8sti.mati011 of functiuns
Ute fdrm
IJf
III
,\I.
:>: 6 1,1.
i=1
(i
i-il'lcre
V
illc)d" j
E(.;:) '"
1
=
,
1, ... , m) are parameter", associated with a linear
X~
wi.th cuvar Lance mat ox
III
\)v
)
i=1
... , Ill)
(i = 1,
V,
and
1,
.L
are real '3ynllllt'tric matrices.
aisc di.2ctl°sed in Seely (1970a,
Seely and
~o8ng
The method i.s
1970b) and Seely and Zysklnd (1971).
(197]) consider [he MINQUE principle using an approach
aV'ng the Ll.tws of Seely (J969),
[i,t,
The
m'2thcd ad,)pted in this thesis i.s the one developed by Seely.
c,mcept;~
] ,.,t
y,
and results necessary tnt' nur purposes are presented next.
be an
n x 1
expect.ati~Yn
randonlvcctor wi th
X~.
and
covariance matrLx
m
V
i.=l
where
X
IJ
i. i
18 a known
n x p
matrix uf rank
p
eac h
V
0
:1.
i.s a known
64
n x n
~ =
real synnnetric matrix and
(Sl ••• Sp)'
and
Yo
= (v 1 ••• v m)'
are vectors of unknown parameters.
G.
Let
let
G.
G.,
l:..~.,
denote the set of the
=0
a,
h
real synnnetric matrices and
denote the space of quadratic estimators with matrices from
a = Lz' Ay:
quadratic estimators
AX
n x n
A e G.1 •
y'AZ,
We shall restrict considerations to
A e G. , that satisfy the condition
So, the space of estimators we consider is the subspace of
= (Y'AZ:
A e G.,
AX
= 01 .
Definition 4.1
A parametric function
m
6 .V
L:
•
~
i=l
~
A e G.,
is said to be estimable if there is an
x'Ay
AX
=0
, such that
is an unbiased estimator of the parametric function.
One reason for limiting considerations to the subspace
h
of
quadratic estimators is the desirable property that a quadratic
estimator
the form
y'Ay
y +
be invariant under the class of transformations of
~
for arbitrary
~.
This seems to be a reasonable
requirement of estimators for linear functions of the parameters in
the covariance matrix.
Also, the class of multivariate normal
distributions
L:
m
(Nn(X§..,
v.v.):
ae
i=l ~ ~
under this class of transformations.
considering only che subspace
quadratic form
y'AZ
h
P
R ,
~
m
R }
is to require that the variance of a
be independent of the parameter
implied that
remains invariant
Another criterion which leads to
used this criterion and showed that requiring
independent of
Y. e
AX
=0
•
§..
Var(Y..,'Ay)
Hsu (1938)
to be
65
Theorem 4.5
VI' ••• , v m are all estimable if and only if
'th ~J' th element tr (NV , NV . ) ,
w~·~....
The parameters
~
th. e rna t rlX
B
~
J
is nonsingular.
In checking for the estimability of the variance and covariance
components, the following resul t may be of interest.
Corollary 4.3
The parameters
matrices
Theorem
NV1N,
VI' ••• , Vm are all estimable if and only if the
.oo, NVmN
are linearly independent.
4.6
If the parameters
VI'
unique unbiased estimator
••• ,
V
m
of
are all estimable, then the
is the solution to the set of
equations
B.Q.
where
B
t
vecor
w~
=c
(4.8)
is the matrix defined in Theorem 4.5 and
'th
-
....;th
eleme·nt
c
is the
m x 1
1'NV
. ,.1 NY .
It can be seen that Seely's estimator, provided by the solution
of (4.8) is, in fact, an unweighted MINQUE estimator,
l.~.,
a MINQUE
estimator with the prior estimate of the covariance matrix replaced
by the identity matrix
I , as representing complete ignorance of the
66
true variance and covariance components.
(~.&.,
compare equation
(4.8) above wi.th equation (4.4) in Rao (1972).)
The reason for choosing Seely's estimator is its computational
advantage, si.nce the deri.vation of the MINQUE estimator would require
AT x AT
the inversion of the
about the covariance matrix.
matrix representing prior information
Besides, as usually no reasonable guess
on variance and covariance components i.s available, one may do better
by not ilsing any pri.or infurmati.on in the estimation procedure.
Let
Y(h)
and
h
,
= 0,
••• , M , denote Seely's
estimators of the corresponding unknown parameter values.
Let
dt
be
defined by expression (4.3) with Seely's estimators substituted for
the correspondi.ng parameters,
i:.~.,
M
yeO)
+
2 1: y(h)COS(TTth/T)
h==l
t == 2, 4,
••• , 1'-1 (T odd) or T (1' even)
(4.9)
M
+ 2 E ~(h)cos[TT(t-1)h/TJ
h=1
t
== 1, 3, ••• , l' (1' odd) or 1'-1 (1' even)
These estimators are not guaranteed to be nonnegative definite.
In order to get a positive definite estimator for the covariance
matriX, we define
67
/
I
",,2
(]
a
",2
(]b
\ cr~
=
:=
,
(0
(
(
,,2
(]b
if
,,2
a
(]
> 0
otherwise
,
if
,,2
(]b > 0
~\
(4.10)
otherwise
0
......
otherwise
where
c
is an arbitrary positive real number.
at least one
c
In case
"d
t
> 0
for
t , we may set
= min 1.rOd t· d t
0
>
O} .
teT
We use as estimator of the covariance matrix
V in the GLS phase
of the estimation procedure the matrix
(4.11)
where
AT
= diag(dl ...dT )
and the other matrices on the right-hand
side are as defined above.
The proposed estimator of the vector of regression parameters
o
§.
is
(4.12)
68
";~
The matrix
V
i.s chosen over the matrix
V
with the parameter
.....
esti.mators replacing unknown parameters,
V, because:
,,,,"ow'
(1.)
··k
V
has a known spectral decompositi.on, maki.ng its
',lse cllmputa t:i.onally feasible for large
......
( ii)
the substitution of
kn.own results
C,)
"'k
V
for
V
T
,
allows the use of
prlJve the unbiasedness and
asymptotic normality of the regressi.on coefficient
esti.mators.
,.,
~
The use of
*V
instead of
*V, i.!:.., V*
with Seely's estimators in
place of the unknown parameters, guarantees a positive definite
estimator for the covariance matrix.
69
f ,)
:'1
l-
j
rlt ,"
.1
n I-J :;-1 ~; t;
of the two step
tab:
c:
, !
"..:.+ '" i
L
,:1
and
djJt::
a
c:,~
E'
,-ami.' d5vmnt'.,CLc multivari.ate
1rlla t .~ r
>
('
X ~! It> ii co d by
pi
t...-'-"
,3 L. )
\, 4 c 12) wit h
ng the corre spond iog
~ ..
t
L'
In
~
u
at
!-
fy the i\s,oumptiDuS 1
~
7,
f
a)\
1
If
crt,
a linear function of
rm
';
\'J
l~
~,
)
~H-
k
",'
3
a,l
d
lJ.'"d
in (3.'::1),
,lmbE
\ >
'''"~I
I,~'/v
"
:
;:.
N
r, .
"\'A +
I,' ~,
fj
'of
V
t 1, <i j'
~,
!
v
1.6
invariant
;) ince, by
70
Abburnptiun I,
by
8
(t'2111t
1I
is 8yrmn,o.tcLcally distributed about zero, it follows,
uf Kak\,vanl. (1967),
that
~
~
*-1~
~-1 -1
(XiV X) XlV
;
is an
o
m'tb.1.ilSCtl cc;timator of
fl-
if its expectat.ion exists.
To demonstrate that the expectation of
~
exists we follow
along the lines of TI!eorem 2 in Fuller and Baltese (1974).
.'V
"k
an arbitraLy linear cc'mbinat1.cn of
when':
n
"'"
v
Q. -
l?. ,
Consider
nam,,:!y
]'X(~
-
is an arbir.rary vector from AT-dimensional space.
B) ,
Using
i.nequaliti,e:3 (lZ.L) and (l2.3),we obtain
!n'Xc[- §,) I =
In' (X~
- X@) I
where the last inequality follows from the fact that
*-1/2
~-l -1 *-1/2
V
X(X'V X) XlV
is an i.dempotent matrix.
Nm" , the maximum and minimum charac terLs tic roots of
r,~spec-tively,
T~2 +
a
A(j2
b
+ max '([
teT
t
and
mto.
LeT
dL
V
are,
71
Then, according to inequality (12.2),
IV
I'n' X ([ - §.) \ ~
min d
t
teT
!
i
l___
_2
But
TO' a
.....,2
+ AO'b + d t
(5. 1)
has the form
M+3
M+3
eky'NVkNy + eO - E e u'NV Nu + eel'
k=1 kk k=1
.
L
1S
= 0,
k
where
1, .", M+3 , are real numbers,
bounded by a multiple of
'"
~k
0
\ll'X(§. - §.)
ulu
So,
Then, we get from (5.1)
u'u
I :s
K ----...,.-,,...
. "'d )1/2
( m~n
t
teT
for Some
K
> 0
Now,
,
1
=
min d
t
te T
By definition,
'"d
t
T
1
max
d
te T '"
"H,
= dt
T, where
d
t=l '"
t
arbitrarily chosen, and
t .. 1,
1
:s 2:
B
t
(5,3)
t
=c
if
dt
d
has the form
t
> c,
otherwise, for
dt
is a symmetric matrix,
0< au'V
~t
where
-1
d
t
1
-
t-
Then, since
a
and
.
u < u'B u < bu'V-u
t
°
= vlNA Nv = ulB u
.~
t ~
t-'
positive definite, there are positive real numbers
that, for set set
c >
=
1, "., T,
b
V,
such
is
72
1
-1-1
b (!~I V ~)
Now,
< (uiB .u)
-
-1
1
< -
a
t"-
impl ies that
u'v -1 u -
treedom.
Then, the expected value of
AT > 2.
This implies, by (5.4), that
set
d
J.AI1J.en;
t
=
dt
(~'V
-1-1
_U)
2
X
.
with
(£'V-l£)-l
l/~
(t:= 1, .0', T) •
t
Since
t
=
1,
••• ,
T •
AT. degrees of
exists provided
is integrable over the
J/ci L
(t:= I, ... , T)
is constant avec the complement of this set, it follows that its
expectation exists.
Then, by (5.3),
Eel/min ~t)
te T
exists provided
AT > 2 •
·
b y Assumptlon
.
7,
S ].nce,
L
E(·~,~)2.
.
eXlsts, we conc 111 d e, uSlng
Cauchy's inequality (12.1) on the right-hand side of (5.2), that the
expectation of
'*
]'x(~ -~)
5.2
0
exists.
0
Asymptotic Distribution
To establish the main result of this section we need some
pre liminary resul ts whose proofs are very tedious.
Lemmas 5.1 and
5.2. summarize these results; their proofs are outlined in the
Appendix
0
Lemma 5.1
Let
Suppose
Then,
V ,
k
X
k = 1, ••• , M+3, be the matrices defined in (3.9).
satisfies Assumption 1 and define
N
=
I - X(X'X)-lX I
•
73'
O(AT)
,
tr(NV .NV.) = \
~
J
i
=j =
i
=
i
= j = 3,
.:".;:
i
= l',
j
=
2,
i
= 2',
j
=
3
i
= 2',
j
= 4,
1
=2
j
,
M+3
e ..... ,
•
If
•
,
(5.5)
M+-3
M+3
otherwise
Lemma 5.2
Let
Vk '
k
= 1,
satisfy Assumption 1 and define
°
of tr(NV.NV.NV.NV1,)
~
J
~
~
••• , M+3, be as defined
N
=I
(3.9), let
- X(X'X)-l X '
j
i
1,
1
1
1
1
1
1
1
2
1
1
1
3
1
1
1
k
1
2
1
2
1
2
1
3
1
2·
1
k
1
3
1.
3
1
-)
"
1
I.
1
k
1
k
1
k
1
k'
2
1
2
2
'tr(NV . NV .NV .NV1,)
~
J -~
4
AT
3
AT
3
AT
3
AT
2 2
A T
Ai
2
AT
?
AT2
AT
2
AT
2
AT
3
A T
Then, the order
is as follows:
Order of
i
.
X
Order of
i
j
i
1,
3
1
3
2
AT
3
1
3
3
AT
3
1
3
k
AT
3
2
3
3
AT
3
2
3
k
ATO
3
3
3
3
AT
3
3
3
k
AOT O
3
k
3
k
AT
3
k
3
k
AOT O
k
1
k
2
AT
k
1
k
3
AT
k
1
k
k
AT
tr(NV.NV.NV.NV
.~
)
~
t
)
14
,
'2
'1
A'T
3
-
k
1
k
k~
AT
k
2.
k
3
k
2
k
k
AT
ATO
')
2
.2
-,
L,
1
2.
2
..,
A~r
k
4
A T
?
"3
"-
2
2
3
A r
k
.2
k
kJ
ATO
.2
2.
2
k
3 oJ
A T
k
3
k
k
2
3
2.
3
k
3
k
k"
AOT O
AOT O
2
"3
2
k
k
k
k
k
AT
k
k
k
kl
AOT O
k
k' k
kl
AT
k
k ll k
ke
AOTO
2.
k
2.
k
')
k
2
ka
t.
2
A T
A2. TO
2
A T
) rJ
A"'I
kl! < k l < k
where
k, k l
and
, k"
= 4,
~
c
~
,
M+3
V2mrna 5.3
,----Suppose the regression equations (3.3) 2atisfy Assumptions 1 - 7
and let
w
k
l
=
"k'
=
k
1,
00.,
M+3 , be as defi.ned in (3.9).
2 -1
(AT) l'NV1N,r:)
= 3, 0.',
M+3 •
w
and
ASbume
A"" Ct'T
for Some
ex>
k
Define
::;
o.
Then,
::; 0 ((AT)-1/4)
P' ,
.
Proof:
o
By assumption,
o
V
\)M+3 M+3 •
y ... NAT(Xfi, V) , where
V::; \) 1\)1
+ ... +
5/
Then-
.? IIf 1" '" N(~,
L:) , then Var(y'Ay) '" 2. tr(AL:)2
Searle (19 71a), page 57.)
+ ~!AL:1\k..
(~.
,8..,
75
Var(y: NV i N.l)
2 tr(NV. NV)
2
1
2 tr(NV.NV)
+
0
0
4~ u X' NV. NVNV. NX!3
1
1.-
2
1
M+3 M+3
= 2 tr
I:
I: ~.V.NV.NV.NV,NV
1J
L J 1. £,
i=l j=l
(5.6)
Then, according to (5.6) and using the results of Lemma 5.2, we
obtain
Since
A = aT
for Some
a > 0 , we get
Var(w ) = O«AT)
1
Similarly, we obtain
-1/2
).
76
(AT)
-2
Var(z'NVkNy)
= 3,
k
••• , M+3 •
The result of the lemma follows immediately by Corollary 12.1.
0
Theorem 5.2
Suppose the regression equations (3.3) satisfy Assumptions 1 - 7.
be as defined in (3.8).
then the estimator of
o
~
A
= aT
o
is estimable,
~
provided by the solution of (4.8) satisfies
k = 1,
(We assume that
If
for Some
••• , M+3 •
a > 0 .)
Proof:
According to Theorem 4.6, if
o
~
is estimable, its estimator is
the (unique) solution of the system of equations
Lemma 5.1 these equations have the form
~
=c
Now, by
77
\
2 AT
AT c
ll
ATC
AT
Z1
AT cAT
12
Z
A TC
ATc
AT
AT
AT
ATc
l3
14
o AT
AT C
AT
ATc
n
AT
ATc l , M+3
Z3
AT
Z4
o AT
C
z, M+3
A
V
ATc
ATC
AT
3l
AT
41
ATC
AT
ATc
32
AT
43
ATc
o
AT
o
o AT
AT c M+3 , Z
0 AT
A l' c 3 ,M+3
34
A01'0 cAT
ATO cAT
42
o
AO l' 0 cAT
33
0 AT
A I' c 4 , M+3
44
0 AT
A l' c M+3 , 4
/
y'NV1Ny
!
y'NVZNy
I
=
!
\
Xl NV 3Ny
I
y'NV4 Ny
\
J
!
\\ y'NV
,
where
k, j
r AT cx), CX)
t c kj 1A =1, 1'=1
= 1,
••• , M+3 •
second equation by
get
(5.7)
Ny
M+3 -
are bounded sequences of real numbers,
If we divide the first equation by
AZT
AT
and each of the other equations by
Z
, the
AT , we
78
1 AT
-c
T
1 AT
-c
T 14
13
1 AT
-c
A
\J
AT
3,M+-3
1 AT
--c
AT
4,M+3
\
\
AT
cM+-3,1
1 AT
-c
T M+-3,2
1 AT
-c
AT M+-3,3
AT
I
AT
1 AT
-c
M+3,4
C
M+ 3 ,M+3/
"\
/
!1 1
AT2
y'NV Ny
\
1
=
\
\
\
When
A .... co and
~T
T .... CD suchtha t
i
z'NVM+3 NZ / /
A
= aT
for some
(5.8)
a > 0 , the
matrix on the left-hand side of (5.8) converges to the nonsingular
matrix
79
o
o
o
o
o
o
o
o
o
o
C
o
o
o
o
o
,
I
c M+3 ,1
o
cM+3,M+3
t
where
c
,
AT
' = 1 ~m c .
kJ
A-+ CO k J
T-+co
Then, it follows from (5.8), according to Corollary 12.2 in the
Appendix, that
~ - E(~)
o
The following result is immediate:
Cora llary 5.1
Suppose the regression equations (3.3) satisfy Assumptions 1 - 7.
If
~_'_
~
=
(,0,v 1 '
••• ,
0) ,
\)M+3
as defined in (3.8) is estimable, then the
estimator provided by the solution of (4.8) is a consistent estimator
of
that
~
(in the sense that
A
= aT
for some
~ - ~, ~ 0 as A -+ co and T -+ co such
a > 0) •
80
Lemma 5.4
----ok
Suppose
X
is a matrix satisfying Assumption 1 and let
the matrix defined in (4.2).
Then,
V
be
XIO-1X = O(AT) •
Proof:
-;'~-l
Using the spectral decomposition of
X 'V~'(-lX
Let
= (A~b2
nv
TO'o2
a
+
, we have
U
d )-1 I (
IvI
X ~l )( £1
1
X £1 \-"'J
i·x-J ~l ) IX.
+
T
(A!J2
I:
b
t=2
+
o -1
-~ ".
dt) X(£l Xi~t)(£l X' ~t) 'X
+
A
I:
i=2
+
o -1
d 1 ) X' ('£i
+
T
A
I:
I:
t=2 i=2
(TC a2
M = max(A, T}
= (A!JZ
b
=
+
V
(A 02
O'b
M
oZ
TO'
a
x
.9.. ) IX
• 1.
d~lXI(o.
iX)~t)(o.
"Xi.9.. t ) 'X
-1. --:L'.
and
+
\xj .9.. 1 ) (£i
m = mintA, T}
.
,
Then,
+ d1)-l(AT)-lX'JATX
T 02
+
+-0'
M a
0
d
Ml)-lM-lAT[(AT)-ZXIJATXl
= M-lAT 0(1) = Oem) ,
since, according to Assumption l,
finite matrix as
AT
~
co.
(AT) -ZX I J A-I
Similarly, we obtain
converges to a
(5.9)
81
T
=
02
L: (Acr b +
t=2
= 0 (1.)
A
at) -1,X (~~
1
'~'. i~l.tg.~) X
T
ix' L:
-1-1 d t=2
X' (0 0'
(] q U ;. X
,~t<·,·t.
1) X,[l
f,'X' (
= 0(A ' A J AI IT -
T"1.,_.
.]T) J<
= 0(1)
O(AT) = OCT)
A
A
02
= E(T
o +
i=2
= 0(1)
T
a
O(AT)
do 1. )
-
1
X' (
,,"X':\
' • 'X
..r.:. i·'~' 1. ';.9 1~(l )
= O(A)
T
, (1)
~XJ
Sot .<1't )X
--
l\
I: X"(o.o',
·~··1.··-'1
i=2
82
= 0(1)
= O(AT)
O(AT)
•
The lemma follows by replacing these results in (5.9).
0
Theorem 5.3
Suppose Assumptions 1 - 7 hold for the regression equations (3.3).
.
Th
. en, te
h
est~mator
where
is the
D
elements of
A
= Q'T
.t:..
.t...
p x p
,';'\-1
X V
for some
t = (X '''''V~'~-lX) -lX.'''''V~'~-lv
sa tis f ies
diagonal matrix composed of the diagonal
(It is assumed that
X
Q' > 0 .)
Proof:
If we premultiply both sides of equations (3.3) by
(X't-1X)-lx,t- l , we get
which gives
Then, we can write
D
"""
1/2"~
0
(13 - ft)
=
83
*-1
V
= "'a( *-1
V
If we let
T
'
we obtain
(5.10)
Now, using the expressions of the spectral decompositions of
~'(-l
V
and
*-1
V
,we have
f
A
+
f.1.. M.1..
2:
i=2
+
T-l
A
2:
2:
M
·t ·t
ftM i t ,
t=2 i=2
(s.H)
where
,...
d
f
ll
=
T
,...
2
AC b + 'lS'2
0' a + d 1
,...
d
f
·t
T
= ,...2
,...
Ac'b + d t
,...
d'
f.1.• =
T
Tc;'2
,...
0' a + d 1
,...
0
d
d
T
T
f = -t
dt
t
a
aT
AlJ2 + TO'02 + d
b
1
a
0
0
d
T
0
02
Aa b + d t
t = 2,
.. -. , T
0
d
T
02
TO' a + d 1
i ... 2,
0
... ,
t = 2,
M = ('£1
ll
®.9..1) ('£1 ®.9..1)
M. t = ('£1
®.9..t ) ('£1 ~ .9..t )
... ,
A
... ,
T
T-1
I
I
,
t
...
2,
84
i
= 2,
i = 2,
••• , A
••• , A;
t
= 2,
••• , T-1 •
From (5.11) we obtain
D
-1/2 ,*-1
X V
XD
-1/2
1 2
0 -1/2 *-1 -1/2
= d.f
X' V XD
+ f 11D- / X'M 11XD-1/2
T
+
L:
t=2
A
+ L:
i=2
T-1
A
1 2
1 2
L: f t n- / x'M itXD- /
•
t=2 i=2
+ L:
(5.12)
We next determine the probability order of the scalar
coefficients on the right-hand side of (5.12).
Expanding by Taylor's
series about the true parameter point up to the first partial
derivative terms, we get
d
o
T
~2
d
T
-.,......-----:~--=-:::-------:~--
"",2
"'"
02
02
0
hJ +lCJ + d
At:! + TO' + d
b
alb
a
1
85
for some point
This gives
and
r::=2
02)
\cr a - cr a
dT
---~2=----~2:------ (d 1
2
Ac b
M (M
To'
d
M
M
1
+ ...-1!:. + -)
+. ----:-/=-----2---Ac
To'
d
M(...-.E. + ...-1!:. + -l)
M
M
M
-
d1)
2
(crT -
dT )
86
+
1
1
(AT) 1/4M ~b
,,=2
_
+
M
-2
TO'
[( AT)
-
d
a
1/4'"
(d
0
- d )l
T
T·
1
M+M""
Similarly, we have
1
= _ --=--1/-4-
(AT)
+
~.
=
2
(a b
a
\O'b
+ -1:.)2
O'b)]
A
1 1/
1
1/
A(AT) 4 -2
at [(AT) 4(d T :.. ~ T··)l
O'b + A
'"
d
f.
A
dT
--....::.,...[(AT) 1/4 r-: 2 _ 02
T
",2
To'a
+ d1
~
- T02
cr a
T
+
;} 1
=
-Td
(Tcr
2
a
T
+ d 1 )2
(cr2
0'
a
02
~ 0' )
a
87
... _
1
~2
[(AT) 1/4 \,0'
+
1
U
--.J
f
t
...
1
T(AT)1/4 -2
0'
d
-!
d
t
d
- -
~
T
a
+..1.
T
a
-
dT--.J
... - -
ci2
t
02
) J
a - 0' a·
(AT)1/4 T
(d
t
[(AT)1/4(dT -
1
Q
t
--.J
- d ) +t
d
t
...
-
+
1
_1 [(AT)1/4--.J
0
(AT)1/4 d
(dT - dT)]
t ....
... 0 P (1/ (AT) 1/4) •
(d
~T)J
0
- d )
T
T
88
From the spectral decomposition of
*-1
V
we obtain
T
+
L:
t=2
A
+
L:
i ... 2
(5.13)
Then, since
we have
+ a )-In-1/2x'M xn- 1/ 2 = 0(1)
(~2b + ~2a
lII
hence
n- 1 / 2x'M xn- l / 2 • O(M) •
11
Therefore, according to previous results,
op (1/(AT)1/4)
Also, according to (5.13),
hence
'
89
T
O(l/A)
~ n- 1/ 2x'M xn- 1/ 2
·t
t= 2
=
0(1) ,
i·~· ,
T
~ n- 1/ 2x'M xn- 1/ 2 = O(A) •
t=2
•t
Therefore, by previous result,
Similarly, from (5.13) we get
which gives
A
~ n- 1/ 2 X'M. XD- 1/ 2
i=2
+"
= O(T)
•
Hence,
A
~
i=2
By
(5 •. 13),
which gives
T-1
~
A
~ n- 1/ 2x'M. xn- 1/ 2
t=2 i=2
~t
= 0(1)
,
90
hence,
T-1
A
L;
E
t=2 i=2
Using
~hese
f
t
D- 1/ 2X'M XD- 1/ 2
it
results in (5.12), we obtain
where
It follows that
(5.14)
where
On the other hand, we obtain from (5.11)
T
+
E
t=2
T-1
+ E
f
2
D- 1 / X'M
·t
A
E
t=2 i=2
A
u
• t-
+ E
i=2
f. D- 1 / 2X'M. u
~.
~'-
(5.15)
91
Now,
But, since
M
ll
is idempotent,
= O(M)O(M) = O(M2)
Also,
X'M
11
= (AT)-lX'JAT
~'<
elements, and
V- V has
,which is a matrix of uniformly bounded
M(M+l)A
hence
which
impl~es
that
Therefore,
Var(D
Hence
-1/2
X'Mll~)
•
=
non-zero elements (if
T> 2M );
For the third term on the right-hand side of (5.15) we have
T
r;
t;:=2
But
T
-1/2
Var( r; D
X'~.t~)"
t=2
T
T
r; D- l / 2X'M (V •t
t-2 s=2
+ r;
~)M • s XD- 1 / 2
where
T
r;
t=2
= O(A)O(A) = O(A 2)
and the second term on the right-hand side converges to zero as
T
-+ co.
Then,
Thus,
93
Similarly, for the fourth term on the right-hand side of (5.15) we
have
A
r;
i=2
But
A
-1/2
Var( E D
i=2
XIM. _u) •
1
•
A
L:
A
E
i-2 j-2
•
A
L:
A
L:
D- 1/ 2X' M, VM XD- 1/ 2
1
0
J'o
D-1/2XIM.~, XD- 1/ 2
1"
i-2 j=2
J.
where
= OCT)
A
L:
D- 1/ 2X'M. XD- 1/ 2
0
i=2
.. O(T)O(T)
1
= 0(T 2 )
and the second term on the right-hand side converges to zero as
T
~CO
0
Thus,
A
-1/2
Var( E D
i=2
Therefore,
X1M. u)
1
0
-
= OCT 2)
•
94
For the last term on the right-hand side of (5.15) we have
T-1
A
L:
L:
t=2 i=2
But
T-1 A
2
L: D- 1/ X'M. u)
Var( L:
~t. 2
t=2 ~=
T-1 A A
D- 1/ 2X'M
.. T-1
t=2 5=2 i=2 j=2
L:
L:
L:
L:
A
.. T-l T-l
L:
~
T-l T-l
A
A
L:
L:
L:
L:
. 2
t=2 s=2 i=2 J-
+
L:
L:
it
D- 1/ 2X'M
t=2 s=2 i-2 j=2
it
where
A
A
L: D- 1/ 2X'M ~M XD- 1/ 2
i t jS
. 2
t=2 s-2 i=2 JC
L:
E
L:
T-1 A
1 2
1 2
= L:
L: d t D- / X'M i tXD- / = 0(1)
t=2 i=2
and the second term on the right-hand side converges to zero as
T
"'00.
Thus,
T-l A
1 2
Var( L:
L: D- / X'M. ~ "" 0(1) •
t=2 i=2
~t
This implies that
T-l
A
L:
L:
f D- 1/ 2X'M. u .. Op(1/(AT)1/4) •
t=2 i=2 t
~t-
jS
D- 1 / 2X'M. VM. XD- 1 / 2
~t
JS
1 2
( V - V)M jSXD- /
T-l T-l
VM XD- 1 / 2
95
Using these results in (5.15), we get
-1/2 *-1
D
X'V u
= dtD -1/2x'v*-1~
0
+
~2
(5.16)
'
where
Substituting results (5.14) and (5.16) in (5.10), we obtain
where
From this result and Theorem 12.4 in the Appendix it follows that
l 2
D /
d[ -~)
A -+ co and
and
Dl/2(~ -
a)
T -+ co such tha t
have the same limiting distribution when
A
= CiT
for Some
Ci
> a •
Now, according to Hannan's result, for large T
96
where
G
= D-1/2x,t-1xD-1/2.
Therefore, under the conditions of
Theorem 5.3,
*
1/2
~ Approx.
-1
D
(~-~)
_
N (O,G ) •
P
~
So, for large
A
and
T, the distribution of
approximately multivariate normal
*
~
is
In this way,
approximate tests of hypothesis and confidence intervals on the
regression coefficients
~
can be constructed.
97
6.
ESTIMATION FOR MODEL B
In this chapter we extend the results of the previous two
chapters to Model B specified in Section 3.3.
We recall that Model B is the same as Model A except that the
termS
i
... , A
= 1,
, are omitted.
The consequence is that the
covariance matrix
V
covariance matrix
V of Model A except that the term
of the vector of observations is the same as the
VIV
is
I
omitted.
It is easy to extend the considerations on estimation of Model A
to Model B and particularly to show that all the results established
for Model A hold also for Model B.
The proofs consist in checking
o
X
each step in the corresponding proofs for Model A with
substitutes for
X and
~, respectively, and
VI
and
We note that all considerations concerning the matrix
X
for
since the only assumption with respect to
theory developed for Model A is Assumption 1.
VI
X
X
and
~
as
omitted.
remain valid
used in the
In what follows we will
comment priefly on the necessary modifications.
The first consequence is that Seely's equations (4.8) for estimating
the variance and covariance components become
B~
=c
(6.1)
where
B
element
tr(NVi+lNVj+l)
element
y' Nv i+1Ny
(k
=
2,
o •• ,
M+3)
,
with
and
N
is the
c
=
(M+2)
is the
x (M+2)
(M+2) x 1
I - x(5c'X) -lX'
,
and
as defined in (3.8) and (3.9) •
matrix with
vector with
0
Vk
and
Vk
.. th
~J-
.th
~-
98
The proposed estimator for the set of regression coefficients of
Model B is then
=
~
'*
- ~--1- -1-'--1
(XlV
X)
X V
~
(6.2)
where
,....
'k
= cr~(JA G9
V
with
and
IT) + (IA
®QTATQ~)
defined similarly as in Section 4.2.
Theorem 6 1
0
Under the specifications of Model B, the estimator (6.2) is uno
biased for
§..
Proof:
The proof is step by step the same as that of Theorem 5.1 with
the adaptations mentioned previously:: substitution of the symbols
X
0
N
and
"
= 02a
\11
(J
0
§.
by
and
X
VI
N
.
and
Ii , respectively, and the omission of
0
We observe that although
A
is fixed by Assumption 3a, the
sample size can be increased by increasing the number of time points
T.
So, the asymptotic considerations that follows will aSSume the
sample size increas ing with
T ....
CO •
To establish the asymptotic properties of the estimator (6.2) we
first prove results analogous to Lemma 5.3 and Theorem 5.2.
we use results contained in Lemmas 5.1 and 5.2.
For that
99
Lemma 6.1
.
,
Suppose the regression equations (3.10) satisfy the assumptions
of Model B.
Let
=
w
k
1/2
= 0 (T)
k
k = 2, ""
P
= 2,
•• 0, Mf.3 •
Then,
M+3 •
Proof:
According to (5.6) and using results of Lemma 5.2, we obtain
2
= T- 0(T)
=
l
O(T- ) ,
k
=
2, ••• , Mt3 •
The theorem follows immediately by Corollary 12.1.
0
Theorem 6.2
Suppose the regression equations (3.10) satisfy the assumptions
o
o
of Model B.
If
~
is estimable, then the estimator of
~
provided
by the solution of (6.1) satisfies
k = 2, ••• , M+3 •
Proof:
o
If
~
is estimable, its estimator is the (unique) solution of
the system of equations (6.1), i.~., ~
= c.
By considerations
similar to those of Theorem 5.2, we obtain equations analogous to
(5.7) with the first row and first column of the matrix on the lefthand side omitted,
"~
replaced by
~
from the vector on the right-hand side.
and the first element omitted
Since
A
is now fixed, we
100
,...
.,'t:
Since
matr:i.K g
V
pi
is positive definite,
= 7..-AT1 / 2
(Q
IV1 Q )
A t:I T
is a real
I
Then, as poi.nted out in Section 4.1, the expression (7.4) is
~
equivalent to
=
§,
(X~'~ UX'k)
Therefore, the estimator
-1
'"
"k
§.
Xi( I y"'f<: , where
Xi\'
= piX
and
y..~'(
= ply"
.
can be obtained by the OLS regression of
The transformation of the data prior to the use of the
OIS
proced~re
can be carried out in two steps:
transforma tion of the data matrix into
We first perform the
(x..id(X·k~·~)
= (QA ®QT)
I
and then perform a second transformation which gives
(yi( Xi()
'j\~~/2
z
(l.fd'
X""f~)
•
We note that
(ZI1 ••• zIT z21.·· .zAT) I
of the vector
~.
(QA
® QT)
I
~
,where
(X. X) ,
=
=
is, in fact, a two-way Fourier transformation
This can be accomplished by first performing a
Fourier transformation of each of the vectors
(z11 ••• zlT) ,
(z21. g.z2T) , • g., (zAl·· gZAT) , and then a Fourier transformation of
each of the vectors
transformation.
We note that, again, this two step transformation can be
performed wi.thout the need to store any
AT x AT
matrix, because of
the Fourier transform algorithm and the diagonal form of
",-1/2
!tAT
•
~
!
.j'
.100
omit the symbol
by
A.
Now, if we divide both sides of these equations
T. we obtain
~T
en
-T
c
-T
23
-c
-T
33
-c
I-T
T 34.
c
32
I-T
T 42
I-T
T 43
~C
1-T
T M+3,3
...
I-T
I-T
T 2,M+3
1 - Ny
-y'NV
I-T
T 3,M+3
-1.'NV
T
3 Ny
2
T"'-
;i.
I-T
-c
T 4,M+3
1
-
-
1
-
-
=
"T1.'NV4 Ny
-T
c
M+3,M+3
'Tc M+3,4
~c
·~c
-c
-T
c
44
"~c
1-T
rcM+3,2
...
I-T
T 24
c
(6.3)
where
M+3 •
i
1.
-T
CO
c 1
kj t=l
As
are bounded sequences of real numbers,
k, j = 2, ••• ,
T ... co, the matrix on the left-hand side converges to the
nonsingular matrix
c
c
32
33
o
o
0
a
C =
where
C
=
kj
0
0
o
o
-T
lim c
T...
.
oo k J
c
a
44
o
c M+ 3 ,M+3
It follows from (6.3), according to
Corollary 12.2. in the Appendix, that
v
E(y)
o
101
l\ f'art.icalar consequence of this theorem is that under Model B,
if
~
estimable, the estimator provided by the solution of
1.3
eq'Jat:i.C)'lf:) (6.1) is a consistent estimator of
T~~u.:
\J
•
rem 6.3
_ _
.~.,--,e,,-,""'-~"-
-.C_>:~"
Tre e8timator (6.2) for the regression coefficients of Model B
'1/2
0'
\ihe re
D
f.l~lnentd
P'C"CJ;,J
~
o
<a - ~)
the
1.:'0
diagonal matrix composed of the diagonal
Xi~-lx
of
=
* (X'V* 1X)-lX'v* 1y .
[=:
and
f;
The proof follows along the same lines of the proof of Theorem
5.3.
By similar considerations, we obtain
-
-
*
*
= [:5 1 / 2 (X' V-IX)
-101/2]13- 1 / 2X,v-l~
where
0A ~.£.) +
U~,
·'k
'"
T
f
·t
t=l
aT
"'2
dT
,....
d
t
A
L: f M.
t ~t
t=l i=2
fM
+L:
·t .t
T
'"
d
Acr b +
::::
t
+ L:
V
d
=
1-1
T
--1
Cd'
Now,
and
'k
~'J-l
f
e
t
t
0
-
d
T
0
d
t
(6.4)
t
= 1,
••• , T-1
= 1,
••• , T
(6.5)
~02
M
t
= 1,
••• , T
i
= 2,
••• , A
l.t
-
)x,ry- 1x5- 1/ 2 = d
c
t
= 1,
••• , T-1.
--1/2=-1--1/2
. X'v XD
D
T
T
+ L:
t=1
+
A T-l
L:
L:
--1/2-
ftD'
---1/2
X'MitXD
.
•
(6.6)
i=2 t=l
NevI, "'q:-a.'1.d
;,ng
by T'lylor's series about the true parameter point up to
the first partial derivative terms, we obtain
in the line joining
thL" gives
-Ad
f
T
•t
- <1 1)].
and
103
we would obtain
~j~!.la~ly,
"k
__ ·l
0'~
spectral decomposition of
V
~k
~
gives
T
I:
--1/2--1----1/2
D
X'V
~XD'
.<
t""l
A
+
[;-'8.",
T
q-1--1/2-,
2:
2: a D
i=2 t..,l t
---1/2
X M. XD
l.t
since
-- l/2-,:-l---1/2
n
"_
X V XD
= 0 (1) ,
~ (~~
+
t=l
~t)~lD-l/2X'M.tXD-li2 = 0(1)
:5 - 1 I 2v I -.t
M Xi) -1/ 2.
L',
= 0(1)
•
Iherefsre, according to previous results,
;imilarly, we would get
1:"Lng
+.::her.~e
results in (6.6), we obtain
,
•
104
- q \7~-lCl T-1-(1)
.Tl·',(
·X' -1-1/'
. ' . x-,
.. ) . D· ...
"k
51 / 2 (X iV-IX) -15 1 / 2
+
/);k
1
(6.7)
On the other hand,
--1/2-XiV:-1-u
D
=
+
A.
I-I
L:
L
1.""2 t""l
By procedure similar to that. uBed
HI
tbe proof of Theorem 5.3, it
could be shown that
:/..:
ai'-l/2xJcv-l:~ +
62
(6.8)
Finally, substituting results (6.7) and (6.8) in (6.4), we
obtain the result of the theorem.
0
Now, according to Hannanis resul.t, for large
Then o under the conditions of Theorem
where
6.3~
T
it follows that
--1.)
N ((J,G
p' .
105
7.
COMPUTATIONAl, CONS IDE RAT IONS
In this chapter we discuss methods of reducing the storage
required in the computations for the estimation method described in
Section 4.2.
If the variance and covariance components are estimable, their
~
estimators
are the (unique) solutions of the set of equations
(4.8), i.~.,
=c
B~
~=
where
(7.1)
(V • ..VMt-3),
1
(M+3) x (M+3)
(M+3) x 1
is as defined in (3.8),
ijth
matrix with
element
~--
i.s the
tr(NV.NV.)
XINV.Ny.
1
and
J
1.
.th
vector with
element
B
c
is the
So, the computation
of the estimate,s, of the variance and covariance components requires
the calculation of the
... , M+3
..., M+3 .
(i s; j)
i,j = 1,
i
= 1,
(M+3) (M+4)/2 distinct quantities
, and the
(k
= 1,
quantities
J
1.
,
,
Z'NV.N.l
1.
These computations apparently demand considerable
core storage, since they involve the
V
k
M+3
tr(NV .NV .)
••• , M+3) •
M+3
distinct
AT x AT
matrices
Fortunately, however, exploitation of the
special pattern of these matrices allows storage compression.
We note that
tr(NV.NV.)
~
where
P
J
= X(X'X)-lX'
= tr(V.V.)
1. J
•
Let
0
- 2 tr(V.PV.) + tr(V.PV.P) ,
1.
be an
J
AT x p
1.
(7.2)
J
matrix with
0
columns which spans the same space as the columns of the matrix
Let
form
be the
column of
o •
normal
X •
Then, expression (7.2) takes the
106
tr(NV.NV.) "" tr(V.V.) - 2 tr(O'V.V.O) + tr(O'V.OO'V.O)
J
1.
l.J
+
+
p
2:
°kv .00' VJ-k
.0
k=l -
p
2:
1J
l
p
2:
J
l
= t r (V 1.. VJ.)
(7.3)
0k'V.ok,ok"V,o k •
k==l k'''''l -
Thus, the computation of the matrix
r-
-
T-
B can be achieved through the
calculation of each of the three terms on the right-hand side of
expression (7.3) for all the combinations of indices
i
~
j.
(i)
i
and
j
,
These can be done as follows:
tr(V.V.)
l
J
can be generated directly by using Lemma
12.1;
The relevant fact is that the vectors
V.(Ot )
·~l
,
Ot
== 1, ••• , AT
i == 1, ••• , M+3 , may be generated directly as needed, using
Corollaries 12.4 - 12.7, and need not be stored.
Thus, for example,
107
.Y.l(Q')'£k'
£k
Q'
= T(i-l)
+ t , is the sum of the elements of the vector
in the positions
tr(NV.NV.),
~
J
i,j
T(i-l)
= 1,
+ s,
s
= 1,
••• , T.
This allows that
••• , M+3 , be computed with a substantial
saving of core storage by the use of subroutines that perform the
computations indicated in (i), (ii) and (iii).
~'NV.N~
:L
We observe that
the entries of the vector
c
= u'V.u
~~-
,where
~
= (I
- OO')~"
So,
on the right-hand side of equations
(7.1) can be computed by the same subroutine that computes the
expression in (ii), by setting
2.
The estimates
j = 3,
io~.,
V = 1
•
AT
3
obtained as the solution to the set of equations
(7.1) is used in the next step to estimate the regression coefficients
o
~.
u
Now, the parameters
estimated by (4.12),
~
of the regression equations (3.3) are
i.~.,
(7.4)
where
*
V
is the matrix defined in (4.11).
Since
*V
is the matrix
";'~
V defined in (4.2) with the variance and covariance components
replaced by the corresponding estimators defined in Section 4.2, it
,...
ok
follows by Corollary 4.1 that
where
Q
A
is the
A x A
diag(~ll •••rlT ~21 •• ·1AT)
V
can be factored as
N
matrix defined in (4.4) and l\:AT =
is the diagonal matrix whose elements are
defined in Corollary 4.1 with the estimates, computed in the previous
step replacing the corresponding parameters.
109
8.
NUMERICAL
ruus TRAT ION
In this chapter we illustrate the estimation method and the
computation procedure for obtaining the estimates of the regression
coefficients of Model A.
mosquito abatement.
In the exemplification we use data on marsh
The data is a set of yearly observations collected
at 29 selected mosquito control distri.cts i.n the Southeastern coast of
the United States during the years 1959 - 1971.
Detailed information
about the data is given in DeBord (1974).
We consider the regression equations
u.
It
i. :;; 1, ••• , 29
(0)
Yit
(8.1)
,
t :;; 1, ••• , 13 , where
denotes the value of the per capita expenditure
for the
(1)
x
(2)
x.It 2
itl
i
th
:;; 1,
th
t-
location in the
i:;; 1, ••• , 29
t
:;;
year;
1., ••• , 13
denotes the number of mosquitos per light
trap night;
(3)
x
it3
(6
(4)
x
(5)
x
(6)
x
:;;
it4
it5
it6
denotes the wage rate raised to the
6 power
labor's share of total expenditure);
denotes income per worker (deflated) ;
denotes population in the district (N)
denotes state grants per capi.ta for mosquito
abatement (deflated);
mosquito abatement/N ;
x. 6 :;; state budget for
l.t
_e
llD
(7)
x
it7
denotes tourism per capita;
x.1t 7
= number
of employees working in all lodging establishments
(tourism proxy)/N •
In this illustration the error term
of relationship (8.1)
is assumed to satisfy the hypothesis of Model A, with the random term
e
a sample from a realization of a moving average process of order
M < 12 •
It was mentioned in Section 3.1 that the assumption of a finite
moving average process for the residual term
e
it
is effectively
equivalent to using a truncated estimator of the spectral density of a
•
general stationary process.
estimators
(~.~.,
In fact, any of the known smoothed
Hannan (1960), Anderson (1971»
place of the truncated estimator (4.9).
could be used in
The properties of the
estimators discussed in Chapter 5 would still hold for such estimators.
However, since the choice of any smoothing operator must depend upon
some prior hypothesis concerning the smoothness of the spectral
density for its justification
tend to zero as
h
(l.~.,
the speed with which the
y(h)
increases), we limited considerations to the
truncated estimator.
The choice of
M depends on the nature of the spectral density
being estimated and the sample size.
Too small an
peaks being overlooked, and too large an
with too much irregularity.
M can be.
M may lead to
M may yield an estimate
So the smoother the density the smaller
One approach is to fit the largest possible value,
M = T-2 , and plot the estimated spectral density.
Then, reduce the
-e
III
value of
M sequentially until the smoothest spectrum which reasonably
approximates the first plot is reached.
(See the authors cited above
for additional comments and suggestions.)
For purposes of illustration, we fitted all possible values of
M
,l.~.,
M = 0, !, ... , 11.
The 018 estimate was also computed.
The computations were performed in electronic computer using a system
of subprograms developed by Dr. A. R. Gallant.
The regression co-
efficient estimates and the respective estimated standard errorS are
presented in Tables 8.1 and 8.2.
It can be seen that the corresponding estimated regression
coefficients and standard errors vary considerably for small values of
•
M and tend to get closer for larger
and 11
M values, namely
,
M.= 8, 9, 10
The form of the spectral density estimator (4.9) explains
these results.
The last step in the estimation procedure is a
weighted regression whose weights are linear combinations of estimates
of the variance components and multiples of the spectral density
evaluated at specific frequency points.
The truncated estimator (4.9)
is a weighted Sum of estimated autocovariances with cosine .functions
evaluated at particular frequency points as weights.
Thus, the
truncated estimator can oscillate considerably as
increases if
M
the autocovariances do not decrease sufficiently fast with increased
lag size.
This is the situation of the present example, as shown in
Table 8.3.
The consequence of the wide oscillation of the spectral density
••
estimate for different values of
M is the assignment of extremely
high or low weights to Some specific frequency points which allows the
-.
•
•
.,
... _--.--
Table 8.1
Regression coefficient estimates
..
-
~
--
M
OLS
B,
82
a3
a.
a5
0
-7.09827 -5.2B79
1
2
,
4
5
6
7
8
9
10
11
-2.69900
21.,n5
16.70ll
-2.04218
0.76990
-5.ll987 1ti.70'0
-2.59757 -5.46789 -5.07070
-5.'5B2
-0.01107
-0.12259
-0.04160
-0.05435
0.01555
0.0'482
..().O2401
-0.05455
-0.05302
-0.04525
-0.04204
-0.0469D
-1.8'629 ..o.~a050
...1.74491
..1.42888
-1.95205 -0.67516
-0.46988
-0.97054
-1,95202
-1.25154
-1.16591
-1.B475
-1.188(8
1.26625 -0.47402
1.20724
1.50950
1.27011
1.52274
0.01596
1.57485
-0.55116
1.,15419
0.79916 -0.n528 -0.47402
1.06204
0.57655
-O.4,:no -0.06529 -1.24026 .1.02009 -0.49n6 -0.43510 -0.48762
-:'.• 02009
-0.48547 -0.48224 -0.50051 -0.49180
8,
0:55955
0.27909
0.1012}
0.17'95
0.12729
0.19987
0.26045
0.1808'
3.12729
0.17409
0.18876
a,
0.54587
0.29486
0."785
0.B770
0.098&1
0.17614
0.09910
0.25497
0.09861
0.25514
0.26291
0.19002
0.24046
----_
... - ..
0.17120
__._0.26411
....
....
N
Table 8.2
.
- - ,.
EstImated standard errors
...... .
OLS
0
1
2
,
4
,
''1.
1.70264
1.71588
0.109'9
0.10955
0.10982
1.2'216
"2
0.02988
0.02'75
0.000015
0.000015
0.000019
",
0.'''8'
0.'6621
SR4
0.22542
""5
"6
"
•
•
•
•
K
6
7
8
9
10
11
0.95'48
1.568'0
0.11161
1.51090
1.57'82
1.60556
1.6259'
O.014U
0.0102'
0.02020
0.000028
0.018'2
0.01984
0.01961
0.0200'
0.000212
0.00027) 0.000'05 . 0.2095'
0.14757
0.'1110
0.000446
0.28928
0.'1597
0.'1686
0.'1928
0.22810
0.00022),.
0.000270
0.00:)276
0.1557)
0.115'"
0.20645
0.00040'
0.20012
0.20911
0.21192
0.21(62
0.0'525
0.08107
0.000118
0.000111
0.000120
0.06'55
0.05114
0.07704
0.000175
0.07)8'
0.07678
0.07608
0.077'2
. 0.0'649
0.04045
0.000019 0.000022
0.000022
0.0227)
0.0167'
0.0'121
0.0000'2
0.02659
0.0'01'
0.0297'
0.0'089
0.04022
0.04864
0.000042
0.000047 0.0'281
0.02509
0.0'667
0.000069
0.04285
0.04520
0.04525
0.046"
0.000045
....
....
W
•
•
•
r,
•
Estimated variance and covariance components
Table 8.3
M
0
1
2
)
4
5
6
7
8
9
10
11
.2
0."198
0.'2)58
0.'1581
0.'07'1
0.'0017
0.29407
0.268'2
0.2e008
0.27068
0.25800
0.27425
0.285)4
0.
0.00'653 0.00'6)2
0.00,605
0.00'574
0.00'5'9 0.00'5)'
0.00l519
0.00l505
0.003<99
0.003<99
0.00'498
0.003<98
0.08)785 0.092666
0.10075
0.10950
0.11679
0.12217
0.12879
0.1l109
0.14650
0.14918
0.1429l
0.1'184
••
-2
'1(0)
.... (l)
0.052560 0.060752
0.069581
0.076895
o.oeng6
0.068921
0.097221
0.10665
0.109"
0.1o,o8
0.091988
'Y (2)
0.0"8"
0.05l7<2
0.061095
0.066505
0.07lU8
0.081462
0.090909
0.09'59'
O.OBn40
0.076254
0.0·U871
0.052264
0.057684
0.064'28
0.072659 0.082110
0.084796
0.078544
0.057456
0.03<27) 0.0'9719
0.046'02
0.054711
0.06'161
0.066845
0.060592
O.O~9504
0.Ol8025 0.047458
0.05010
0.04'889
0.0)2802
0.0"714
0.045847
0.Ol9592
0.0285o,
0.02810) 0.0l1578 0.040265
0.0"011
O.O2292}
0.Ol0276
0.024015
0.012927
0.006565
0.000'17 -0.010778
~(3)
;(4)
;0)
0.02'01) '0.029695
~ (6)
0.025l7l
0.04'16'
;(7)
;(8)
~(9)
yOO)
I
0.02758'
-0.012'67 -0.02'461
y(ll)
,-0.0165'6
.....
.....
./>0
·'
~.
115
regression through particular frequency points to dominate the results.
If the regression lines for different frequency points are
heterogeneous, as in our case, then very distinct estimates for the
different values of
M may result.
Table 8.4 presents the estimates of
2lT
times the spectral
density evaluated at the specific frequencies indicated in (4.9) for
M = 0, 1, ... , 11
Figures 8.1 - 8.12 show the corresponding plot
of the estimated spectral density.
estimate~
values.
We note that each of the
of the spectrum for M = 1, 2, 3, and 7 has two negative
In order to obtain positive definite estimates, the negative
values were arbitrarily replaced by the corresponding values of
•
max~
teT
t
The conclusion from the above observations is that for small
values of
namely
T
we should assign to
M = T-2
(= 11
M the largest possible value,
in our example).
The knowledge of the
estimated autocovariances and spectral density for Some other values
of
M should be taken into consideration to evaluate the reliability
of the estimates of the spectrum.
In case the spectral density has a
too wide range, an improved estimate can be obtained by a smoothing,
say a moving average smoothing, of the estimated spectral density to
put the spectrum estimate within a conveniently narrow band.
-.
•
.,
Table 8.4
•
Estimate of 2n times the spectral density
..
•
AAOIAI'\.S
•
o,;oJ""ArE
J 2~
O.llt3
I
I
0.251
:.11(,[; 00
1
,J. 314
0.377
O.sqljt-Ol
C.4"0
'J .. :::(,7r.-Ol
I
0.5r.3
0.56'3
0.626
C.l "113£;-01
O.l;)i'f;-02
1
O.1)1j3
I).
O.I~Qt
C.e4lSE-OI
(j .~~S[;-C2
0.7'",.
c..7e7F.-a?
C.I 1 "\1;-01
o. ell
C.17i.f'-OI
0.6'11
0.6'tO
(,0.2'=5[-01
o. r;42
O.2CRr.-OI
I. oJc;s
0.1 nl;.-Ot
G.114;:-01
I.C6e
I
I
I
I
1
I
I
I
I
I
1.445
O.J72F.-02
1.~O!t
C.<;I~t-C2 .
1.57.
0.132'E-"1
I
I. (:3"
I. flO; 6
G.14 f H:-Cl
I
(,.14'iF-,),
1.7:";')
lo. e22
I . ~~'!o
I. c;.4 II:!
2.t:J I I
O.II2f:-OI
".711(.-02
C."":H-CZ
I
I
C.J4~t-C2
2. C 7J
O.3t>3f-C2
c.eo;:.H:'-02
2.136
c .. 7~>if:.-n
2.1"?
c..or. ~:Jf:-"2
2.?62
o .<.;6ge-02:
C .1'.f,6f-02
C.c.l\la:-o?
J~'5
2 •.He
2. oil 50
2.!:1~
'.6~~f-C2
C. i'.JLt-CZ
Z.576
0.1 SC.f.-C;,?
2.c.J9
2.7e»S
C.IQIF.-,n
C.JIl;F-O:O
O.454E--O?
2.827
O.~e2e-02
2.6')0
O.SQ4F.-OZ
C .!.4o.;f:-C;:>
Z.7O?
2.~~:J
-c:!
J.OI~
C .4',,>'
J~079
C.J77F.-02
3.1_2
·O.JoIl"E-02
•
-• •
I-
•
1
•
•
•
•
•
•
•
-
I'
I
I
I
I
I
I
•
•
•
•
•
1
,.,.
•
•
-•
--
•
•
•
t·
C.464E-CZ
0.0
0.0
c.o
o.c
',,-
I
I
I .. I JI
1 .. 1'9.
1.2 'i 7
I. J 1 ;
1.3e2
2.
•
_________________________________________________________________ e._________________________________
00
O.lt~·e 00
~.1 e:3E 00
0.1 ~"e CO
0.'
•
-••
-•
,--
•
•
•
I-
I-
I •
I •
I
I-
t·
-
I'--~'
.. ' .. -- ... -'_ .. -. __ .
Figure 8.1
. d
.. ... .. - .... ~_
.- - -
Estimated spectral density. M= 0
.. ._"
--
. -...
....
........ .
..
.
•
RAorANS
0.315r:-:)1
O •.n4F.-CI
0.IZ6
C.Jl.JE-Ot
IJ.JI2E'-OI
O.J~a
0.)14
0 • .)77
·0 .... 40
.
1,l."e·,F-CI
D.2e~f:-Cl
0.691
0.7'54
lJ.:I!1U;-OI
';".2~:?t'-OI
o.a~o
'.:i.~S4F.-la
0 .. <;42
l.o'A!>
1.0'\6
1.IJI
1. I lj,4
1.2'57
').?"~r-IH
c.z.cqr.-OI
n.ZI9~-GI
C.2r;""-OI
0.1'0<;1;-1)1
o oIt'G!.:-!)l
J.J~~
C.17fOF-CI
1.445
I. ~(;I!
I.'HI
I. f.)"
I .. "l'> e
1.7':.9
1.1l ZZ
I .. !:S':!':\
o .ll'."r. -01
C.I ~P',".-CI
C.I .. -rr~-Cl
CI.l~7r.-Cl
.1?7C-n I
0.11".<=,-01
0.1 CM;-Ol
(I
C.<;~~E-G2
I .. c;,"!J
') .. '!59F-02
2.1)11
~. 0 1 J
7. .. I ~~~
Z. 1 ';')
1).1t u:-c.2:
0.t,(;"l-02
O.!.7dC-OZ
c.,\r.lr:_~2
Z.l62
O."Z"l;-~l
2.325
2.3"1A
C ••nc.F.-Ol
3.1"2
•
G .;:O:!7'.-OI
a. .]1"
2.1',5
2. P,~7
2. UQ~
2.";':»]
3.')lb·
.hefr,
•
C.2(r,~-OI
o. !JJ1
2.GJ'J
2.7;:12
•
c .?~Sf.-C2
G.I~(>I-02
O.121f'-02
O .. (;2C~"03
C.d15{--04
o.t';
0.0
0.0
0.0
e.e
.,.0
c.o.
o.t
•
•
•
0.2<;')"'-01
o .2t;1lE.-r,ia
0.1'018
Z.51.:3
2. ::116
-----------i~-------------------------------------------------.----•
C.:! lor-Ol
C.3l';.71-CI
C .. 303t-~J
0.5)J
0 .. :.0"'-;
z. "':;?
•
OOOr ...... TE
0.'
0.\,0"3
O.Z~I
•
•
•
'.
'
,-• -•
-•
--
•
•
•
•
•
•
•
•
•
•
•
•
•
•
-
•
•
•
•
•
•
•
'.
•
•
•
•
•
•
-••
\.
....
....
0>,
•
•
Figure 8.2
Estimated spectral density, M = 1
•
0.0
0.06:'
(1.126
,;).19:1
O • .?~I
O• .J 1"
\/ • .J 11
O. "Ill')
O.50:J
Cr:;OINATE
C .IIlal'lF.-OI
O."l)~f -01
G.IIle,;cr-OI
c •• e~f:-Ol
C o 47::,r:-CI
0."',0:-01
0.44"[-01
O.":;~F'-Ol
O.'\C'f.·F.-OI
O.~6'5
O.J8"r.~CI
0.62 ..
O.Jt IF-OI
O.3JH.-Ol
O.6QI
0.7'">"
.0.eI7
--_:.._--~... _-----~---------------_ .. _--------------:--~....,.---------------I
I
O. c;4Z
1. C)~
O.23CF.-Ol
O.2c:,r-Ol
C.l HE-OI
I
I
I
I
C.129f"-OI
I •.;Uq
c .EJ4f-02
0.1
l).b)ljF.-C:Z
1. "45
I. 5 'J~
O.4f4E-Q2
I.'HI
,
I
I
O.
Jcc;r-o&'
1.0<:(;
1.1"ir,l
C.O
1. A22
(
I. tH:l~'
2.CII
0.0
0.0
0.0
••
Z. :11.]
2.1.36
c.o
1.0.0
•
..
C .;:~OI.-03
o.loJr-02
0.1 c;or-02
G.2ij""(-02
C.J~ll':·O?
2.4']0
2.!i I .)
C.oIHIJf -(\2
2.576
C.~1l\r-oz
Z.
e .J')
2.7.12
2. 7li~
2. A~7
2.AlO:O
2.9!,J
J.O 16
J. ':'7Q
J.142
0.(;74(-02
O.1oJf-IlZ
0.1 14(,(-02
0.919£-(:2
C.ClHf"OZ
0.1 (;:,..--01
O.IC7E-OI
0.1'9£-01
(I.lh:tE-OI
•
•
I
I •
i).176e-02
O.(;f3E-03
0.0
Z. J ';' oj
•
•
•
O.1';5f-OI
I. )t·Z
2.199
2.2(,1
2 ••1,~5
•
•
~-!E;-CI
•• I Jt
I . I C)O\
1.&''57
1.0;4"
•.
•
O.lPoH;-QI
c.<:~71-01
1.6J4
\,
c.Jlor-o!
c.eu'J
1.l)~U
•
•
"
A"OIAh$
,
•
•
•
•
•
•
•
•
•
•
.. •
•
•
•
•
•
•
•
•
•
,.
I •
.I
I
I
I
I
I
I
I
I
I
I
I
I
•
•
•
•
•
•
••
•
•
,.......
'"
•
•
•
Figure 8.3
: Estimated spectral density, Me 2
•
•
A.OIA~S
L~OINATf
0.'
O.7J~E-Ol
0 .. 0 (~3
C.126
0.1 R9
O. Z'B
C: .. J 14
O.(:7J!:-OI
C.f,4;.!(-"Jl
C .6(71.;-01
:).317
C.4 .. r"
0.519[-01
C.7:~(-'H
O.t1lj,Zt-OI
0.,,,,')(7-01
'J.5~5
O. e.2B
O.416E'-OI
C .JeU.'-OI
0.6'')1
C dG'lE-Ct
o. 7~A
0 ..
0 .. 1111
O.13P.Q
O."C!;F.-Ol
O.-;4l
C.ll(,l"-Ol
1.0J5
O.764r;-02
O.4~tlt-02
0.212£-02
0.1 efl(-OJ
1.257
C.O
1.319
0.0
J .. Jfl2
O.C
G.O
I ..
1.
soe
~
11
J .. t;J4
1.6,," ":l
I .. 759
l.flZ2
t. 'Hl5
1.'''''
2.011
0.0
0 .. J2C'f-OJ
0.114E-02
O.JJ4F-O?
O.SCZE-I)Z
I-
--,---
II
I
O.671E.-02
C.f!:!!"·f-OZ
I
I
O.<;71E-C2
I
2.1<;?
O.127E-Ol
I
I
I
I
2.2~2
,O.10:!6f;-OI
I
2.u1J
2.136
2.325
2.J;JO
2.4'30
2.51.J
2_576
2. t Jq
2.7':2
O.lC.I'JF-Ol
0.11'3(-01
C.124f-Ol
0.1221;;-01
C.115F.-Ol·
O.IC4F:-:)1
O.Cf21E-02
O.7dCF.-o:)2
O.t,l!9(-02
I
I
1
I
O.474F.-r)2
I
I
I-
2.7(,5
2.1127
2. A';O
O.J22F.-02
O.lelf:-02
a.552f-03
2.q~3
0.0
0.0
,J.016
--
C.J~Af:-r.l
l-"'~tI
1."45
\.~
2~!:.E"-CI
1. I J I
1. I <; ..
;J.e,?
C.O
3.142
0.0
•
------------------------------------------------------------------.---------------------------------,,
-
~.56!>f~-Ol
O.~JJ
"
I
I
--.-
•
--
--
- -
--
--••
-
-, -
•
•
-
•-
•-
•-
...
o'"
----- ......
_.- .... _--..Figure 8.4
Estimated spectral density, M = 3
,
•
--
RAOIAf\5
Q~OJr.:4Te:
0.'
Oe~OlE-r)l
O.O~J
0.13<;2£'.-01
0.226
C.JI!t.7E.-Cl
1"8
O.eZ7E-OI
G.77;::E-Cl
(I.
0.251
0.314
fJ. J77
C.440
C.7C7~-Ot
C.t:::JE-')I
c.';)e.""f.-G1
0.5(;3
C.46?t;.-Ot
".~65
O.
0.626
C.JC,.4r-I'U
C• ..,'11
C.2?9E-Ct
O. 7~'"
0 .. RI.,
C.l f;.:!E-Ol
C.l eSE-OJ
o. ;}!)~
!E~H:-Ot
O.51l~F.-02
D.?/.
O.lJ~f,-02
1 .. ot;-:;
0.0
I.Ute
0.0
1-111
1.1<;.
c.e
c.c
~. Z~H
C.):::tE-OJ
I.JI9
I. Jqz
1. 44 S
1. ~~ 8
1 .. '71
1.F:H
I. (:Go6
1.7:;9
1. "'2:.!
1. ees
(l.41J1':-02:
O.62tJE-02
Q.1l2H£:'-!)2
C.lelF.-OI
O.t 1"1[.-01
I). 12.1lF-:H
C.12'H.-Ol
O.127E-Ol
t.'H"
2 .. ') 1 t
2.~ 7J
2.136
2.1')9
2.262
2. J<!S
2.JtJ8
2 •• ~o
2.513
~
.2·::Q(-02
0.121':;:-01
O.ll1r:-')J
O.<06IlE-I)::!
c.a:::.11:.-02
Q.b74F.-02
O.51?E-"2
(j .. ~
76(-02
C.2t;IC-OZ
0.1 nf-Cz
0 .. J 24£:.-02
C.1IIE-C2
C.1JJf;-02
0.1 e1f:-1)2
O.2(jt·l;.-02
0.J611:-02
0.4"_"_02
Z.~53
O ..... ~Jr:.-C2
O.72JF.-1')2
C.7l'll'r-02
O.7eJF.-02
fJ •
----------~------------------------------------------------------.----------------------------------I
•
"-
I
I
I
I
I
I
I
II
I
I
I
I
I-
•
•
•
•
•
•
•
•
-.
•
•
-.
•
•
2.-:;11')
2. toJQ
2. 7n~
z. 7~S
2.627
2.eyO
3.016
3.!':11
3.I_Z
•
•
,
!')t)_F-02
•
•
••
I'
I
••
I
I
I
I
,
•
I
I
•
•
I
I
I
I
I
I
,.•,.
I
•
•
•
•
•
•
•
•
•
•
•
•
,. •
I'
I
I
I
I
I
I
I
•
••
•
•
•
....
....
N
Figure 8.5
Estimated spectral density, M = 4
RADIANS
OPDl~ATe
0.0
O.ll)"!l~
0.063
a.tOAr 0')
0.126
O. I OS
c..toet
'). ;><> I
o.eo;,el:-Ct"
O.,:H4
o• .J71
0.".0
o
00
00
.q~';lr-Ol
o.na-c.t
o• !.!",t.r.-C I
o. OS"'.]
C oS.Sf-C'
O.4Jt'.r-OI
----------~------------------------------------------------------.---------------------------------I
•
1).~1!I5
C.JJ'.!!-tli
c .:':.ltol'f-Ol
0.617
O.~C~I;-02
O.'l~O
C.?r.7I:-02
O.~4l
1I.f.CQE-Ol
(,.45f.r.-CJ
c..1J2E-CZ
•
O.2!f~-()2
C!.I\/j(,f,.-02
I'
I
I.
~"J9
1. 131
1. t <;l"
O.l".CJ'-~l
c.~7a-J2
00tl7Qr-02
I
O.F.<;~(-02
1.3112
" .t,I,;n;-o?
I
I
1.44 S
'J • 1 C"".-')1
I.
O.lIZE-Ct
1. !i71
O.JC?f.-Ct
heJ"
O.IC~r.-OI
1."'9b
J. 1~?
1." 22
!;.7l;BF-02
1).67H,-CZ
1.
~e5
I. ~4P.
2.011
2. ';73
2. 1 J'1
2. I 9'1
2.:1f:2
2.:J~S
2. J".O
2.
c.r,;I~Jr-02
o.~ebf7-C2
C .... P H.'-OZ
C.42Qr-C2
O.4t~(-02
•
(j.1::;'~~-02
O.7~,"[-C2
C.1"',.f'-G2
0.72..-£;-\)2
2.7JZ
(; • (.'11 l:-02
2. 76~
<:!.RZ7
O.~1(,E;-C2
J.l)Il'J
(l.4nF.-02
":) • oJ 7SC-G2
O.21]C.E-C2
O ..H.4f -02
). C 7q
... I
'.14,1
'.131t'-CZ
"!~r.-C;!
•
•
•
•
•
•
•
•
,•
•
I
I
I
I
I
I
I
•
•
••
•
•
,•
O.411oE-02
0.:: Jl!f-OZ
O.I... (<:l!-OZ
G.610.-C2
",'SI)
2.1\1;0
2. <1').3
•
I
I
I
I
'.4::''1F-Ol
2.51 J
2. ~76
1. ~ 'J')
•
•
•
-.
•
•
•
•
I. O!S7
1. J I?
~':6
\. ......
I
I
I
I
I
I
I
I
I
I
I
I
I'
0.1028
0.6"';1
'J.7':o"
1.0(,,5
•
•
•
,
,•
•
•
•
•,
,
•
I
I'
I'
•
....
•
t->
t->
,•
Figure 8.6
Estimated spectral density, M = 5
,.
•
RADIANS
OPOl~ATe
0.0
O.IO'H: 00
0.063
0.126
0 .. IOAr: 0')
G • 1 ~ r.:r: 00
O.II.l~
O.9~';lr-Ol
O. ?')1
O.er~ef;:-Cl·
O.JI~
O. 7r.a:-C 1
O.(,!",M:-CI
C .545F-t'1
O.4lN·-OI
O.J77
0.-\40
0.';"'.1
l).ti"i'
C d;!?l;-tll
O.LZa
G. :'A/lF-,1I
O.6"';J
').7';.4
O.I'.C'--')1
C. <; 1C E- 02
0.1:'11
O. 'ldO
0.le1(-02
1I.~05~-02
U.<;4l
tI.f..C'1E-OJ
1.0(,5
I.C'J'J
t.o 131
O.45(;r.-CJ
Ct.132E-C2
O.2H~-O?
1. 194
~'''6(,t-02
I. ~S7
I. J 1'7
O.079r-02
C .~':~(-O2
".9<';7t-:-O?
J.
3~2
I •• "S
I. :;":0
"J.l C!l~.-':)l
O.112E-Cl
1.!",71
I. (J4
O.IC?f-Ct
I • .,9b
C.<;I'JI'-02
I. 1'3?
1."2.2
I •.':leS
I. "4~
2.0 It
2.<;:73
.2. 1 j(J
2. 1 9"
2.21"2
t;:.7<;!lF-02
O.61c.1;-CZ
O.lC~r.·OI
O.!::~6f.-OZ
0.'\"1';-02
O.42C;r-.:z
O.41~t-lJ2
(,.4?"lF-Ol
O.c.14E-02
0.: Jer--07.
2.J~S
O.,~,r,r;:-02
2. 4<;0
2. '31 ]
G.t. 7"r. -02
(j.o 7 ~'~~-O2
O.7::·!'.[-CZ
z. JP.'-5
2.
~76
•
,
C.7·;~.r-G2
2.(. ]')
0.72"'f:-"'2:
2.7J2
lo: • (.';ll;-02
2.76~
o.~71,e-CZ
~.1\i!7
0."" 17F.-02
~."l;O
~.J7S(-C2
2.9';J
O.?J]G!:-C?
J.!jll!l
0.Zt.4f -02
1.;79
3.1"1
'.137t-C2
".I~~t-C:!
,
-----------------------------------------------------------------:.------:---------------------------
..
•
RAOIANS
O .. IZ!lE 00
0 .. 063
0 .. 12 3F 00
0 .. 117e 00
C.l C 7E 00
0.1 f;lS
,). Z!,;l
, • .J 14
C .<;"f,f-Ol
<;.8C41'.-01
0.377
C .c.eO(-OI
C.4"0
o .~Ct.(-Gl
I
II
0.5(,,5
O.fllS
O.?S~t..:-OI
II
O.lf.:lr:-OI
·O.976E-C2
II
C .~1~F-02
0.3')4=-02
I
O.~17
O.';I\Z
O.:.i91C-OZ
c, .. ~ =f,C-O?
1.0::'3
0.')77 ... -':)2
t .. C '.8
1. t 31
1. 194
1.2'17
C .t!')\.(-OZ
(..1 (4(-01
t.:JIQ
0."51£.-')2
1.30t
C .6151'"-(;7-
0.8')0
O."~~f-'J2
0.1 C!i£;.-Ot
1.44~
C.656£;-C2
I_ 5;;1'1
1.571
I. (; :,.
O.5=:~r-:l~
I . b'l~
1.75<)
o .J"~f.-O?
0.4 I <::E-,)2
1.p.~2
O.~4"E-02
8~S
(".7C9E-t)i!
(,..e7?r,-O?
I.
1.9"8
? .. {) I I
2. e1J
C.:"Hlr--02
O.~~:"ll.-C2
'.hl"IF.-~l
C.llfJl!-Ol
2.1.j6
2.19'l
0.1
2.2(,2
C .<;"'.E.-CZ
2. '25
0.71'0:-02
0 .. t 12,,-"1
C7l~-!"
2 .. J!3'J
C.~I;>f.I;-02
2.
(I::):)
C .JP.5r.-02
2.513
2.516
C .2In-J?
G.A;<.C-03
2.
O.2J:'f.-OJ
f,,J'1
2.7·;'?
2.765
2.
eO:! 7
l.d'90
z. ';5J
J • ., 16
3. (. 7!W
1. l4l
C..,~4'J'!-OJ
o .114E,-02
O.24St::-02
C.oClif-02
tI.'57Gf'-02
,? 71 U'-(l2
c.
C-(lZ
"c:..
o.e:lq~.-o?
\
- .
~
•
I
I
1I
C.J71F-C}
(,. .. '.') I
•
"
--------~--------------------------------------------------------.-----------------------
o. ~.))
O.7!i4
,
QqO[Pl:ATE
0.0
O.t2~
•
,
. •
•
-
-
-
-
-
-
I 1 -
I I I
-
II
I
1
I
••
-
I I -
I I'
-•
--•
•.-- I-•
-I --
....
I'>
...,
II
I
I
I
--
Figure 8.7
Estimated spectral density, M = 6
',,;.
, ( i <.
•
.... 01 ANS
0 ••
0.1)63
0.126
O.I/!IS
0.251
0.314
0.377
.
O.151F. 00
0.14er; 00
G.I~e~ 00
O.122E 00
O.ICJF 00
,).,. ''''r.-':a
OdU7
1).61i')
0.<;42
c.t4~i:.-CI
0.1 J4E-CI
O.11 4 f.-OI
O.S26F.-J':
1..I.2:$4!;-02
".:16':€-03
O.t
). 'jo7l
0.'::'J71.-0Z
c .544E-02
o.eSf,r.;-C2
f.I.113"-01
C;.1:l2l:-01
O.I='I1C-Ol
2.::!~~
2.J~e
?'."'3J
2. ~1 J
2.57':1
2. e.19
2.7~2
2. 7t 5
2.U:l7
2. ti ~O·
2.<'SJ
J.Co 16
..\
.....
•
~
-.~
'.
•
!: .52~f.-:n
c. 1 J~f.-:H
O.114f;-Cl
O.!'1·J2Il-~·2
C.~l71l-(;2
C.)7'JF.-02
I 'nf.-:)2
C.112E::-O?:
C. I ~4r:-C2
o.
C.242F.-02
c .... C4f.-C2
C .~'77l-02
0.71<;1:-02
O.7c;71:;-02
Ii. 71; !!'-:2
O.7I)eE-C2
O.S6!;)E-OZ
O.]IBt-C2
O.2J~r.-CZ
3.":79
C.11'1[-C2
3.142
G.71lf-03
-
•
-
•
•
•
-
•
.. ..
-•-
o.a~2f,;'-02
1. "'<\5
1. !:,; R
1.7r,f)
l.eZ2
I. A~S
1. ')4 0
2.011
2.073
2.116
2.1<;9
2.2(·2
----
o.e7JC-OZ
Uolilf-r.l
C.1JI':-')l
t.:.~a
1. e 3"
1."1-;""
\..,
C.6"~IE-C2
1.0':. 5
1. 1 J 1
1. 19"
1.257
1. J 19
I. 31:.'2
.'
.
.------------------------------------------------------------------.---------------------------------•
G.(,2C.f:-Ql
G."elf-Ol
O.17Cf<:-OI
O.IClF:-OI
c. 7CSf.-0~
0.691
o. 7~4
.~.
lj.e7.(,!;;-~1
0.4\')
o. (,2.,
'.,
· ".
CJ:OINATE
O.!"':l
0.565
.-,',
I-
-.-,-
--
•
I -
-•
-1 ,
, -1
1
I
I
I
1I'
--,-
,II' -
, --
-,-1
I
I -
'"'"
I-
~
Figure 8.8
Estimated 'spectral density, M = 7
~'.
~. ~
..
•
RADIANS
0.0
o.elJ
0.12(,
O.I~9
'J..
•
URDINATE
C.192E I)'
O.I77E CO
C. 1 e2E CO
C.1JC;E 0'
I
0.314
". J 77
'0
o.~..; J
C.56':i
o. t • .-!!'
C.l1lF. 00
(,.flZ.Jr;-OI
o. ','ll
o. 7~"
C.t;;"Q[-02
2~
c."
o.
CI7
o. t:e:;
O. "4?
I • ,.; 5
G.<;SflE:-Ot
'>'J4U':-OI
1I.1 I\""~-Ol
C.IC!W-OI
c. 7C;~'r'-02
o.t:nr.-Ol
0.1 ~l'-L-OI
t . l P.'.(-Ol
(.II.!<:>I:-OI
".l~)F-lH
I.~f;~
".1·:7(-01
1. 1 :! 1
1. 1"4
1.257
1.31 q
t • j~.?
1.445
I . :".;8
I • ~ 71
I . (:)"
C.551(-02
I. b'J(,
I•
7<;~
•• P.?2
I .. CdS
I. ';"'9
2. ::'11
2. [. 7 J
2. 1 J6
2.19')
2. ~C1Z
2 • .375
2.3d!)
0.1 CeF.-02
0.0
0.0
c.o
C.217F.-02
C.tl;'E-02
(..o;e4F-C2
O.12!.C-OI
C.l :H.E-CI
C.l.3 i 'I';-OI
0.1 It l!-Ol
'='.fl<jl:Ii-C2
('wt:
10'£-02
(..4;1f.-~2
C.j"'Of.-C2
0 •. 1"-;1;.-02
o .5,21e-'J2
t.67JF-C2
O.I'lCr.!;;-02
C.es:''':'-GZ
2.450
o. 'IC'Jf?-12
~
IJ
C.t-7~f:-C2
;!. ~ 76
,
I
I
I
I •
I
I
I
.1
I
I
I
,.
I'
I I I
I •
I-
O.t.8.JE-O,J
C.I C7E-02
0.23I::r-02
(.424F-02
O.7~.::!r-~(;?
-
I
2.7(5
c..1t ".('-02
.
I
I
I
J.:'r.~.t:-02
3.142
-
•
•
I
I •
O.I~JC-02
O.614t-OZ
•
I
2.7)~
2.eZ7
-•
•
, • -•
, -•
.-• ••
-, --•
,-,- --
C.I\P','F"-OZ
2. E';'J
2. "'53
J. u 16
3. C 7?
\-.
I
I
I
I
J'J
2. '.
'.'
--------~-------------------------~-----------------------------.---;-------.
----------------------.
I
I •
I-
2.
,
.".'
-
--,-
.....
N
V<
I II •-
Figure 8.9
Estimated spectral density, M = 8
t~·'";
. ~" ..... -~'.
"'~'
,
;
,"t:'
•
RADI "',..5
O.J
a.Of.J
a.IZ6
o.
I
e~
O. i? S 1
0.3111
0.377
C.IIII'
O.51.1J
0.510)5
0.62d
o. ~?
O.7'i1l
a. el7
o.
ft:tl
O.tal
1. O~"i
t.Ofl6
I. l:i 1
1. lli4
1.;>57
I.,J 19
1. J II?
-
"
.-
,,' 0
"
0.0
o.J
•
•
•
•
o.r.
O.5fa:-03
o.Je'Jf:-oz
2. 1~'j
2.38f1
2.4';0
2. !.1 J
2.576
z. Co'Jr,
2.7':jl
2.765
2. eZ7
2. "l;90
1. COOS.J
•
.'-
------------'------------------------------------------------------.----------------------------------
0.7"':[-OZ
0.103E-Ol
C.119r,-OI
0.12::F.-01
2.262
-.'
l'
O. I Q2E 00
0.1 eo'JE CO
n.1 !.liF. 00
C • J ., 3F 1)0
(;.11 ~!: CO
f,;'.e I!:I(-C'I
C.f :!".!;-Ol
lJ.J 16r:-Ol
O.170F.-1)1
O.'i14t;-02
O. E:4'H'-I)i?
C.IIIE-Ol
O.I'SI(-Cl
C.ll·![-·,I
C.IC;IIE.-vl
0.119('-1)1
C.14C.e-.-OI
0.t<'ia-02
O.3'3::if;-02
1. '5,J lJ
!.571
I. t; J4
1. 6~t
1.7'59
1. '122
1.8l'!S
1. ~lId
." .".'
... ~
'." ,"
,
•
CJ:OIPIIATE
1.1145
2 • .:>1 I
2 • .;) 7 J
2. 1.3 '!I
2. 19/1
:'0::
O.113f~-OI
o.st' :!F':-02
r..777(-02
C.t;Z3F-'::2
c. .!;JII!'-GZ
C.511(:-02
O.St..,jE':'02
c.. f .'!t' r-Ilz
O.7C4r;-02
O.7~':I::-CZ
~
•
I
I
I'
I
I
I
I
I •
I •
I'
I'
II •
I •
I'
C.II11l:.-C.2
I'
O.3''''~-O?
I'
O.J4~E-OZ
C.449r-Q2
-
•
•
•
,.
c..~
0.211,;e-02
O.2laE-C2
0.ZlleE-02
-
I •
I •
I I
I •
)<o:::-cz
-
.•
-
-••
C.'.42€-C?
C.71~~-02
•
•
-
- -
-•
•
•
•
•
•
I'
I'
I'
3.01 b
).~lo~r..-CZ
J. C7<'
3.142
C.',l~·I.-O~
I •
o .63"E-')2
I •
.....
N
C1'
- - _ _ _. __ 0
Figure 8.10
Estimated spectral density, M c
~
....
•
n ..." ... 1\05
Lf;.)
I"'. 'F.
O.C!
{'.I JJf, -nl
O.:"l~J
O.IJ::r-I''1
".Ilh
':.ltH
" .. ~ J I
('I.I.:L1~-"'1
0.) I"
0.377
o. ""1
C.IJJf-CI
t:.'.IJJr.-JI
0.1 JJf -1)1
"."i6~
G.IJJl"'-Cl
'" .I.'J(--Cl
C.IJJI'-Cl
\) ... '.1
0.754
O. 'II 7
o.
A~lJ
0.9'':I. ('lC~'
1 .. ~\l' oi
1.131
I. I'."
I • .:~ 7
I. J ,,,
1. J fl.1
I ••US
1. "')11
1.571
1. f· J"
I .. (. ~t'
I. 7'iQ
l.fL!Z
1 .. -'t'"
I .. '._ '.!
2.\~11
2.,:'D
? ..
Z. I CO)
~. ;:"';':
'Z. J,"!o;
2 •.~:!~
7. ... ::;0
~. ~ t J
2.516
:::. ~ J'J
? 7 ~·2
2. 1{' '5
;'!. I
1) ..
·').IJ~~-Ol
\.l.1 :;:JF-C' I
".133':-01
li .. I 3:U"-('I1
C.IJ.!~-~I
O.l::!::!(·-OI
C.l JJt.-OI
.~3f
-01
C.IJJr-OI
C.I.!Jf-CI
O.IJ.1f-Ol
" .. I J~'::-': I
1,;..1:':'"-('1
O-l.l:!r-Cl
·••
('.lJJI-~1
('.133£.-01
..... I JJf.-C.1
O.IJJf-c..1
.:,. .. I.!Jt.-~1
~·.1 JJf:-C I
J.I ::'~l -C'l
C.I.!:,r-CI
Il. 1 J )':- (II
(.13.'[-01
~ .1 :!Jf.-Ol
I :IJ!.-t:lt
i).lJ3f.-Ot
o.
-"
-J
C.l~~I,-CI
3.('1100
J. '::1~
l'. t J~!-Ol
C.IJJf"'Ol
J.t".?
".lJ~~-Ot
1
·
133<'"-01
l'.1
.
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
••
O.IJJF-CI
i:.
','"''
l.~~D
:-t~
'-
C.l::'Jr:-CI
O.I:J;H--Ot
('.1 13£·-JI
('. I JJf -,)1
n.l:!.!r-\:I
".I.!.JI"-OI
2.
•
C.1J::!E-(l1
I).I))(-Cl
!.~'l
,
,
------------------------------------------------------------.-----.---~----------------------------,
•
O.l~)F-OI
O.~~J
J ..
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
••
•
•
Figure 8.11
Estimated spectraf density. M"c 10
...
N
.....
•
•
AAO( • PolS
0.'
a.CEo)
C: ... 12~
0.16'\
o. ~51
0.314
o. J17
C·,
<:!q- CO
C .1ZOr. 00
C ... I 1 C;f. 00
C ... J ,r;'" CO
C ."1 Jf·-"t
"•• 1 Z7~
~.I
I
I
C.',f!7£-Ol
I
C .401llt -01
I
0.;1..,£-::11
C" ... Sf. OJ
O. 6l~
C.(·~6~-C2
C.'='
o. ("i1
Co.I"~F-C2
0.7<;"
'J. PI 7
C. ttctO
C ... ~lE-C2
O.':I4~
G.7,)7lZ-0t
1.0':0;
:.:..",e~.f-~l
1.
v.117.f-(1I
Il
I ... 1 ""
".I~I[-02
t.O
I. JI9
I. JCZ
1.445
I . ~-:. e
C.":
<..:-.6Cr-CJ
~'ZZ
C.7e:'F-O?
(,.IZJiL-OI
f:oIS-:f.-CI
('.1 )'1f -01
C.H.:lf -~1
(.14;;:>(,-02
".5 I !'",r.-cz
I. 'l.eS
(..IIl·.... E -(,2
1. (.rtf,
I ... 7-:'''
1.
1. QAB
c..~7t.E.-OZ
2' •
2.J1J
C .. 746£ -~?
C .1;I',c,;r·-~'2
2. 1:6
C ...
2. 10;9
(;.7<;7(--0,)2
2.2':2'
o.t.<;If-C2
l • ~ l',;!:-\J?
G.45Ct-G2
C.4SIf.-(2
.J
II
2. :J25
~. ':He
2.450
.,~7f-
11
c.4~l"r-02
G.~":t'r-oz
Z.-e. 39
O.477l-0?
, .3'O!l-CZ
2.
~
7;'~
2.U.~
•
•
O.2.C:~f-C2
;'."7)
.3. 1 It:!
-02
•
•
••
•
•
•
•
·••
I
I
I
•
•
•
,. •
<. •
.Jf)f.(
•
•
.!. .... ~.,
to.!; 1'j"-(.2
C.t4l;;r-c?
o .(:~'H -1)4:
•
•
•,
I'
I"
•
•
(·.~I'!{t--cz
I"
•
•
------------------
•
•
•
•
z.'P. fi']
J.
•
•
•
-•- -
,•
•
I·
n~7
•
•
O.2Jd£-C,2
2.
•
•
•
-02
?.511!1
2.
I
/). c..'l!.-C 1
(j . ,
I ... ( J4
I
r:.le~[-'-Cl
I,. 2-:: 7
t.S71
~
I
:---------\~---------------------------------------------------
C.441,)
'f:
•
Go:10rr.ATE
l). S~J
1. I J I
•
I"
I
I
.....
N
'"
•
•
Figure 8.12
Estimated spectral density, M = 11
.~
129
9.
ANALYSIS WITH UNBAIANCED DATA
The linear models presented in Chapter 3 aSSume that each of the
A cross-sectional units has the same number
T
of observations.
The
estimation procedures developed in Chapters 4 and 6 presuppose this
balance of the data.
However, in real situations it may happen that
different cross-sectional units are observed at different number of
time points.
For such situations a method analogous to the well-
known "missing plot" technique of experimental design analysis can be
used to create an appropriate set of balanced- data:.to which the
estimation procedures developed above can be applied.
If the observation for the
•
,lli
~--
cross-sectional unit at t h e
and
time point is missing, we define
p.
x
itk
= 0 ,
k.
IClI
1,
... ,
For each missing observation we add a dummy variable which aSSumeS
value one for the corresponding missing observation and has value
zero otherwise.
So, if there are
m dummy variables are defined.
m
missing observations, a total of
In this way a set of:.linear regression
equations is defined for a "complete" set of
of
A
T observations on each
cross-sectional units, with zeros substituting the missing
values for the dependent variable and the corresponding values of the
independent variables, and
relationship (1.1).
m columns added to the
X
matrix of the
The estimation procedures of Chapters 4 and 5 can
now be applied to these regression equations under the specifications
of Model A and B, respectively.
-.
The first
p
elements of the vector
of estimates of the regression coefficients, corresponding to·the
explanatory variables of the relationship (1.1), are equivalent to the
••
130
estimates which are provided by the GIS technique applied to the
original set of unbalanced data.
•
This result for Model A is
established by t.he theorems belm',
Completely analogous procedure
would prove the result for Model B.
Suppose
m
observations are missing in a set of cross-sectional
ti.me series data with
A
cross-sect.ional uni.ts and
T
time points.
Let
denote the re1a tionship (1. 1) omit ting the equations corresponding to
the missing observatiuns.
•
Let
(9.1)
denote the regression equations for the set of observations balanced
by the inclusion of the ":zero observations!!; hence
:1..*0 =
~
where
o
=
[~J
[:]
X
X* =
o
0
:.]
=
l~J
,
represents a naIL ma trix of appropri.ate order and
such that
-.
ik
= 0
and
Var
~J
u
'-m
is
••
131
where
V*
is the same as the matrix
V defined in (3.6) with the
elements rearranged to conform to the order of the observations in
•
(9.l).
We note that the specification of the last
m equations in
(9.l) is artificially constructed to conform to the covariance
structure of Model A.
Theorem 9.l
and
with
be the nonsingular
elements respectively
(M!-3) x (M!-3)
tr(NOViONOVjO)
SJ,
be the
iONoViONoXo
and
•
X*
= I AT
-
are as defined above,
X*(X~X*)-~~
ViO
NO = I
= 1,
and
(M!-3) x 1
and
V
ll
of
vectors
~~N*Vi*N~, where
Xo ( XoIX0 )-1X I '
O
is the submatrix of
(3.9), corresponding to the submatrix
i
_
AT m
matrices
Vi' defined in
V*, and
V *'
i
••• , M+3 , are the matrices defined in (3.9) with the elements
rearranged according to the order of the observations in (9.1).
the solutions
B;t.
= £"
and
of the sets of equations
BJo
=
Eo
Then,
and
are identical.
Proof:
The proof is straightforward by checking that
= tr(N*Vi*N*V j *)
M!-3 •.
and
~NOViON~ = )''';'N*Vi*N~,
tr(NOViONOVj0)
i,j
= l,
... ,
0
Theorem 9 ._~
•
..,.
With the notation defined above, the estimator
(9.2)
••
132
is equivalent to the solution ~
of the generalized normal equations
•
(9.3)
!,.roof:
The proof uses the well-known result of matrix algebra
(~.£.,
Rao (1973), page 33) that the inverse of the partitioned matrix
in terms of. its submatrices
•
n , Vl2 ,
V
V
21
and
V
22
V*
is
where
with
Substituting
-.
in termS of its partitioned form, we obtain from
(9.3) the two sets of equations
••
133
•
By premultiplying the second equation by
-1
XbVllV12
and then adding
to the first equation, we obtain
•
whose solution
~
is identical to (9.2).
0
It is immediately obvious that the properties of the estimators
of regression coefficients established in Chapters 5 and 6 hold for
the regression equations (9.1) with the specification for the last
equations artificially.'constructed to conform to the covariance
structures of Model A and Model B, respectively.
~.
m
••
134
SUMMARY AND CONCLUSIONS
10.
The estimation of linear relationship'involving economic
•
variables from cross-sectional time series data has been the subject
of many research papers that have appeared in the econometric
'literature.
The diversity of approaches proposed in the literature
stems from both the different sets of assumptions adopted for the
error term of the linear relationship and the different estimation
procedures adopted.
Most of the alternative approaches are based on
oversimplifications of the more realistic assumptions used here.
The
variance component approaches ignore the possibility of serial
correlation in the time direction.
•
The seemingly unrelated
regressions approaches, we feel, needlessly aSSume heteroscedasticity
with respect to cross-sectional units and treat cross-sectional unit
effects as fixed rather than random.
In this study we adopted a set of assumptions thought more
realistic to situations of pooling of croSs section and time series
data.
Two models are proposed to fit alternative situations.
Model A
aSSumes that the: linear relationship to be estimated is affected by a
random disturbance
u
it
with three random components 3 namely
u1." t = a"l. + b t + e"1.t , where
b
e
t
it
a.1.
is a time invariant individual effect,
is an individual invariant time effect and the third component,
' varies with both individual and time.
For the third component,
we take into account the possibility of serial correlation in the
time direction.
•
Model B specifies the same set of assumptions except
that the time invariant individual effects are considered fixed •
••
135
The serial correlation among the disturbances for a Same individual
is taken into aCCO'.lUt by the specification that the residual term
•
e
it
originates from a sample of a realization of a finite moving average
process.
This specification is effectively equivalent to using a
truncated estimator of the spectral density of a general stationary
process ..
The covariance matrix of the vector of observations has a
complicated structure which makes the study of the asymptotic
properties of the two step generalized least squares estimator with an
estimator of the covariance matrix intractable.
This problem has
been overcome by substituting an approximating matrix, following
•
suggestion by Hannan (1963),
This approximating matrix uses a
circular symmetric matrix in place of the covariance matrix of the
stationary moving average process.
It has been shown by Hannan (1963)
that in regression problems with stationary errors, under mild
conditions, the generalized least squares estimator with the
approximating covariance matrix is an asymptotically efficient
estimator for the regression coefficients ..
,men the investigator's
~.priori
Hannan's method is useful
knowledge about the stochastic process
of the residual is minimal.
The proposed estimator for the regression coefficients is
essentially a Hannan's type estimator with the variance and covariance
components substituted by the corresponding estimators obtained by a
method suggested by Seely (1970b).
•
It has been shown that this two
step generalized least squares estimator is unbiased and has the Same
asymptotic multivariate normalclistribution as Hannan's estimator.
,.
136
As usual with asymptoti.c pr"'jpertiec, they wi.!l hold approximate.ly
for a sample provided the sample si.ze is "large en·:>Ugh".
,
It should be
stressed chac the proposed estimator is a double approximat i.on.
More-
over, as Hannan's method requireE the estimation of the spectrum, it
is unreliable when the sample size
series) is not sufficiently large.
(i.~.,
the length
T
of the time
Also, .it is unnecessary i f the
stochastic process of the. residual can be mo're exactly speci.fied to
be, say, a first order cr a second o·rder aut.oregressive system, in
which case it i.s desirable to use such knowledge in estimation.
H0W
large should the time series be depends on features of the spectral
density of the proc.ess.
•
The spectral. density of an uncorrelated
sequence can be estimated by a relati.vely short series, while a
spectral density with many peaks will. require rather large val.ue of
for the asymptotic theory to be a good approximation.
This question
can be answered only by Monte Carlo simulation studies for specific
situations.
It is hoped, however, that the method should prove more
appropri.ate for many situati.ons of pooling of crOSE section and time
series data than t.he simplistic ones which ignore the time series
aspect of the data.
T
137
11.
LIST OF REFERENCES
Amemiya, T. (1971). The estimation of the variances in a variancecomponents model. International Economic Review, 12:1-13.
Amemiya, T. and W. A. Fuller.
(1967).
A comparative study of
alternative estimators in a distributed lag model.
Econometrica,
35:509-529.
Anderson, T. W. (1969). Statistical inference for covariance matrices
with linear structure • .~~ P. R. Krishnaiah (Editor), Multivariate Analysis - II, 55-66. Academic Press, Inc., New York,
New York.
Anderson, T. W.
(1970).
Estimation of covariance matrices which are
linear combinations of whose inverses are linear combinations of
given matrices. In R. C. Bose, I.M. Chakravarti, P. C.
Maha1anobis, C.R. Rao, and K.J.C. Smith (Editors), Essays in
Probability and Statistics, 1-24. The University of North
Carolina Press, Chapel Hill, N.C.
•
Anderson, T. W. (1971). The Statistical Analysis of Time Series.
John Wiley and Sons, Inc., New Yurk, New York •
Anderson, T. W.
(1973).
Asymptotically efficient estimation of co-
variance matrices with linear structure.
The Annals of Statistics,
1: 135-141,
Balestra, P. and M. Nerlove. (1966). Pooling cross section and time
series data in the estimation of a dynamic model: The demand for
natural gas. Econometrica, 34:585-612.
Bellman, R. (1960). Introduction to Matrix Analysis.
Book Company, New York, New York.
Blair, R. D. and J. Kraft.
(1974),
McGraw-Hill
Estimation of elasticity of
substitution in American manufacturing industry from pooled cross
section and time series observations. The Review of Economics
and Statistics, 56:343-347.
Brehm, C. T. and T. R. Saving.
assistance payments.
(1964).
The demand for general
The American Economic Review,
Cochrane, D. and G. H. Orcutt.
(1949).
54:1002-1018~
Application of least squares
regression to relationships containing autccorrelated error terms.
Journal of the American Statistical Association, 44:32-61.
•
Cramer, H. (1946). Mathematical Methods cf Statistics.
University Press, Princeton, New Jersey •
Princeton
138
DeBord, D. V. (1974). Demand for and cost of salt marsh mosquito
abatement. Ph.D. Dissertation. Department of Economics,
North Carolina State University, Raleigh, N.C. University
Microfilms, Ann Arbor, Michigan.
v
Dhrymes, P. J. (1971). Distributed Lags: Problems of Estimation and
Formulation. Holden-Day, Inc., San Francisco, California.
Duncan, D. B. and R. H. Jones. (1966). Multiple regression with
stat·ionaryerrors. Journal of The American Statistical
Association, 61:917-928.
Engle, R. F.
(1974).
Band spectrum analysis.
Inte rna tiona 1
Economic Review, 15:1-11.
Fishman, G. S. (1969). Spectral Methods in Econometrics.
University Press, Cambridge, Massachusetts.
Fuller, W. A.
(1971).
University
•
Harvard
Introduction to Statistical Time Series.
Bookstore~
Iowa State University, Ames, Iowa.
Fuller, W. A. and G. E. Battese. (1974). Estimation of linear models
with crossed-error structure. Journal of Econometrics, 2:67-78 •
Graybill, F. A. (1969). Introduction to Matrices With Applications
in Statistics. Wadsworth Publishing Company, Inc., Belmont,
California.
Graybill, F. A. and R. A. Hultquist. (1961). Theorems concerning
Eisenhart's Model II. The Annals of Mathematical Statistics, 32:
261-269.
Grenander, U. and G. Szego. (1958). Toeplitz Forms And Their
Applications. University of California Press, Berkeley, California.
Hamon, B. V. and E. J. Hannan. (1963). Estimating relations between
time series. Journal of Geophysical Research, 68:6033-6041.
Hannan, E. J. (1960). Time Series Analysis.
Ltd., London, ED~land.
Hannan, E. J.
(1963).
Methuen and Company,
Regression For Time Series •
.lI).M. Rosenblatt
(Editor), Proceedings of the Symposium on Time Series Analysis,
17-37.
John Wiley and Sons, Inc., New York City, New York.
Hannan, E. J. (1965).
distributed lags.
•
The estimation of relationships involving
Econometrica, 33:206-224.
Hannan, E. J. (1967). The estimation of lagged regression relation.
Biometrika, 54:409-418 •
,.
139
Henderson, C. R.
components.
(1953). Estimation of variance and covariance
Biometrics, 9:226-252.
Henderson, C. R.
(1971). Comment on "The Use of Error Components
Models in Combining Cross Section with Time Series Datal!.
Econometrica, 39:397-401.
Hildreth, C.
(1949). Preliminary considerations regarding time
series and/or cross section studies. Cowles Commission
Discussion Paper, Statistics No. 333. University of Chicago,
Chicago, Illinois (Unpublished).
Hildreth, C. (1950). Combining cross section data and time series
data. Cowles Commiss ion Discuss ion Paper, No. 347. University
of Chicago, Chicago, Illinois (Unpublished).
Hoch, 1. (1962). Estimation of production function parameters
combining time series and croSS section data.
Econometrica,
30:34-53.
•
Hsu, P. L.
(1938). On the best unbiassed quadratic estimate
of the
variance. In J. Neyman and E. S. Person (Editors), Statistical
Research Me;;oirs, 2:91-104. University of London, London, England •
Hussain, A. (1969).
327-336.
A mixed model for regressions.
Biometrika, 56:
Johnson, P. R.
(1960). Land substitutes and changes in corn yields.
Journal of Farm Economics, 42:294-306.
(1964). Some aspects of estimating statistical cost
Johnson, P. R.
functions. Journal of Farm Economics, 46:179-187.
Kakwani, N. C.
(1967). The unbiasedness of Zellner's seemingly
unrelated regression equations esti.mators.. Journal of the American
Statistical Association, 62: 141-142.•
Kmenta, J. (1971). Elements of Econometrics.
New York City, New York.
The Macmillan Company,
Kmenta, J. and R. F. Gilbert.
(1968). Small sample properties of
alternative estimators of seemingly unrelated regressions.
Journal of the American Statistical Association, 63:1180-1200.
Kmenta, J. 2.nd R. F. Gilbert. (1970). Estimation of seemingly unrelated regressions with autoregressive disturbances. Journal of
the American Statistical Association, 65:l86-197.
•
••
MO
Kuh, E.
(1959).
The validity of cross-sectionally estimated be-
havior equations in time series applications.
Econometrica,
27:197-214.
Maddala, G. S.
(1971).
The use of variance components models in
pooling cross section and time series data.
Econometrica, 39:
341-358.
Mann, H. B. and A. Waldo (1943). On stochastic limit and order
relationships. Annals of Mathematical Statistics, 14:217-226.
Marschak, J. (1939). On combining market and budget data in demand
studies: a suggestion. Econometrica, 7:332-335.
Marschak, J. (1943). Money illusion and demand analysis.
of Economic Statistics, 25:40-48.
The Review
Mitra, S. K. and C. R. Rao. (1968). Some results in estimation and
tests of linear hypothesis under the Gauss-Markoff model.
Sankhya, Series A, 30:281-290.
•
•
Mundlak, Y. (1961). Empirical production function free of management
bias • Journal of Farm Economics, 43:44-56.
Mundlak, Y.
(1963).
Estimation of production and behavioral
functions from a combination of cross-section and time-series
data,
In C. F. Christ et al. (Editors), Measurement In
Stanford' University ?ress, Stanford,
California.
Economi~, 138-166.
Nerlove, M.
(1967). Experimental evidence on the estimation of
dynamic economic relations from a time series of cross sections.
Economic Studies Quarterly, 18:42-74.
Nerlove, M.
(197la).
Further evidence on the estimation of dynamic
economic relations from a time series of cross sections.
Econometrica, 39:359-382.
Nerlove, M. (197lb).
39:383-396.
A note on error components models.
Parks, R. W.
Efficient estimation-of a system of regression
(1967).
Econometrica,
equations when disturbances are both serially and
contemporaneously correlated.
Association, 62:500-509.
•
Journal of the American Statistical
Rao, C. R. (1967). Least squares theory using an estimated dispersion
matrix and its application to measurement of signals. In L. M.
1£ Cam and J. Neyman (Editors), Proceedings of the Fifth Berkeley
Symposium on Mathematical Statistics and Probability, 1:355-372 •
University of California Press, Berkeley, California.
.e
141
Rao, C. R. (1970).
linear models.
65:161-172.
Estimation of heteroscedastic variances in
Journal of The American S tati.stical Association,
Rao, C. R. (1971a). Estimation of variance and covariance
components - MINQUE theory. Journal of Multivariate Analysis,
1:257-275.
Rao, C. R.
(1971b).
Minimum variance quadratic unbiased
estimation of variance components.
Journal of Multivariate
Analysis, 1:445-456.
Rao, C. R. (1972). Estimation of variance and covariance components
in linear models. Journal of the American Statistical
Association, 67:112-115.
Rao, C. R. (1973). Linear Statistical Inference and Its Applications.
Second Edition. John Wiley and Sons, Inc., New York City,
New York.
•
Schultz, H. (1938). The Theory and Measurement of Demand.
University of Chigago Press, Chicago, Illinois •
Searle, S. R. (1971a). Linear Models.
New York City, New York.
The
John Wiley and Sons, Inc.
Searle, S. R. (1971b). Topics in variance component estimation.
Biometrics, 27:1-76.
Seely, J. (1969). Estimation in finite-dimensional vector spaces
with application to the mixed linear model. Ph.D. Dissertation.
Department of StatistiCS, Iowa State University, Ames, Iowa.
University Microfilms, Ann Arbor, Michigan.
Seely, J. (1970a). Linear spaces and unbiased estimation.
Annals of Mathematical Statistics, 41:1725-1734.
The
Seely, J. (1970b). Linear spaces and unbiased estimation application to the mixed linear model. The Annals of
Mathematical Statistics, 41:1735-1748.
Seely, J. and S. Soong.
estimability.
(1971).
A note on MINQUE's and quadratic
Mimeographed report.
Oregon State University:
Corvallis, Oregon.
Seely, J. and G. Zyskind.
(1971).
variance unbiased estimation.
•
Statistics, 42:691-703 •
Linear spaces and
rn~n~mum
The Annals of Mathematical
••
142
Sims, C. A. (1971). Are there exogeneous variables in short-run
production relati.ons? Center of Economic Rese.arch, University
of Minnesota, Mi.nneapo1is, Minnesota. Discussion Paper No. 10.
Stone, R. (1954). The Measurement of Consumers' Expenditure and
Behaviour in the United Kingdom, 1920-1938, VoL 1. University
Press, Cambridge, England.
Swamy, P. A. V. B. and S. S. Arora. (1972). The exact finite
sample properties of the estimators of coefficients in the error
components regression models. Econometrica, 40:261-275.
Theil, H. (1971). Principles of Econometrics.
Inc., New York City, New York.
John Wiley and Sons,
Tobin, J. (1950). A statistical demand function for food in the
U.S.A. Journal of the Royal Statistical Society, Series A,
113: 113-141.
•
Wallace, T. D. and A. Hussain. (1969). The use of error components
models in combining cross section with time series data.
Econometrica, 37:55-72 •
Wold, H. and L. Jureen. (1953). Demand Analysis - A Study in
Econometrics. John Wiley and Sons, Inc., New York City, New York.
Zellner, A. (1962). An efficient method of estimating seemingly
unrelated regressions and tests for aggregation bias. Journal of
the American Statistical Association, 57:348-368.
Zellner, A. (1963). Estimators of seemingly unrelated regressions:
Some exact finite sample results. Journal of the American
S tatis tical Associa tion, 58:977-992.
Zellner, A. and D. S. Huang.
(1962). Further properties of efficient
es rima tors for seemingly unrela ted regress ion equations.
International Economic Review, 3 :300-313 •
•
1.44
.•
an~~~~~~
Some Concepts
12.1
In this section we review briefly
conce.pts and resul ts en
Some
stochastic convergence useful to the asymptotic theory developed in
the main text.
textbooks;
Let
Most proofs are omitted since they can be found in
~ •.&.,
X
n
,
Fuller (1971).
n = 1, 2, ••• ,
probability space
F()
•
(O,G,P)
and
and let
seque~ce
for every
€
> 0
lim
n->=
p
=-+
n
•
(X:1-i
of random variables
probability to the random variable
n
n = L, 2,
F ( )
J
and
be the corresponding distribution functions.
The
X
be random vaelab les en a
X
as
X
P[!X n - Xl > €]
=
is 6a id to con·ve rge in
n
0
tends to infinity i.f
Thts i2 denoted by
X •
Definition 12.2
(X n 1
The sequence of random variables
is said to converge in
distribution (or in law) to the random vari.able
infinity if the distri.bution functio:l
the disrrl.bution function
F( ) •
We write
X
n
F()
of
X
F ( )
n
X
of
n
tends to
C(;!1ve-rges to
at all cont.inuit.y point.s of
~ X •
Concepts of order of magnitude are very u.seful
the limiting behavior of random variables.
••
aE
i.n i.nvest.igating
We ferse defLne t.he
cepts of order of magnitude of sequences of real numbers •
con~
••
145
(a}
n
Let
tr n }
be a sequence of real numbers and
a sequence of
poaitive real numbers.
pefinition 12,1
We say that
a
= OCrn )
n
\a Ilr
n
n
$
a
n
is of orde rat moS t
r
, denoted by
n
, if there exists a positive real number
M for all
n
M such that
greater than some positive integer
N ,
De finiticn 12,4
a
We say that
a
•
= o(r)
n
n
if
n
is of smaller order than
lim a Ir
n->=
n
n
r
n
and write
= 0 ,
The following properties are well-known in Real Analysis,
They
are easily established by using the definition and the properties of
limits of sequences of real numbers.
Theorem 12.1
Let
(s}
n
(a n 1 and
be sequences of real numbers and
sequences of positive real numbers.
(i)
( ii)
n = OCr n )
and
a b = OCr s )
n n
n n
a
= blr )
and
oCr n s n )
a
= OCr )
and
If
IE
a
a
n
a b
n n
(iii.)
IE
a
n
' n
n
a b = o(r s )
n n
n n
••
b
n
= O(s )
n
, then
b = O(max(r ,s ))
n + n
n n
b
n
n
= o(s )
n
+ b !l
b
n
, then
= 6 (max(r ,s })
n
=O(s)
n
,
then
n
tr n}
and
146
The concepts of order of magnLtude for sequences of random
variables are closely related to convergence in probability.
The
definitions which follows were introduced by Mann and Wald (1943).
Let
[Xnl
(r }
n
be.a sequence of random variables and
a
sequence of positi.ve real numbers.
-,Definition
12.5
-~
We say that
X
n
is of probability order
iE for every
M
number
€
N
and an integer
e
€
such that
p[IX I~ r M ] ~
n
n
€
€
€
We say that
xn
write
there exists a positive real
n > N
for all
•
> 0
r n ' denoted by
= 0
P
X
n
is of smaller order in probability than
r -1X
(r )
n
n
~ 0
--..
n
as
r
n
and
n -+co
Mann and Wald (1943) proved that the algebra of order relationship established by Theorem 12.1 holds also for order in probability.
Because of the close connection between the concepts of order in
probability and convergence in probability, the following well-known
i.nequality is very useful in investi.gating probabi.lity order of
sequences of" random variables.
Theorem 12 •.:£
If
€
••
> 0
X
(Chebyschev's i.nequality)
is a random variable with finite variance, then for every
and finite
~
p[IX - ~I~ €] ~
E X_.)2
-4f
€
••
147
The foilowing result is a consequence of Chebyschev's inequality:
_._--_...
Co ro llarv 12.1
--~---
i.s a sequence of random variables satisfying
If
2
r(X ) '"
, then
xn
By assumpti.on, for
= OpCr ) •
n
n > N , there is an
Now, by Chebyschev's inequali.ty, for
M.z>
M
l
such that
0 ,
E(X2.)
•
p[IX I~ ~2r ] S
n
n
2.
M
r
~
2 n
Then, given
£
>
a
by choosing
for all
n
> N •
o
The Definition 12.1 applies also to vector random variables
where it is understood that
jXn
-
Xl
is the Euclidean distance.
The
following theorem e:.tablishes a useful result.
Let
X
-n
n ~ 1, 2,
variables (k fixed).
if
.•
Then,
~,
and
X
be k-dimensional random
j
= 1,
.•. , k , if and only
p
X
-"'11
-+~.
Then, the concepLs and properties of oeder relationship can be
••
extended to vector-valued random variables as follows •
••
148
(X}
-n
Let
(['}
n
be a seq~ence of k-dLmensLonal random variables and
a sequence: of poaitiiie '.ceal numbe'rs.
We say that
x
·~n
~
01'(" ) , if every element of the vector
.. n
probability
X
~~
~
r
X'
n
Jn
;
r
X
n
= 0p(r )
ev~ry
if
n
r
element: of
!...
n
,deno ted by
is of order in
-n
j ;
(L,(" )
r
1,
is of smaller order in probability than
probability than
•
is of order in probability
Y..
-n
...
rn
k •
~
and we write
is of smaller order in
n
We next state two important well-known results which are used in
the ma in tex t.
rneo rem 12 '-~
If
(x
}
-n
and
variable.;,; 3ueh that
L"-~.11.:.'?
If
:L~g;.n
~.
(y }
arc sequences of k-d imens ional random
-n
f.)
-.
X
'-n
X
and
-y
X
-n
42
P
-+ 0
,
then
Y
-n
~x
(Slutsky's theorem; Cramer (1946), page 255)
{zln}' .... :- (zkn}
are seqaences of random variables converg-
probability to tr.e constants
cl' ... , c
any ratio2al function
R(2. , ••• , Zk )
thE:. constant
:l
R(c ,
1
.•.
n
J.r..
c )
k
k
' respectively, then
converges in probability to
, pro\tided that the later is finite.
(z 1 be a sequence of k.. dimens:i.onal random variables and
-n
let:
be a sequence of
k x k
no~singular
matrices of constant
••
149
elements such that
p
_c
Z
A
and
...J.
constants and
A
n
. . . . __.
Res!.ll ts .:;£
Matri~-r.
)2.2.1
(1)
.
is a vector of
Then,
is a non2.i.ngu1.ar matrix.
==c:...::=~
c
A , whe:.re
-t
T2eorv
~_
Inequalities
If
x
J. are vectc,rs in the n-dimensional
and
n
R. ,
Euclidean space,
then
(12.1)
This is the so-called Ga 2.chy's inequality.
1
•
(2)
Let
A
x
be an
n
any vector in
R
n
t'eal symmetric matrix and
n
.
x
T:.1en,
x 1P....x
"S (A) .:> _ . - :>
x'x
where
"S(A)
(12.2)
and
"L(A)
are, respectively, the
smallest and the largest characteristic roots of
A.
Moreover,
xtAx
"S (A) =
min
~€
(3)
If
x
Rn
---j-'"
x x.
x'Ax
and
"I,(A) =
~~
is a vector in
max
n
~8
n
R
and
P
R
x'x
is an idempotent
(12.3)
••
~
~1.50
•
Qef.ini.tion 12",2.
A
L€ t
m x n
an
2
and
2
be an
a!ld
B
The di:::ect prGdllct (or Kronecker product) of
A
macT'ix with
ml. x
matrix.
(in chat order), denoted by
B
~_ @ B
a ..
'.J
,
.i.2 the
m m. x n n
l 2
l 2
par ti t iO!led rna t r ix
•
The next result follows inunediately f-rom the above definiticn.
2orollary 12.3
Let
y
....
,
...... ,
01
be a
C
T)
T x T
Then, the
b .. c
, where
l.J ts
ations
matri.x with
A x A
be an
and let
A)
(t,s = l,
i.3
B
= (i-l)T
ex
+ t
and
and
i3
.. tn
1.J--
rna tei>: wit.h
element
th
ts--
b ..
l.J
element
13 t.n
01 - - element of the matrix
(1., j
c
;
1,
ts
Bllile
have the -reGpec t. i.v€. unique represent-
13
~
(j-l)
+ s •
The following cO!lsequences of Corollary 1..2.3 t.o part.icular forms
of t.he matrices
Band
C
will be useful i.n
computational procedure i!l Chapter 7.
t~e
di£cu.sc:i.on of the
,-
151
Corollary 12.4
Let
~
0' = T(i-1)
where
S
columns
th
or--
denote the
v
-1(0' )
+
Then, the row vector
t
V1 = I A (iJ J T
row of the matrix
v
-1(0')
has a
1.
,
in
for which
S = T(i-1) + s ,
s = l , ... ,T
the remaining elements are 0 •
Corollary 12.5
Let
0' = T(i-1) + t
•
th
be the
v
-2(0')
.
cr-
V2 = J A
row of the matrix
Then, the row vector
has a
v
-2«}' )
1
@9
IT
,
where
in columns
S
for which
S = T(j-l) + t ,
the remaining elements of
j = l , ... ,A
are
v
--2 (0')
o•
Corollary 12.6
Le t
~3 (0')
0' = T(i-1) + t •
th
or--
be the
row of the ma trix
Then, the row vector
V = I
A
3
has a
v
-3 «}')
1.
@ IT
' where
in co l.umn
where
S = T(i-l) + t
are
the remaining elements of
•
.9orollary 12.7
Let
•••
o •
k
= 1,
v
-k+3 (0')
be the
••• , M , where
0'
th
or--
row of the matrix
= T(i-1)
+ t , and
r(k)
T
is a band matrix
S
••
152
whose
~ off-diagonal elements are 1 and all other elements are
zero.
Then, the row vector
has a
v
-k+3 (0')
in column
1
13
for
which
= T(i-l)
+ t + k
if
t,;; T -
13 = T(i-l) + t - k
if
t> k
13
k
and
the remaining elements of
o.
are
v
-k+3 (0')
The next theorem colle cts well-known useful properties of direct
product of matrices. Proofs are given in many matrix textbooks;
•
~.a.,
Graybill (1969) •
Theorem 12.6
If
a
is a real number and
A,
Band
C are any matrices,
then
® (a B)
(aA)®B = A
(b)
(A ll9B) @C = A X (B® C)
(c)
(A@B)' =
(d)
If
A
tr(A
(e)
If
and
® B)
A, C
A'0 B'
B are any square matrices, then
( f)
If
A
n x n
and
~
= tr(A) tr(B)
are
matrices, then
.e
= a(A ~ B)
(a)
mxm
B
D are
n x n
(A®B)(Ce9D) = Ac0 BD
B are
matrix,
matrices and
m x m matrices and
(A + B)
(i9 C
C
is an
= (A ~ C) + (B
liD C) .
••
153
(g)
If
A
and
A Q9 B
(h)
(i)
P1
Pl
®P2
If
Q
l
and
A
P
are idempotent matrices, then
2
is also idempotent' •
and
Q2
are orthogonal matrices, then
is an orthogonal matrix;
Ql@Q2
If
then
is nons ingula r, and
If
(j)
Bare nonsingular matrices,
is an
matrix, then
m x m matrix and
0
B
is an
n x n
m
det(Aug B) = det(An)det(B )
The following theorem establishes a very useful representation
•
for a real symmetric matrix.
(Bellman (1960),page 64)
Theorem 12.7
Let
A
be an
n x n
real symmetric matrix with characteristic
i = l , c .. ,n,andlet
roots.f..i'
~i'
i = l , .... , n , b e a
corresponding Set of orthonormal characteristic vectors of
A
A.
Then
can be expressed as
n
E. = v.v!
where
l.
satisfying
(12.4)
L: AiE
i
i=l
A =
-~-l.
E.E. = 0
l.
J
... ,
i = '1,
if
i
."
j
n
are positive semidefinite' matrices
n
2
E. = E. , and
E. = I
L:
l.
l.
l.
n
i=l
•
The expression (12.4) is called the spectral decomposition of the
ma trix
A •
-e
154
The following theorem establishes a known result.
a proof was not found in the surveyed literature, we
However, since
provid~d o~e.
Iheorem 12.8
Let
C be
Band
A x A and
TxT
symmetric matrices,
A
respectively, with spectral decompositions
T
i: i1 t.£t.£~
t=l
decomposition
C =
Then,
~
(B
9
(B
+ (IA
®C)
The identity matrix
e
multiplicity
T
i.xl
IT
IT) + (IA
A
=
01 C)
i:
.
~=l
A.b .b ~
~-J.-~
and
has spectral
T
i:
i:
i=l t=l
has a characteristic root
and any orthonormal set of
dimensional space, say
B =
t=l,~
£r(t)
set of characteristic vectors for
IT.
T
of
vectors from Tcan be chosen as a
•• ,T
So,
1
IT
has spectral
decomposi tioo
T
IT = t:l
Similarly,
I
A
£r(t)~(t)
•
has spectral decomposition
A
I
where
A
=
~(i) ,
i:
i=l
i = 1, ••• , A , is any orthonormal set of
from A-dimensional space.
~(i) = ~i
get
-e
'
i = 1,
A
vectors
So, in particular, we can choose
...... , A , and
o
""'f(t)
= c
-t'
t
= 1, ...... , T ,
t.o
~.
155
A
c c')
-t-t
( L
1=1
A
+ (L
i=1
-1-1.
T
L
A
L
=
b_b~
i=1 t=1
L
[O"E.,.!?:
M\X !.L c c')J
~ ~ ~ iX'
I:':J ~t~t') + (b.b
-'.-1~ CJ
t-t-t
1=1 t=l
T
A
=
t=l
T
L
A
=
T
C')
X '..." "....C
t-t--t'
L
L
1=1 t=l
A
T
L
L
1=1 t=1
•
A
L
=
o
i=1
12.3
Proofs Of Lemmas 5.1 And 5.2
Le.mma 12.1
k
= 1,
••• , M+3 , be the matrices defIned in (3.9).
Then,
Eor
i = j = 1
i = j = 2
1 = j = 3
i = 1 ., j = 2, 3
i = 2', j = 3
i = j = 4,
-e
i = l', j =
otherwise
... , M+3
4, ... , M+3
-e
156
Proof:
The proof consists of showing the result for each pai.r of values
•
i,j
(i,j
= 1,
••• , M+3) •
For example, according to the properties
of direct product collected in Theorem 12.6, we have
tr(VlV ")
J
e
3
= tr[(IA !X'
J )(1 fx1r(j-3)n = tr(I 'x' J_r U - ))
\~ T
A '0 T "
A 1:J T T
= A.(number
of elements in
= 2A(T-j+3),
j = 4, ... , M+3.
0
Lemma 12.2
be the
Let
elements are
1
TxT
band matrix whose
and all the other elements are
. th of f ."d'.1.agona "I
1:-
O.
Then,
(12.5)
k' ,k = 4,
.e
••• , T+2,
k I < k , whe re
••
iS7
I
E
l
=
(
0 .
0
I
I
0
T-(k-3)
\
~
~}
o )]
0
k-k'
<-(k-3)
k'-'
and
0
E
Z
=
0
0
I
0
T-(k-3)
)
t
k'-3
2
T-(k-3)
i
k-k'
f
0
J
Proof:
•
Let
G(i)
be the
TxT
right of the main diagonal is
band matrix whose
1
element to the
and the remaining elements are
o .
Then,
r(k'-3)r(k-3)
= [G(k'-3)
+ G(k'-3)'][G(k-3) + G(k-3)']
(k'-3)'G(k-3)
+
+ G
Now, we can write, alternatively,
••
.th
1-
G
(k'-3)' (k-3)'
G
•
(lZ.6)
_e
i58
e'
--k-2.
k-3
~
= \•.;!.•••_
'"
0 -l"'-'[-(k-3)'
e
e
) or
0'
T-(k-3)
0'
o
where
with
i
th
is a
T x 1
r x
is a
vector of zeros, and
element uuity and all. the other elements
0
1
The:!,
,
e
~k'-2
k-3
e'
.oJ['
~
(0
0 e l •• .~.
'k 3»
~ ... ~-,L-t-
O'
r-(k' -3)
,k-3
k'-il
~~
=
(20· •• Q 9.···Q ~1···~T-{k-3)-(k'-3»
= G(k-3)+(k'-3) •
••
vee. tor
159
Thi.s implies, by symmetry,
= [G(k-3)+(k'-3)1'
•
Then,
+ [G(k-3)+(k'-3)J'
=
•
r (k-3)+(k' -3) .
(12.7)
On the other hand,
,
~k'-2
...
e'
-T
(~k-2· •• ~ Q ••• Q)
0'
T - (k'-3)
O'
k-3
~,
= (~k-k'+1·· ·~-(k' -3)+1
£.. ·2)
_e
160
0'
...
•
k-k'
0'
e'
-1
...
=
T-(k-3)
(12.• 8)
e'
-T-(k-3)
0'
K'-3
e
0'
and
0'
k'-3
0'
(Q••• Q ~1·· ·~-(k-3)·)
e'
-1
~
k-3
e'
-T-(k'-3
-e
=
(t;} ~kl_2"·~-(k-k')+1)
k-3
·e
161
0'
k'-3
0'
~
e'
-k-2
=
T-(k-3)
(12.9)
e'
-T-(k-3)+1
0'
k-k'
e
0'
The theorem follows by substituting the results (12.7), (12.8) and
(12.9) in (12.6).
0
Corollary 12.8
Let
rei)
T
elements are
be the
1
TxT
band matrix whose
and all the other elements are
i
th
O.
off-diagonal
Then,
k = 4, ••• , T+2 ,
where
•
-e
E
is the matrix
(12.10)
_.
162
I _
k 3
0
0
0
2I T_ 2 (k_3)
0
o
o
if
k,.
T
[2"J
+ 3
or
I
o
•
o
T- (k-3)
o
01
o
o
-T+2(k-3)
o
I
if
k
> [~J + 3
T-(k-3)
Proof:
Immediate from the previous lemma.
0
Lemma 12.3
Let
Then,
.'
~.
V ,
k
k = 1, ••• , M+3 , be the matrices defined in (3.9).
tr(V.V.V.V,)
~ J ~ '"
has order
0
of magnitude as follows:
.e
163
Order of
i
j
1
1
1
1
i
1
1
1
1
1
.b..
1
2
3
1
1
1
k
1
2
1
2
1
2
1
3
1
2
1
k
1
3
1
3
1
3
1
k
1
k
1
k
1
k
1
k'
2
1
,
2
2
•
2
3
1
2
k
Order of
tr(V,V,V.v£,)
J
.L
AT
AT
AT
L
4
3
3
3
AT
2 2
A T
i
j
-
j,
3
1
3
2
AT
3
1
3
3
AT
3
1
3
k
AT
3
2
3
3
AT
3
2
3
k
AOT O
3
3
3
3
AT
3
3
3
k
AO·i
3
k
3
k
AT
3
k
3
k'
AOT O
k
1
k
2
AT
k
1
k
3
AT
k
1
k
k
AT
k
1
k
k'
AT
k
2
k
3
AT
k
2
k
k
k
2
k
k'
k
3
k
k
k
3
k
k'
AOTO
AOTO
k
k·· k
k
AT
tr(V ,v ,v .V£,}
1
J
1
"
e
2
2
AT2
AT
2
AT
2
AT
2
AT
2
AT
3
AT
?
A"T
2
A T
/
A'I T
2
2
2
2
2
2
2
3
2
2
2
k
3
A T
AOTO
2
3
2
3
2
A T
k
AOTO
2
3
2
2
AOT O
AOTO
2
k
2
k
A T
k
k
k
k'
AOTO
2
k
2
k'
AOTO
k
k' k
k'
AT
k
k" k
k'
AOTO
kit < k I < k
where
and
k, k'
,
k" = 4,
." ,
~
Moe 3
.
?..E£2.!.:
The proof consists of showing the result for each of the stated
combinations of indices of
ViV/iV£,
.
The procedure is too tedious.
To indicate the method of proof we will work out three typical cases ~
.e
For that we shall use the properties established by Theorem 12.6 and
the reS:llt of Corollary 12.8.
164
•
= tr(I
ix) J_r(k-3)
J r(k-3»
TT
TT
A -
e
Now, i f
Band
C
are matrices with
respectively, and the product
Be
.. th
~J-
e 1 ements
b ..
~J
and
c.. ,
~J
is defined, we have
(12.11)
Then,
T-(k-3) -T+2(k-3) T-(k-3)
(12.12)
•
where
~
is a
T x 1
vector with all elements equal to
we have two cases to consider.
-e
2.
So,
165
( 1)
k ,,;
T
[2"J +
3
T-2(~-)
k-3
•
k-l
tr(J_r(k-3) J r(k-3), ~
T T
T T
-
k-3
T-2(k-l)
k-3
?
= 4[T -
(ii)
(k-3)
r
T
k > ['2J + 3
T-(k-3)
•
T- (k-3)
T-(k-.3)
= 4[T
- (k-3)l
T-(k-3)
2
_
,Therefore,
k
••
~
c=
4, •••
= A tr(J_r(k-3)r(k-3)r.(k-3)\
',T
T
T
'
M+3 •
;1
•
166
w~
compute
results (12.12) and Corollary 12.8.
tr(J_r(k-3)r 2 (k-3»
,-
1- T
,,Ie fixsc ('Gmpute
We have to consider foue cases.
T
Te si.mplify
notation we omit the subscript indicating t.he dimeu.si.on of the
vector.
( i)
=1'tr(JJ(k-3)r 2 (k-3) J ) :
T
T T
T
T'
r(k-3)r2(k-3»
tr ( J'F T
T
T-4(k-3)
k-3
k-3
=ftr[(l...1
e
2 ••
k-3
·3. 2:.••• 1, _~ ...2, 1.···1)
2(k-3)
T-4(k-J)
2(k-3)
Q...• L 1....L
z....Z.
1···1. .1.••. y]
Z[ZT-5(k-3)J
(ii)
-Hil (k-3)
k-3
T-3(k-3)
T-3(k-3)k-3
l~tr[ (10. •••1. .£···1 .£···1.£···1 1···1)
T-Z(k-3) -T+4(k-3)
•
-e
T-2(k-3)
<1. ••• 1. 1....1. Q••• Q 1,···.1 L·· .j),J
= Z[ZT-5(k-3)J •
k-3
-e
167
( iii)
T
[3] +
3 < k ~
T
i 21 + 3
-T+3(k-3)
-T+3(k-3)
T-2(k-3)
T-2(k-3)
T-2(k-3)
1
-T tr (l...l
1.. ·1
~ ... ~
1... 1 l .. ·l)
T-2(k-3)
T-2(k-3)
Q... o Q...Q Q...Q l .. ·l) .
(l...l
= 2iT-2(k-3)l
(iv)
k> [i1 + 3:
In this case we have
2(k-3) > T , which imply that
•
( k-3)
rT
We next compute
tr(J
E)
2
2k > T +
r T (k-3)
= 0 •
6 ,
hence
Therefore,
E defined in Corollary 12,8.
We have two cases.
( i)
( ii)
k 3
tr(Jrfi - )E)
T
k"i 2]+3
k >
T
[21 +
3 :
We now can obtain
expression (12.10).
(i)
tr(J
ri
k 3
- )E)
tr(VkV1VkV )
k
= 2[2T-3(k-3)J
= 2[T-(k-3)J
•
•
using these results and
There are three cases to consider.
_.
168
(ii)
(iii)
T
[)"J + 3 < k,;;
k>
T
[2"J
T
[2"1 +
3
+ 3
------So, in summary,
T
8[T-?(k-3)J, k,;; [3J + 3
T
T
2[3T-5(k-3)l, ['3] + 3 < k,;; [2"J + 3
•
T
2[T-(k-3)1, k> [2"J + 3
Lenuna 12.4
Suppose
k
= 1,
X satisfies Assumption 1 of Section 3.2 and let
••• , M+3 , be the matrices defined in (3.9).
X'X = O(AT)
•
••
Then,
V ,
k
••
169
Moreover, the presence of one or more of the matrices
k = 4, ••• , M+3 , in any position between
X'
and
X
in a finite
product of matrices does not affect the order of magnitude of the
product.
Proof:
The proof of the first part of the lemma is immediate from the
observation that, according to the hypothesis,
•
To prove the last assertion of the lemma we observe that
V3 = I AT '
hence its presence in a matrix product does not affect the order of
magnitude of the product.
Vk '
(k
k :: 4,
= 4,
... ,
The same is true for the presence of any
M+3 , since any matrix
... , M+3)
B
premultiplied by
is a matrix whose elements in a column are
elements or Sum of two elements in the corresponding column of
B.
0
Proof of Lemma 5.1:
The proof can be done by showing the result for each combination
••
of the indices
first note that
i
and
j
in
tr(NViNV )
j
i, j = 1, ... , M+3 .
We
170
,-
+ tr[X'V.X(X'X)-lx'v.X(X'X)-ll
J
~
(12.13)
.
Now the proof of the result for each combination of indices
and
j
is obtained by using in (12.13) the results of Lemmas 12.1
and 12.4.
•
For example,
•
=
AT
2
- 2T
2
2 -1
-1-1
tr([(AT) X'V1Xl[(AT) X'Xl J
Since each expression enclosed within square brackets is
follows that
tr(NV1NV 1)
= 0(AT 2 )
•
0
N
=I
0(1) , it
Proof·of Lemma 5.2:
We first note that, letting
have
••
i
- P ,
P
= X(X'X) -1X'
, we
••
171
tr(NV.NV.NV.NV )
~
J
~
.t
= tr(V.V.V.V
)
~J~.t
- 2 tr(V.V.V.V P)
~J~t
- 2 tr(V.V V.V.P) + 2 tr(V V.V.PViP)
~.t~J
.t'~J
+ tr(V.V V.PV.P) + tr(V.V.V.PV P)
~.t~J
~J~.t'
+ 2 tr(V.V.PV.V P) - 2 tr(V ViPVjPViP)
1J
.t
~.t
- 2 tr(V.V.PV.PV P) + tr(ViPV.PV.PV P) •
1J~.t.
J
1
.t
(12.14)
•
The lemma can be proved by checking the order of magnitude of
tr(NV.NV.NV.NV)
1J~.t,
= 1,
(i,j,.t
for each combination of indices
i, j,
,t
To determine the order of magnitude for a
... , M+3)
particular combination of indices we have to compute the order of
magnitude of each term of the corresponding right-hand side
expression (12.14).
So, the proof is very tedious.
of
However, it can
be shortened by observing that each term on the right-hand side of
(12.14) is a multiple of the trace of a product of matrices
(k = 1,
with
P
... , M+3)
Since
and
P
P , always starting with some
= X(XIX)-~'
position in the product of matrices.
k
= 1,
and ending
, the commutativity of the factors
under the trace can be used to move the last factor
V '
k
Vk
X'
to the first
Then, the special pattern of the
••• , M+3 , allows simplification of matrix powers and
commutativity, so that Lemmas 12.3 and 12.4 can be used and the order
_e
172
of magnitude of the trace can be obtained by use of Theorem 12.1. Thus,
for example,
.
~
•
(__1__ XlV X) (1- X'X)-l]
Ai
(1AT
~.
1
AT
.
X' X)-l] + T4 tr[(__1__ X'V X)
AT2
1
(1- X'X)-l(__l__ X'V X) (1- X'X)-l
AT
AT2
1
AT
..
'
••
173
since, according to Lemma 12.4, each expression enclosed within
parenthesis is
O( 1) •
0
"
.. '
"
...
''..
t
.
,
•
··li._.
",
"
i.
\
,
"
.'