Multiregression Dynamic Models
Author(s): Catriona M. Queen and Jim Q. Smith
Source: Journal of the Royal Statistical Society. Series B (Methodological), Vol. 55, No. 4
(1993), pp. 849-870
Published by: Wiley for the Royal Statistical Society
Stable URL: http://www.jstor.org/stable/2345998 .
Accessed: 22/10/2013 11:53
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp
.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact [email protected].
.
Wiley and Royal Statistical Society are collaborating with JSTOR to digitize, preserve and extend access to
Journal of the Royal Statistical Society. Series B (Methodological).
http://www.jstor.org
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
J. R. Statist.Soc. B (1993)
55, No. 4, pp. 849-870
Multiregression
DynamicModels
By CATRIONA
M. QUEENt
and
University
of Nottingham,UK
JIM Q. SMITH
University
of Warwick,Coventry,
UK
[ReceivedJuly1990. Final revisionJune19921
SUMMARY
Multiregression
dynamicmodels are definedto preservecertainconditionalindependence
structures
overtimeacross a multivariate
timeseries.Theyare non-Gaussianand yetthey
can oftenbe updatedin closedform.The firsttwomomentsof theirone-step-aheadforecast
distributioncan be easily calculated. Furthermore,
theycan be built to contain all the
featuresof theunivariatedynamiclinearmodeland promisemoreefficient
identification
of
causal structures
in a timeseriesthanhas been possiblein thepast.
Keywords: DYNAMIC LINEAR MODELS; GRANGER CAUSALITY; INFLUENCE DIAGRAMS;
MULTIPROCESS MODELS; NON-LINEAR TIME SERIES; RECURSIVE SIMULTANEOUS
EQUATION MODELS
1. INTRODUCTION
Thispaperintroduces
a classof multivariate
statespacetimeseriesmodels,called
multiregression
dynamicmodels(MDMs), definedon a multivariate
timeseries
{Yt t>1.Thesemodelshavebeendevelopedtoforecast
timeserieswhosecomponents
arehypothesized
to exhibit
certainconditional
in anyonetimeframe
independences
andthereis believedtobe a causaldriving
mechanism
within
thesystem.
To illustrate
thetypeof timeserieswhichmayhavetheseproperties,
Section5.1 considers
the
problemof forecasting
themarketshareand relativemarketpriceof a particular
brandina productmarket.Thecausaldrivethrough
thisparticular
is clearly
system
seenas thebrand'srelative
pricetendsto dictateitsmarket
share.
MDMs definea class of non-Gaussian
timeseriesmodelswhichdecomposethe
forecasting
systemintocomponents
whoseconditional
distributions
are univariate
Bayesiandynamic
linearregression
models(DLMs) (HarrisonandStevens,1976).As
in simultaneous
equationmodels,thecomponents
ofthevectorYtareregressed
on
othercontemporaneous
of thatvector.Sincetheforecasting
components
modelis
expressedin termsof univariateDLMs, facilities
availableforthe modellingof
univariateseries,such as intervention,
trendsand seasonals,can be transferred
on to thesemodelswithno appealto stringent
directly
symmetry
conditions
being
necessary(as in Harvey(1986) and Quintanaand West (1987)). Althoughthe
modelledprocessescan be highly
non-linear
theyareofteneasilyupdatedin closed
form.Thismakestheprocessespecially
becausethemodelsareamenable
interesting
to analyticalinvestigation.
or numerical
Approximate
methods,whichare often
requiredfornon-Gaussian
series(forexample,Kitagawa(1987)and Pole and West
(1990)),withall therobustness
issuesthatsurround
them,arelargely
unnecessary.
Thepaperis organizedas follows.TheMDM is firstsetup in Section2 andthen
tAddress for correspondence:Mathematics Department, Universityof Nottingham,.University Park,
Nottingham,NG7 2RD, UK.
? 1993 RoyalStatisticalSociety
0035-9246/93/55849
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
850
QUEEN AND SMITH
[No.4,
defined
inSection3. Resultspresented
inthelatter
sectionareformally
provedin
fully
theproperties
oftwointeresting
Section4. Section5 examines
specialcasesofMDMs
and it is shownthat,despitebeingnon-linear,
thefirsttwomoments
of theirjoint
can be calculatedalgebraically
in closedform.Section5.1
forecastdistributions
howanMDM hasbeenusedtoforecast
illustrates
a brand'spricerelative
totherestof
shareandtheformofthenon-linearity
withitsmarket
themarket
together
thatthey
a discussion
isstudied.Finally,Section6 offers
ofsomeoftheproperties
exhibit
ofan
MDM. It is shownhow an MDM, represented
by a givengraphof an influence
a specific
conditional
diagram
(defined
later),exhibits
'Granger
causal'structure.
Itis
howsuchcausalstructures
canbe testedbyusingmultiprocess
illustrated
modelling
techniques.
Throughoutthe paper the followingresultsfromthe theoryon conditional
andgraphswillbe used.Anyrandomvectors
independence
X, Y, Z andW willhave
conditional
thefollowing
independence
properties:
(a) X lI Y I (Y, Z),
(b) XlIYIZo* YlIXIZ,
X IIYI(Z,W)
with
(c) XI1(Y, Z) IW X* together
XIIzIW
ofY givenZ (seeDawid(1979)).
whereX 1IY IZ readsX is independent
Let (X1, . . ., Xm)be an orderedsetof randomvectorson which*1 1 *is defined.
m- 1 conditional
statements
aregiven:
Supposethatthefollowing
independence
XrliP'(Xr) I P(Xr)
2 < r< m
(1.1)
whereP(Xr) C {XI, . . ., Xr I} and P'(Xr) = {Xl,. . ., Xr I} \ P(Xr) where, forany
in B whicharenotin A. A setof
twosetsA and B, B\ A denotesthoseelements
such as thesecan be usefullyrepresented
statements
conditionalindependence
A graphG(X, E) consistsof a set of nodesX and edgesE, wherea
graphically.
directed
edgefromXj to Xk is denotedby (Xj, Xk) e E. A directed
graphwitha
directed
edgefromXitoXrifandonlyifXie P(Xr) iscalledthegraphofan influence
diagram-seeHowardandMatheson(1981),Shachter
(1986)andSmith(1989,1990).
m fromXito Xk is a
ThesetP(Xr) is knownas theparentset ofXr. A path oflength
sequenceof nodes,Xi = Xi,0,. . ., Xi m = Xk, where(Xi,(j-1), Xi j) e E foreachj = 1,
withpathsto Xk is calledtheancestorset of Xk and is
m. The setof vertices
withtheconditional
denotedbyan(Xk). Thegraphofan influence
diagramtogether
statements
independence
(1.1) is calledan influence
diagram.
2. SETTING UP THE MODEL
howtheheuristic
causalrelationships
Thissectiondescribes
betweencomponent
ofa multivariate
timeseries{Yk }k < t candefinean appropriate
processes
forecasting
MDM.
modelwhichformsthebasisofthegraphical
in recentyears.
Causationbetween
processeshas beenstudiedquiteextensively
of causalityrequired
Granger's
(1969)originaldefinition
that,forall timet, a best
linearestimateof V,based on { Uk} k<t and { Vk}k<t need onlydependon { Vk}k< t. A
of Grangercausalityis givenby Lauritzen(1989). Here,
graphicalrepresentation
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
1993]
MULTIREGRESSIONDYNAMIC MODELS
851
definition
of Florensand Mouchart(1985)is used-theysay
thestronger
however,
thatU is a non-causeof V,ifforall timet
Vt iI (Uk}k<
{
tVk}k
< t
Thus the originalidea of Grangeris thenreplacedwitha definition
based on
itselfandnowthebestestimate
conditional
independence
thanthebestlinear
(rather
estimate)of Vtbased on { Uk }k I tand { Vk}k < t needonlydependon { Vk)k < t. Byusing
a setofconditional
thisdefinition
statements
relatedtocausality
independence
canbe
definedacross processes.Althoughcausalitydefinedin termsof conditional
independence
does notquitecapturewhatis usuallymeantbycausality(Holland,
1986),non-causality
ofXand Ycan oftenreasonably
be considered
as a consequence
oftherebeinga lackofcausalrelationfromXto Y. Wermuth
andLauritzen
(1990)
heuristically
arguedthat,whendealingwithgraphicalmodels,variableswhichare
to be causallylinkedshouldbe connected
hypothesized
bydirected
edgesconsistent
of causality.Thusa heuristic
withthedirection
influence
diagramcan be created
theconditional
representing
independence
relatedtocausality
between
thevariables
at a fixedtimeperiodt.
Forexample,consider
threebrands{X, Y,Z} ina business
so thatthesales
market
ofbrandXcan be considered
as a causalfactorindetermining
Y's salesandthesales
ofbrandY can be considered
as a causalfactorin determining
Z's sales.LetXt, Yt
thesalesat timet ofbrandsX, Y and Z respectively.
andZt represent
Theheuristic
withthecausalrelationships
graphoftheinfluence
diagramconsistent
inthismarket
is glvenin Fig. 1.
Supposethatthesameinfluence
thevariablesatalltime
diagramalwaysrepresents
points so that P{ Yt} = Xt and P{Zt} = Yt for t a 1. Suppose furtherthat the
processesoverall timepointsup to and including
timet can be represented
bythe
graphoftheinfluence
diagramofFig. 2 suchthat
C {xt-l},
P{Xt}
P{yt}
P{Zt}
CZ IXtAt-1),
C
{Yt,zt-'}
thecausal relationships
betweenthesales
Fig. 1. Heuristicgraphof theinfluencediagramrepresenting
at timet of brandsX, Y and Z
Fig. 2.
{Xk}k
the causal relationshipsbetweenthe processes
Graph of the influencediagramrepresenting
t, { Yk}kS<
and {Zk}k<
t
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
[No. 4,
QUEEN AND SMITH
852
where Wt = { W1, . . ., W'I}T for any time series { Wk }k , 1 such that AT denotes the
transpose of any matrixA.
It is clear from Fig. 2 that the conditional independences related to causality are
such that
YtH {Zk}k<tI
{Yk}k<t,
{Xk}k?t.
This leads naturally to a general definition of conditional causality which extends
Florens and Mouchart's (1985) definitionso that Uis a non-cause of Vgiven Wif for
all points in time t
Vtl
{Vk}k<t,
{Uk}k<tI
{Wk}k6t.
A best estimate of Vtnot only now depends on the past series { Vk }k < tand { W }k<t
but also on the currentvalue Wt. So, in particular, Fig. 2 implies that Z is a non-cause
of Y givenX. Thus thebestestimateof Yt based on {Xk }k t, { Yk }k <t and {Zk }k t
and {Xk }k < t. An appropriate forecastingmodel for
need only depend on {Yk}k<t
each variable at time t is simply a functionof its own past series, the past series of its
parents and the value of its parents at time t. For example, the general form of an
appropriate forecastingmodel for the series { Yk}k t iSgiven by
Yt =f(xt yt-l ot) +
Vt,
wheref( ) is some function and vthas some probability distribution.This reasoning
can be generalized directlyto the case where there are n series.
The MDM uses this methodology for deriving its observation equations and
through the special form of the system equation breaks a complex multivariate
problem into n univariate problems. Notice how the causal relationships between the
variables are accommodated and that the stringentsymmetryconditions imposed on
the variables in the models of Harvey (1986) and Quintana and West (1987) are not
required. The MDM will now be formallydefined.
3. MULTIREGRESSIONDYNAMIC MODEL
Let yT = { Yt(1), Yt(2), . ., Yt(n)} denote the general n-dimensional multivariate
timeseries at time t. The observed value of Yt(r) is denoted byyt(r); let yt(r)T= { yl(r),
.,y(r)} and set
Xt (r)T=
Zt(r)T
{Yt(1),
Yt(2),.
. .,
I
+ 1), ... .
{Yt(r
Yt(r- 1)}
Yt(n)
2 < r<
n,
2 < r < n- 1.
Suppose that for any fixed time t the structureof the series Y, can be heuristically
representedby an influence diagram I such that
Pt YA61) ': Xt(r).
Suppose furtherthat the process
influence diagram I* such that
P{Yt(r),
Let
Et
=
{t(j)T9
ot(2)T,
{Yk }k
t
can be heuristically represented by an
CZ{Xt(r)bYtht(r)et
Ot. f(n)T } be the state vectors determining the
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
1993]
853
MULTIREGRESSIONDYNAMIC MODELS
and lets, be thedimensionofthe
of Y,(1), Y,(2), . . ., Y,(n)respectively,
distributions
set
>
convenience
notational
For
1.
t
vector0,(r),
t(r)T
= {Ot(j)T,.
t(r)T =
t(r+
j)T9
2 < r<
. ., Ot(r- 1)T}
. 0..
t(n)T
2
<
r
<
nX
n-1.
Call {Yt}t> I an MDM if it is governedby the followingn observationequations,
equations
andsystem
wheretheobservation
equationandinitialinformation
system
on theseequationsgivenbelowhold
on thepastandtherestrictions
areconditional
forall pointsintime:
(a)
observation equations1< r
vt(r) - (O, Vt(r))
Yt(r) = Ft(r)T Ot(r) + vt(r)
<
n;
(3. 1)
(b) system equationOt =
(c)
wt - (0, Wt);
1 + wt
GtOtO
(3.2)
initial information(00 IDo) -
(mo. co).
(3.3)
but known
columnvectorFt(r)is allowedto be an arbitrary
The sr-dimensional
functionof xt(r)and yt- 1(r),but not zt(r)and yt(r); Vt(1),. . ., Vt(n)are theknown
scalar observationvariances;thes x s matricesGt = blockdiag{Gt(1), . . ., Gt(n)},
Wt = blockdiag{Wt(1), . . ., Wt(n)} and CO = blockdiag{CO(1),. . ., CO(n)} are
assumedknownand are suchthatGt(r), Wt(r)and CO(r) ares, x sr squarematrices
of pastvectorsxt-1(r)and yt-1(r)butnothingelse. Thus
whichmaybe functions
{Yt(r)
Vt(r).
Iy
1,
withmeanFt(r)T Ot(r)and variance
Ft(r),Ot(r)}followssomedistribution
The errorvectorsvT = {vt(l), . .
.,
vt(n)} and wT = {wt(1)T,
.
.
.,
wt(n)T}, where
errorvectorfor
system
errorand wt(r)is thesr-dimensional
vt(r)is theobservation
.
and
.
.
.,
variables
1
that
r
such
are
wt(1), ., wt(n)areall
vt(n)
vt(l),
< n,
Yt(r), <
with
independent
mutually
are
vectors
the
and
mutuallyindependent
{vt,wt}t,>
time.
the
fora vectortimeseries,consider
howan MDM wouldbe defined
To illustrate
time
fixed
t,
at
any
where,
.
=
{
that
YT
.,
Yt(1),.
followingexample.Suppose
Yt(4)}
MDM
The
3.
in
influence
the
of
the
diagram
given Fig.
Ytcanberepresented
by graph
forYtwouldhaveobservationequationsinwhichFt(1) is a functionofyt-1(l); Ft(2) is
a function of {yt(l), yt-1(2)}; Ft(3) is a function of {yt(l), yt(2), yt-1(3)}; Ft(4) is a
functionof {yt(2), yt(3), yt-'(4)}.
X
2I
Fig. 3. Graphof theinfluencediagramfor Yt(1),...,
Y(4)
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
[No.4,
Thereare twoimportant
resultswhichare centralto thetheoryof MDMs, the
will
be
of
which
giveninthenextsection.It is shownthatif
proofs
854
QUEEN AND SMITH
,1=
I
et- l(r) IYt-
musthold.
thenthefollowing
conditional
independence
statements
Result 1.
,1=1 Iet(r)
I Yt.
(3.4)
Inwordsthismeansthatif{Ot-1(r)}aremutually
independent
giventhedatayt- then
that,
undertheDLM {Ot(r)}arealsoindependent
givenYt.Itwillfollowbyinduction
that{00(r)} areinitially
independent
for
provided
independent,
theparameters
remain
availableinformation.
all timegiventhecurrent
Result2.
Ot(r) l zt(r)I xt(r), yt(r).
(3.5)
intermsofthecomponents
ofytthisreads
Written
Ot(r)liyt(r+ 1). yt(r+ 2), ... .,yt(n)Iyt(I)q. . .,9yt(r).
In wordsthismeansthat,giventhepastobservation
vectorsof thefirstr indexed
oftherestofthepastdata.
series,Ot(r)is independent
theMDM, COis setto be blockdiagonalandso theparameters
Whendefining
for
each variableare initiallymutuallyindependent.
Therefore,by result1, the
associatedwitheach variableare updatedindependently
aftereach
parameters
and remainindependent
at each timepoint.Thus, as each variable
observation
univariate
followsa conditional
distribuBayesiandynamic
model,theconditional
andconditional
tionforeachvariablecanbe updatedindependently
forecasts
canbe
suchas intervention,
foundseparately.
trendsand
Bayesianforecasting
techniques
on to thesemodels.A complexmultivariate
transferred
seasonalscan be directly
beendecomposed
inton univariate
problemhas therefore
problems.
of normality
has beenmadeforeitherresults1 and 2 or when
No assumption
theMDM. Itfollowsthatlargeclassesofmultivariate
suchas those
defining
processes
ofthedecomposition.
discussedbyKitagawa(1987)can be usedas thecomponents
function
of {xt(r),yt-'(r)},becauseit is
However,evenwhenFt(r)is a non-linear
fixedattimet,a normalMDMcanbedefined
suchthat
assumedknownandtherefore
vtand wtare Gaussian so that,conditionalon xt(r), Yt(r) is Gaussian. The MDM is
particularly
simpleto workwithinthiscase. Each variablefollowsa normalDLM
are
distributions
and, as such,updatingand forecastsof conditionalunivariate
identicalwiththosegivenbyHarrisonand Stevens(1976).Section5 discussestwo
dynamicmodel
specialcases of normalMDMs calledthe linearmultiregression
(LMDM) and the correctedlinearmultiregression
dynamicmodel(CLMDM) in
which Ft(r) is a linear functionof {xt(r),
yt-
'(r)}.
In reality,
theseriesareobservedsimultaneously
so thatxt(r)willnotbe observed
beforea forecastfor Yt(r)is made. The marginalforecastdistributions
foreach
variableare therefore
willnotbe
required.In generalthesemarginaldistributions
ofYtformanyMDMs canbe derived
Gaussian.However,themoments
fairly
easily.
ofthefirst
twomoments
ofYtfortheCLMDM andthefirst
moment
Thederivation
in Section5.
oftheLMDM areillustrated
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
1993]
MULTIREGRESSION
855
DYNAMIC MODELS
Results1 and2 combinetoenablethejointone-step-ahead
of
forecast
distribution
Considertheforecast
distribution
forn brands:
Y, to be simplified.
P(Yt IYt 1=
| p{ytIt,yt1}p{tIyt1}
dOet.
statements
of result1, together
The conditionalindependence
withthe structure
of thevectorFt(r)specified
bytheMDM, ensurethatthejointdistribution
can be
of yt(r)with
expressedas the productof the individualforecastdistributions
regressors
xt(r).So
P(ytIyt-1) =
r
=
r
p { yt(r)Ixt(r),yt-(r)}
p I yt(r)Ixt(r),yt-l(r),0t(r)}p 0t(r)| yt-1
} dOt(r).
dt,(,)
statements
of result2 and thespecified
However,bytheconditional
independence
of Gt(r),p(Ot(r)Iyt- 1) onlydependson xt-1(r)andyt- 1(r)andthuscan be
structure
as
rewritten
p{Ot(r)Iyt1} =p1{t(r) Xt-(r),yt-l(r)}
Thenextsectionformally
provesresults1 and2.
4. FORMAL PRESENTATIONOF RESULTS
To provetheresultsofthelastsection,thefollowing
theorem
givenbyPearland
Verma(1987), Pearl (1988) and in its presentformby Lauritzenet al. (1990) is
all the conditionalindependences
required.It identifies
definedby an influence
diagramand gives an algorithmfor decidingwhetherany givenconditional
can be logicallydeducedfroman influence
independences
diagram.
Theorem1. Supposethatconditionalindependence
statements
for a set of
inan influence
variablesarerepresented
whosegraphisI, andthatU, Vand
diagram,
Wdenotesetsofvariableson it.Adapttheinfluence
diagraminthefollowing
way.
(a) Formthe directedsubgraphI, of I whosenodes are in the ancestorset
an(U, V, W) andwhosedirected
edgesarethoseinI whichliebetween
these
nodes.
(b) Joinallpairsofnodes(X, Y) e P(Z) byan undirected
edge,whereP(Z) isthe
thegraphsince
parentsetofZ andZ e II. Thisprocessis knownas moralizing
all parentsofa singlevariablearejoinedbyan arc.Call thismixedgraphI2.
all directed
(c) Forman undirected
graphJbyreplacing
edgesinI2 byundirected
edges.
Then
UllVIw
ifall undirected
a nodeR e U andS e Vmustpassthrough
pathsinJbetween
a node
W.
Te
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
[No.4,
if thisconditionis violated,thenthereis a probabilistic
Furthermore,
influence
diagram
theconditional
respecting
independence
statements
oftheinfluence
diagram
forwhichUIl VI Wis nottrue.
Theorem1 can nowbe directly
usedto provethefollowing
theorem.
2. Let {Yt}t, I be governed
Theorem
byan MDM and,usingthenotationofthe
assume
that
previoussection,
Ot- i(r) llyt-l(r), zt- l(r) Ixt- l(r)
r=2, . . n.
(4.1)
856
QUEEN AND SMITH
t-l(r) Ixt-l(r), yt-1
et-l(r)zt-l(r),
Ot-i(r)ot-i(r)Ot-
l(r)Iy t-I
r= 1,. . . n,
r=2, . . n- 1.
(4.2)
(4.3)
Thenthefollowing
conditional
independence
statements
mustalso be true:
Ot (r)
Ot(r)
r=2, ........
,n,
........
r=l 1.........
....... ,n,
1y t(r) .z t(r) Ix t(r)
1z t(r) .
t(r) Ix t(r).y
t(r)
t
Ot(r) 1 t(r) . t(r) IY
,n- 1.
r=2, .............
.
(4.4)
(4.5)
(4.6)
Proof. To provethisresult,
itis first
thattheconditional
necessary
independence
intheinductive
statements
contained
hypotheses
(4.1)-(4.3)andtheMDM itselfare
represented
1itis thensimpletoshowthatconditional
graphically.
Byusingtheorem
statements
independence
must
also
hold.
(4.4)-(4.6)
For clarityof presentation,
theinductive
hypotheses
(4.1), (4.2) and (4.3) are
represented
arcsintheseparate
bythedirected
graphsofinfluence
ofFigs4,
diagrams
5 and 6 respectively.
In each diagram,variableshavebeenallowedto sharenodes
whenit is convenient
to do so and whenthereis no loss ofinformation.
Thus,for
example, in Fig. 4 zt- l(r- 1) = {yt- l(r), Zt- 1(r)} and At-1(r-
1) = {Ot-l(r), At-1(r)}
Thejustification
forthemissing
directed
arcsinthesegraphsofinfluence
diagrams
willnowbe presented
all undirected
(forthemoment
arcsareignored).
B
I
( 1)r
I_ _V
_(
_
_
Fig. 4. The directedarcsof thisgraphgivethegraphof theinfluencediagramforinductivehypothesis
(4.1); thedirectedand undirectedarcs make up themoralizedgraph
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
1993]
MULTIREGRESSION
DYNAMIC MODELS
857
B
4~~~(r)
I
- - -4-(--)
-
A~~~~~0()0r
arcsofthisgraphgivethegraphoftheinfluence
Fig. 5. Thedirected
forinductive
diagram
hypothesis
andundirected
arcsmakeup themoralized
(4.2); thedirected
graph
B
A
IC
arcsofthisgraphgivethegraphoftheinfluence
Fig. 6. Thedirected
forinductive
diagram
hypothesis
andundirected
(4.3); thedirected
arcsmakeup themoralized
graph
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
[No.4,
statedbythe
(4.2) canbe equivalently
(c) ofSection1,assumption
Usingproperty
pairof assertions
858
QUEEN AND SMITH
i(r)11Ot_i(r) IY- l,.7
Ot_
et- l(r)
(4.8)
zZt- l(r) Ixt- l(r), yt- 1(r).
yt- 1
nodesofsubvectors
(4.8), arcsbetween
(4.1) andassertion
Now,byassumption
of
thanrtothosenodesofsubvectors
allhaveanindexinYtgreater
whosecomponents
In
haveindiceslessthanorequaltorareallowedtobe omitted.
Otwhosecomponents
vectors
parameter
(4.3) allowarcsbetween
(4.7) andassumption
assertion
contrast,
to be omitted.Thisjustifiestheimplied
whichdo notsharethesamecomponents
intheboxeslabelledA in Figs4-6.
statements
independence
conditional
TheblockdiagonalformsofGtand WtintheMDM ensurethat
Ot(r) 1et- I \ {t- l(r)} IYt- 19 et- 1(r)
1 < r < n.
of
setsofcomponents
diagramanyarcsbetween
Thuson thegraphoftheinfluence
This
be
omitted.
can
index
common
no
with
of
of
sets
and
Ot
etcomponents
diagramboxes.
theomissionof arcsinbox B oftheinfluence
justifies
argument
implicitin the observation
statements
Finallythe conditionalindependence
equationsoftheMDM meanthat
Mtr) l et\ fot(r)) IYt- l(r), Xt(r),Ot(r)-
diagram
justifytheomissionof arcsin box C of theinfluence
Thesestatements
graphs.
wheneach
arcsinFigs4-6 arethoseextraarcswhichareintroduced
Theundirected
Itisnowsimpletocheckthatthesetofnodes
inmoralized.
diagrams
oftheinfluence
thenodesassociated
variablesblockall pathsbetween
theconditioning
representing
1the
So bytheorem
setsUand Vclaimedtobeindependent.
indifferent
withelements
D
resultis proved.
statedin thefollowing
in Section3 can nowbe formally
The resultsintroduced
2.
oftheorem
corollary
i.e. if11fl 100(r),then
1. Ifinan MDM theinitialstatesareindependent,
Corollary
forall timest
l1r= I Ot(r) I Yt
and
Ot(r) 11y t(r + 1),9. . . , yt(n) Iyt(I)q
yt(r).
2. Fromthehypotheses
Proof. To provetheresultfort= 1proceedas intheorem
equations(setC) thegraphof
equation(setB) andtheobservation
(setA), thesystem
arcsofthegraphin Fig. 7.
bythedirected
diagramI is represented
theinfluence
arcsof
2, again,theundirected
as intheorem
thesameargument
Byusingexactly
whenthegraphis moralized,and so theconditional
Fig. 7 are thoseintroduced
2 cannowbe deducedfortimet= 1.
statements
(4.4)-(4.6)oftheorem
independence
2 mustbetruefort= 1,andso
theorem
sinceHrLI 00(t)underthecorollary,
Therefore,
2 mustbe trueforall timest.
oftheorem
theassertions
byinduction
from
statements
independence
2 holdsforalltimest,theconditional
Sincetheorem
can be combined:
thetheorem
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
19931
MULTIREGRESSIONDYNAMIC MODELS
B
C~(
)
ID~~~~~
859
I
E()j1
arcsof thisgraphgivethegraphsof theinfluence
diagramfortheproofof
Fig. 7. The directed
1; thedirectedand undirectedarcs make up themoralizedgraph
corollary
1I{.ot(r),CO~()},zt(r)Ixt(r),yt(r).
Ot(r)
(4.9)
Now, by property(c) (Section 1) statement4.9 impliesthat
Ot(r)I ,I{t(r),
,t(r)} zt 1(r),xt i(r),,yt) (r);
therefore,
rn 1at(r Iyt.
Statement(4.9) also impliesthat
Ot(r)llzt(r) Ixt(r),yt(r)
and so
Ot(r) llyt(r + 1), . . . , yt(n) Iyt(I)q
yt(r);
the corollaryand hencetheresultsof Section3 are proved.
therefore
D
Theseresultsapplyto a verywideclass of models.However,itis helpfulinitiallyto
considertheimplicationsfortheLMDM and CLMDM becausethejoint distribution
of the variablesin thesemodelscan be calculatedexplicitlyand the one-step-ahead
forecastshave a particularlysimpleform.
5. LINEAR MULTIREGRESSIONDYNAMIC MODELS
Considerthe MDM definedby equations(3.1)-(3.3). This sectionintroducestwo
special cases of the MDM-the LMDM and the CLMDM-which are especially
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
860
[No. 4,
QUEEN AND SMITH
simpleto workwith.
of yt- 1 and
Supposethat{v,(r),w,(r)}) I arejointlyGaussian,areindependent
Ft(1)doesnotdependon Yt. Now,let
2
Ft(r)T = {xt(r)T,it(r)T},
wherext(r)T =
{ yt(1), .
<
r
n
. . yt(r- 1)} and it(r) is a setof knownexogenousvariables
supposethat
onxt(r).ThisprocessisanLMDM. Alternatively,
thatarenotdependent
Ft(r)T=
[{xt(r) - -ft(r)}T,it(r)T],
where
(5.1)
E{ Yt(r- 1) I yt- 1}]
xt-1(r)andyt-1(r).
and,fromresult2 ofSection3, ft(r)onlydependsonyt- 1through
ft
=
[E Yt(l) Iyt- 1}, . .
.,
onxt(r).
thatarenotdependent
variables
Onceagainit(r)isa setofknownexogenous
Thisprocessis a CLMDM. Thus,giventhecomponents
wt(r)whichprecedeit,each
containedin the
as a univariateDLM withregressors
component
Yt(r)is described
above.
given
vectors
Ft(r)
simultaneous
equations
versionofa recursive
NotethattheLMDM is a stochastic
variableslisted
againsta subsetof contemporary
modelwhereYt(r)is regressed
specialcase of this.Unlikein
beforeYt(r).Harvey(1989) considersa degenerate
buton the
estimation
theemphasishereis noton parameter
economics,however,
continued
thatYtcanexhibit
giventheinevitable
jointdistribution
typesofpredictive
stateintheprocess.
abouttheregression
uncertainty
between
brandsmodelledbyan LMDM havea different
Thecausalrelationships
fromthosemodelledbya CLMDM. Forexample,supposethatthere
interpretation
to be a
arejusttwovariables,{ Yk(1)) 1 and { Yk(2)}k,, I, suchthatY(1) is thought
changein
causal factorof Y(2). If {Yk}k I followsan LMDM, thena long-term
Y(1)'s levelwouldcause a sustainedlevelchangein Y(2). However,if {Yk }k
levelchangein Y(1) wouldhave a
followsa CLMDM, thenthesamelong-term
on Y(2). In thiscase thelevelchangein Y(2) wouldbe short
causaleffect
different
term,followedbya driftbackto theoriginallevel.Thisis due to thefactthatthe
unlikethe LMDM whichuses the actual
CLMDM uses residualsas regressors
observation.Thus the CLMDM essentiallyrelatescausalitythroughforecast
of
distribution
theforecast
of Yt(1)helpsinadjusting
so thatmisforecasting
residuals
would
in
level
a
sudden
Y(1)
change
true.
not
Therefore,
the
reverse
is
whereas
Yt(2)
leadto a largeresidualvalueinthemodelforY(2), thusleadingto a levelchangein
intheregression
Y(2). As themodelforY(1) adaptstothelevelchange,sotheresidual
level.
backtoitsoriginal
and Y(2) willdrift
termin Y(2)'s modelwillbecomesmaller
aniceinwhicha CLMDM wouldbe a usefulmodel,consider
a situation
To illustrate
and Yt(2)isthemarket
creammarket.
SupposethatYt(1)isthetotalsalesofice-cream
shareofice-cream
typeA. Nowiftherewerea heatwavethetotaldemandYt(1) oficeincrease.If brandA hadextrastockandcouldcopewiththe
creammightsuddenly
shareYt(2)wouldbe
brandB, say,thenthemarket
thananother
extrademandbetter
be modelledby a
This
situation
could
wave.
the
heat
over
increase
to
expected
CLMDM suchthat
Yt(2) = Ot+ 0t Yt(1)-E[Yt(1)Jyt-1]}
+ Et,
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
MULTIREGRESSIONDYNAMIC MODELS
19931
861
whereEtis someerror
termand0, is someparameter
associated
withA's ability
tocope
withextrademand.
For boththeLMDM and theCLMDM, theseparatecomponents
{ Yt(r)Ixt(r)}
haveeithera Gaussianor Studentt-distribution
depending
on whether
theforecast
distribution
is assumedknownorestimated.
Theforecast
distributions
andupdating
relationships
associatedwiththeseconditional
univariate
modelsareidenticalwith
thoseofWestandHarrison(1989).Itthenfollowsfromtheorem
2 thattheone-stepaheadforecast
ofYtis simply
density
theproductoftheunivariate
conditional
onestep-aheadforecastdensitiesof { Yt(r)Ixt(r)}, forr= 1, . . ., n. Althoughthe joint
forecast
distribution
ofYtwillnot,ingeneral,
be Gaussian,itsmeanandcovariance
matrixtakea relatively
simpleformand thesewillnow be derivedhere.Assume
thisderivation
throughout
thatall meansand variancesarefoundconditionally
on
xt(r).
The meanofthemarginal
forecast
distribution
forYtwhenitfollowsan LMDM
willfirstly
bederived.
As hasalreadybeenmentioned,
eachvariableundertheLMDM
followsa univariateregression
DLM. Usingthe forecastdistributions
for the
regression
DLMs givenby Westand Harrison(1989),p. 111,it is clearthatthe
oftheconditional
expectation
forecast
distribution
of{ Yt(r)Ixt(r)}giventhepastcan
thenbe rewritten
as
=
E{ Yt(l) Iyt- '(1)} a-(?)(
E{ Yt(r) Iyt-l(r), xt(r)} =
at(?(r) +
I
r-1
i
E at()(r)
Yt(i)
i=l
where at(0)(r)is a function of {it(r), yt- 1(r),xtt(r)}
2 < r< n
(5.2)
only.
The marginalforecastmeansforYtgiventhepastcan thenbe easilycalculated
fromtheseconditional
meansbyusingtheidentity
E{ Yt(r) Iyt-l(r), xt- 1(r)}
=
E[E{ Yt(r) Iyt-l(r), xt(r)}].
(5.3)
Writea (O)T = {at(?(1), . . ., a(?)(n)} and A as then x n lowertriangularmatrixwhose
{ j, k)thelement
ajkis givenby
ajk=
k <j,
at(k)(j)
(0
otherwise.
Then,byusingidentity
(5.3),equation(5.2)acrossallr= 1,..., n canbeexpressed
by
E(YtIyt-1) = a(?) + A E(YtIyt-1)
and so
E(Yt|yt-1)
= (I-A)-1a"0.
for the CLMDM, the expectation
Similarly,
of the forecastdistribution
for
{ Yt(r)Ixt(r)}giventhepastcanbe rewritten
as
E{ Yt(1) yt- 1(1)} =
-
E{t Y(r)
Iyt- 1(r),xt(r)} =
()(09
r-1
jt(O)(r)+ E at(i)(r){Yt(i) - ft(i)}
t=1
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
2 < r< n
(5.4)
862
[No. 4,
QUEEN AND SMITH
toa different
valueoftheanalogousa(i)(r)ofequation(5.2)
whereja(i)(r)corresponds
and ft(r)is thesame as in equation (5.1). By usingidentity(5.3) it is obviousthatthe
theCLMDM aresimply
meansforYt(r)following
marginal
forecast
E{
Y (r) Iyt- l(r), xt- l(r)}
= jt()(r).
of Ytgiventhepastis foundfromthe
Although
themarginal
covariancematrix
in boththeLMDM and theCLMDM, thecovariance
samerecursive
relationships
formandso onlythederivation
ofthismatrix
oftheCLMDM takesa simpler
matrix
to findthevarianceofthe
itis first
willbe shownhere.To findthismatrix
necessary
distribution
of Yt(r),forr = 1, . . ., n.
forecast
marginal
= {IO*(r)T,
isthesetofparameters
contained
0E*(r)
SupposethatOt(r)T
Gt(r)T} where
is thesetassociatedwithit(r). If Rt(r)is the
in Ot(r)associatedwithxt(r)and #t(r)
of {0t(r) Iyt- } thenRt(r)can be expressed
covariancematrixof thepriordistribution
by
=
Rt(r)
R
Rt*(r) Rt'(r))
\R'(r)T At r)!
where
cov{IO*(r),et*(r)Iyt- 1},
*(r)
Rt(r)
=
cov{Ot(r), #t(r)y 1},
R'(r)
=
cov {IO*(r), t(r)
yt- 1}.
forecast
distribution
of { Yt(r)Ixt(r)}given
thevarianceoftheconditional
Therefore,
as
byWestand Harrison(1989),p. 111,can be rewritten
var{ Yt(1) Iyt-
var{ Yt(r) yt- l(r), xt(r)}
1(1)}
= rt2(r)+
=
2(1),
h {xt(r)}
2
Sr <
n
where
h {xt(r)}
= trace[St(r)T{xt(r) - ft(r)}
{xt(r) - ft(r)}TSt(r)]
whereeach scalar'r2(r) 00, r= 1,. . ., n, is a functionof {it(r), yt- l(r),
R'(r), Vt(r)}and St(r)St(r)T= R *(r).
xt- l(r), Rt(r),
forecast
distribution
ofYt(r)giventhepast,it
To findthevarianceofthemarginal
holds:
notedthatforanytwovariablesZ(1) andZ(2) thefollowing
identity
is first
(5.6)
var{Z(2)} = E[var{Z(2)I Z(1)}] + var[E{Z(2)I Z(1)}].
Suppose thatEt is theforecastcovariancematrixforYt suchthat
{2t }jk=
rt(j,
k) = cov{ Yt(j), Yt(k) Iyt- 1}
j, k= 1, .
. ., n
and letEt(r)be theforecastcovariancematrixof{ Yt(1),. . ., Yt(r- 1)} forr= 2,. . ..
withequations(5.4) and (5.5) itcan be
n. Therefore
byusingidentity
(5.6) together
shownthat
at(I, 1) = r2(1),
ot(r, r) = E[var{ Yt(r) Iyt- l(r), xt(r)}]
+
var[E{ Yt(r) Iyt- l(r), xt(r)}]
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
MULTIREGRESSIONDYNAMIC MODELS
1993]
863
2
=
2(r) + trace{S,(r)TE,(r)S,(r)} + a (r)
at(r)
whereit(r)T = {d(l)(r)..
1)(r)}
a(,
r<
n
To findthemarginalcovariancebetweenYt(r)and Yt(k),notethatforthetwo
variablesZ(1) and Z(2)
E{Z(1)Z(2)}
= E[E{Z(2)Z(1)|Z(2)}]
Therefore
supposethatZ(1)
thatforr > 2
=
Yt(r)andZ(2)
ort(r)T= {ft(r, 1),. . .,
= E[Z(2)E{Z(1)fZ(2)}].
=
Xt(r)
andletot(r)be a vectorsuch
o(r, r-1)} = cov{Yt(r),Xt(r)Iyt}
Then
ot(r) = E[Xt(r) E{Yt(r) I Xt(r)} Iyt-11.
Byequation(5.4) thisbecomes
ot(r) = d(?)(r)E{Xt(r) Iyt-1} +
r-1
Z d(i)(r)E[Xt(r){ Yt(i)- ft(i)]Iyt 1
i=l
It is clearthat
Et(r+ 1)
ENt(r) at,(r))
(r) Tort(r,
r)
\rt
So ifEt(2) = a(1, 1) is given,thenEt(r+ 1) canbe calculatedas a simplefunction
of
Et(r), ot(r) and ot(r, r). Hence themarginalforecastcovariancematrixoftheCLMDM
andexpectations
is simply
calculatedfromthevariances
oftheupdateddistributions
fortheseparateconditional
DLMs.
component
regression
Thesemodelsarebestillustrated
bya simpleexample.
5. 1. Example
A simpleexamplewhichillustrates
thisnewclassofmodelscanbe constructed
as
follows.Suppose thatyT = { Y,(1), Yt(2)} is to be modelledbyan LMDM whereYt(1)
thepriceof a certainbrandrelative
to therestof themarketand Yt(2)
represents
thebrand'smarket
share.Supposethatat anyfixedtimethetwovariables
represents
inFig. 8(a). Supposefurther
canberepresented
bythegraphoftheinfluence
diagram
thattheprocesses{ Yk(l )}k < tand { Yk(2)}k < tcan be represented
bythegraphofthe
influence
diagramofFig. 8(b).
The generalLMDM forthissystemis givenby thefollowing
observation
and
system
equations:
Yt(1)
Yt- (2)
(a)
Fig. 8.
Graphof theinfluencediagramsfor(a) Y,(1) and Y,(2) forfixedtimetand (b)
{ Yk(2)}k < t
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
<
{ Yk(l)}k kand
864
QUEEN AND SMITH
equation(a) observation
vt(l) - N(O, Vt(1)),
Yt(1)= Ft(1)TOt(1) + vt(j),
Yt(2) = Ft(2)TOt(2) + vt(2),
equation(b) system
I + wt,
Ot= GtOtO
(c) initialinformation-
vt(2)- N(0, Vt(2));
wt - N(0, Wt);
(0oIDo) - N(mo,CO)
[No. 4,
(5.7)
(5.8)
(5.9)
(5.10)
whereFt(1) = !t(1), Ft(2)T = {l,yt(l)},7
T = {Ot(l)T, 0(o)(2),0(l)(2)}, mT = {mO(l)T,
m60)(2),m6l)(2)},Gtis thes, + 2 square identitymatrix(wheres, is the dimensionof
Ot(l)}, Wtand COare block diagonal and vt(l), vt(2),wt(l) and wt(2) are mutually
independentof each otherthroughtime.
Using exactlythe same notationintroducedto definethemeansand variancesof
theLMDM, and notingthatin thisexamplea(?)(2) = m(t)1(2)and a(l)(2) = m(l)1(2),
to therequiredconditionalforecastmeansand
equations(5.2) and (5.5) lead directly
varianceswhichare givenby
1, yt(1)} = at()(2) + yt(1) at(1)(2)
EYt (2)I yt-
(5.11)
and
var{Yt(1)Iyt-1(1)}
=
t2(j),
var{ Yt(2) I yt- 1,yt(l)} = yt(1)2R *(2) + r2(2)
where'r2(2)is a functionof Rt(2), R'(2) and Vt(2)and in thiscase R *(2) is simplya
scalar.
are simply
The firsttwo momentsof the forecastdistribution
E[Y,Iy
]
Y 1
/
a(O)(1)
(a,?)(2)+ a(O)(1)a(l)(2))
(5.13)
and covariancematrix
(2,9I) ort(2,92))(.4
ort
when
ot(1(,2) = rt(21)
at(0)(2)
E{Yt (1) yt-1} + at(1)(2)EtY(1)21yt-1}
and
2)
rt(2,
=
'r7(2)+ R *(2)E{ Yt(1)2I yt- 1} + a(1)(2)2r2(N1).
A timeseriesplot of { Yk(2)}k,O is givenin Fig. 9 togetherwithvariousone-stepahead forecastsof theseries.The timeseriesitselfis denotedbythedotson thegraph.
The one-step-aheadconditionaland marginalforecastsof Y(2) derivedfromthe
MDM are representedby the thin and thickcurvesrespectively.The conditional
forecastsobtainedfromtheMDM are fairlyaccurate,indicatingthattherelativeprice
ofthebrandappearsto be a good predictorofthebrand'smarketshare.The one-step-
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
1993J
MULTIREGRESSIONDYNAMIC MODELS
865
-
0.25
0.24 0.230.220.21 0.200.19
0.187
O. 19
0.176
0.16
1
2
3
4
5
6
7
8
9
10
11
Time (Years)
Fig. 9. Time seriesplot of Y(2) withone-step-aheadforecasts:...,
, one-steptimeseries;
ahead forecastconditionalon relativeprice; -,
one-step-aheadmarginalforecastderivedfromthe
MDM;
---,
forecasts from the steady model
ahead forecastsof the simplesteadymodel for Y(2) are representedby the broken
curvein Fig. 9. If these are compareddirectlywiththe marginalforecastsof the
MDM, thenclearlythe forecastsobtainedfromtheMDM are farsuperiorto those
obtainedfromthe steadymodel. This illustrateshow the forecastperformanceof a
model for Y(2) can be improvedby using Y(1) as a regressorand eventhoughY(1)
cannot be observed before Yt(2) the MDM can still use Y,(1) to obtain realistic
forecastsof Yt(2).
Clearly, both { Yt(1) Iyt- 1(1)} and { Yt(2) Iyt- 1, yt(1)} have normal distributions.
However,thejointforecastdistribution
of { Yt(1), Yt(2) 1yt 1} is notbivariatenormal
because of theappearanceofyt(1)in thevariancetermof { Yt(2) yt y1, t(I)} . Indeed
thisjoint distributioncan be verynon-normal.This is demonstratedin Fig. 10 in
whichthecontoursof thejointdistribution
of { Yt(1), Yt(2)} afterthemarginof Yt(1)
has been normalizedand forvariousparametervalues are given.
FromFig. 10itcan be shownthatthemodesand antimodeslie on a quintic(and so
exhibita butterfly
catastrophe;Zeeman(1977)). Thejointforecastdistribution
is only
whenthe varianceof { Yt(1) yt-} does not appear in the varianceof
symmetrical
{Yt(2) 1yt }, i.e. when m(1)1(2)= 0. For any non-degenerate
values of the parameters,it can be concluded,aftera littlealgebra,thatthereis a value ofyt(2)suchthat
theconditionalpredictivedensityof { Yt(1)IYt(2)=yt(2)} is bimodal. The worstcase
occurswhenOt(2)is uncertainwherethedistribution
becomesverynon-Gaussian.As
- 0, thentheprocesstendsto a bivariatenormaldistribution.
However,unlike
Rt*(2)
the analogous simultaneousequationsmodelsthe assumedstochasticdrifton Ot(2)
preventsthislimitfromeverbeingreached.
FromFig. 10 it is clearthatthepointE[Yt(1) Iy't 1(1)] (in thiscase Yt(1)= 0) takes
on special significanceas it is thepointabout whichthecontoursare symmetrical
or
asymmetrical.Regressionon the forecastresidual [Yt(1) - E{ Yt(1)Iyt-1(1)}] will
oftenthereforeseem more natural. This gives the correspondingCLMDM whose
meansand covariancematrixare givenby
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
Y2
[No. 4,
QUEEN AND SMITH
866
II
13
2
V2
10~~~~~~~~~~~~S1
-U~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~t
S
to
-LU
~~~~~~~~~~~-to
-10
-IS~~~~~~~~~~~~~R
-4
-3
-U
-L
O
?
-to
U
4
3
-3
-4
-U
-to
4
-a
Y2
Y2
-U
-3
U
L
o
-,
S
Y2
/
a
-tOS
Lo
-to
-US
-U0S
-Uso
-a.a
-a -l.
-I
-0.13
5
,
20
.5
L2L.U
IUS-.s
I5
1L
.I
U-.
I-0.5
a.a.1
U
U.U
no a.
a5-
-S-Ur1
ha-IS-L
ennraiedt0aeOma
-2
a
Yt
VII
5
2
an unto vaiac
2
and mt
(
4
U
L
a
3
Vi
-lU aa-I -5.5
i.a a
5
.
yi
Yi
*0
O
-LsU
,()=
2
0i
h
is
-US
-LU
-ISo
-S
a
-U.S-U
U.S
-4
-3
i1
2
3
YL
-3
-U
I
S
3
VI
LU
tl,2
I~~~~~S t(1
of Y,(1)
of
and
the
when
contour
Various
10.
marginaldistribution
plots Y,(1)
asymmetric
Fig.
Yt(2)
has been normalizedto have 0 mean and unitvarianceand m- ~(2) * 0; Rt*(2) = T2;(2) = 10 in thefirst
row,R *(2) = 10, -r?(2)= 1 in thesecondrowand R *(2) - 1 r(2) = lO0inthethirdrow; mt- (2) variesby
column,takingthe values 1 in thefirstcolumn,5 in thesecondand 10 in thethird
IS
E[Y~Iyt~'I (a)(1))
=
=
where
lt(2boat2,2))
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
1993]
MULTIREGRESSION
867
DYNAMIC MODELS
rt(2,1)
at(l)(2)E[{ Yt(1)- ft(1)}2I yt
rt(2,2) = rt2(2)+ R *(2) E[{ Yt(1)
t(1)}21yt-I + a(l)(2)2 var[{
rt(1,2) =
=
],
Yt(1) ft(1)} Iyt-I
rt2(2) + rt2(1){R *(2)+a (1)(2)2}.
The graphsin Fig. 10 willstillbe appropriateforthiscorrectedmodel.
6. DISCUSSION OF MULTIREGRESSIONDYNAMIC MODELS
Some of the interesting
theoreticalaspects of MDMs will be discussed in this
section.
MDMs definea class of non-Gaussiantimeseriesmodels whichdecompose the
forecastingsysteminto componentswhose conditionaldistributions
are univariate
Bayesian dynamicregressionmodels. As was mentionedin Sections3 and 5, these
univariatemodelscan be normal.In particular,ifFt(r)is a linearfunctionof {xt(r),
yt- l(r)}, theneach variablewould followa generalizationof one of Priestley'sstatedependentmodels(Priestley,1980). Alternatively,
each modelcould be a non-linear
dynamicmodel (West et al., 1985; Migon and Harrison,1985; Pole et al., 1988). If
each componentfollowsa univariateBayesianlinearor non-linearmodel,thenit is
possibleto use theupdatingrelationships
directlyfromunivariateBayesiandynamic
models,eventhoughthemodelregresseson contemporary
componentsof Ytand can
be highlynon-linear.This maketheprocessespeciallyinteresting
because themodels
are amenableto analyticalinvestigation.
Approximateor numericalmethods,withall
therobustnessissuesthatsurroundthem,are largelyunnecesary.However,as can be
seen fromFig. 10, thevectorYt can have a joint forecastdistribution
whichis very
non-Gaussian, even when Ft is a linear function of (Yt(1), . .
.,
Yt(r- 1)). In this
respectthisclass of modelsis verysimilarto themodelsof Wermuthand Lauritzen
(1990) in the factthat the conditionaldistributions
are fairlysimplebut the joint
model is farmorecomplex.
It is interestingthat, unlike conditional independencewithin time frames,
conditionalcausality of the type describedhere depends on the orderingof the
variablesin theMDM. Furthermore
each influencediagramcorrespondsto a unique
LMDM or CLMDM structure.
For example, suppose that for five variables modelled with an LMDM the
followingregressionequationshold:
YM(I) = O(O)(1)+ vM(1);
Yt(2) = a(?)(2) + 0(l)(2)yt(1) + vt(2);
Yt(3) = O(o)(3)+ O(l)(3)yt(1)+ O(2) (3)yt(2)+ vt(3);
(6.1)
Yt(4) = 0O()(4) + O (3)(4)yt(3) + vt(4);
Yt(5) = O()(5) + 6 (3)(5)yt(3)
+ vt(5).
The graphof theinfluencediagramconsistentwiththeseequationsis givenbygraph
(a) in Fig. 11.
It is easyto checkthattheLMDM on theseregression
equationscould alternatively
be representedby the LMDM on componentsof Y takenin the order{ Y(1), Y(2),
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
868
[No. 4,
QUEEN AND SMITH
(a)
(b)
Fig. 11. Two graphswiththesame impliedconditionalindependences,but withdifferent
LMDMs
Y(3), Y(5), Y(4)}. The graph of the influencediagram,Fig. 11(a), would remain
unchanged.
In generalany new orderingof thevariablesthatis compatiblewiththeinfluence
diagramof a singletimeframeof an LMDM or CLMDM, giventhepast,willproduce
equationsthatare algebraicallyidenticalwiththeoriginal.
Howeverbythedecompositiontheorem(Smith,1989),foranygiventimeframet,
theimpliedsetof conditionalindependencestatements
bygraph(b) in Fig. 11, given
the past, are identical with those representedby graph (a), but the conditional
independencesrelated to causalityin the two diagramscorrespondto two quite
different
LMDMs.
For example,fromgraph(a) it is clearthat
Yt(2) Iyt 1 = Yt(2) Iyt1(2), yt-1(1)
whereasgraph(b) meansthat
Yt(2) I yt- = Yt(2) I y"-(2),
yt-'(I),
yt-1(3).
Because ofthedifferent
covariancestructure
on thesystemerrorwtimpliedbyeach of
theseinfluencediagrams,thereis no guaranteethatthesetwo statements
could hold
simultaneously.
Suppose that the contextof the model is such that the time frameconditional
independencesare logicallydetermined.However,suppose thatthereis uncertainty
thecausal structure
regarding
acrossthevariablesintheproblem,althoughthiscausal
structure
is assumedconsistentovertime.In thiscases a class I multiprocess
MDM
(Harrisonand Stevens,1976) can modelthesystem.A multiprocess
MDM considers
m models {M(1), . .
.,
M(m)} simultaneouslywherethereis one model foreach of the m
possible causal structuresfor the given conditionalindependencestructure.They
allow thepredictionof complexserieswithoutanya prioriassumptionsabout causal
Thisis becausetheforecastsare foundbyusingp(YtIyt- 1,M(i)) mixedwith
structures.
probabilitiesp(M(') Iyt-) to give appropriatepredictivedensities. Multiprocess
modelscan also be combinedwithsome decisionrulewhichcomparesthe forecast
of thevariousmodelsso thatthemostappropriatecausal structure
performance
can
be selectedfromthen alternatives.
AlthoughGrangercausalityhas attractedacademic interest,in practiceit has
proveddifficultto discriminatebetweendifferent
causal structures
by usinglinear
systems.However, by embeddingcausalityin the non-linearMDM, multiprocess
MDMs can givean on-lineassessmentof hypothetical
causal relationships
acrossthe
variables.The MDM corresponding
to a givencausal structure
can thenbe made at
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
1993]
MULTIREGRESSION
DYNAMIC MODELS
869
the conditionalcomponentscan be as complicatedas
least plausible. Furthermore,
necessary,containingtrends,regressionterms,seasonal factors,and so on. It
looks promisingthattheMDMs willenabletheselectionofcausal structures
therefore
across practicalmodels.
Noticethatfromthegeometriesof thejoint densitieson themodel of Section5.1
most informationabout causal relationshipsseems to come whenthe relationship
betweenthe variablesis uncertain(in thisexamplethiscorrespondsto R *(2) being
large). This willoccur earlyin the seriesand afterexternalintervention.
This might
explain whyZellner(1984) findsit so difficultto discriminatebetweentwo causal
structures-themodels thathe considersare not dynamic;theywould assume that
is notconsidered.
Rt*(2)- 0 and externalintervention
7. CONCLUSION
MDMs have been developedto modelmultivariate
timeserieswhosecomponents
exhibitconditionalindependencesrelatedto causality.Althoughhighlynon-linear,
the models are relativelysimpleto implement.Preliminaryuse of thesemodels on
serieswitha small numberof componentsseems promising.Use of the MDM to
model a largebusinessmarketis documentedin Queen (1992).
ACKNOWLEDGEMENTS
We wouldliketo thanktheScienceand Engineering
ResearchCounciland Unilever
Researchforthe financialsupportgivenwhileundertaking
thisresearch.
REFERENCES
Dawid, A. P. (1979) Conditionalindependencein statisticaltheory(withdiscussion).J. R. Statist.Soc.
B, 41, 1-31.
Florens,J. P. and Mouchart,M. (1985) A lineartheoryfornon-causality.Econometrica,53, 157-175.
Granger,C. W. J. (1969) Investigatingcausal relationsby econometricmodels and cross-spectral
methods.Econometrica,37, 424-438.
Harrison,P. J.and Stevens,C. F. (1976) Bayesianforecasting
(withdiscussion).J.R. Statist.Soc. B, 38,
205-217.
Harvey, A. C. (1986) Analysis and generalisationof a multivariateexponentialsmoothingmodel.
MangmntSci., 32, 374-380.
(1989) ForecastingStructuralTimeSeriesModels and theKalmanFilter.Cambridge:Cambridge
Press.
University
Holland, P. W. (1986) Statisticsand causal inference.J. Am. Statist.Ass., 81, 945-970.
Howard, R. A. and Matheson, J. E. (1981) Influencediagrams.In Readings on the Principlesand
Applicationsof Decision Analysis (eds R. A. Howard and J. E. Matheson), vol. II, pp. 719-762.
Menlo Park: StrategicDecision Group.
Kitagawa,G. (1987) Non-Gaussianstate-spacemodellingof nonstationary
timeseries(withdiscussion).
J. Am. Statist.Ass., 82, 1032-1063.
Lauritzen,S. L. (1989) Mixed graphicalassociationmodels. Scand. J. Statist.,16, 273-306.
Lauritzen,S. L., Dawid, A. P., Larsen, B. N. and Leimer,H. G. (1990) Independencepropertiesof
directedMarkov fields.Networks,20, 491-505.
Migon,H. S. and Harrison,P. J. (1985) An applicationof non-linearBayesianforecasting
to television
and advertising.In Bayesian Statistics2 (eds J. M. Bernardo,M. H. DeGroot, D. V. Lindleyand
A. F. M. Smith).Amsterdam:North-Holland.
Pearl, J. (1988) ProbabilisticReasoningin IntelligentSystems:Networksof Plausible Inference.San
Mateo: Morgan Kaufmann.
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions
870
QUEEN AND SMITH
[No. 4,
Pearl,J.and Verma,T. S. (1987) The logicofrepresenting
dependenciesbydirectedgraphs.In Proc. 6th
Natn. Conf. ArtificialIntelligence,Seattle,pp. 374-379. Los Altos: Morgan Kaufmann.
Pole, A. and West,M. (1990) EfficientBayesianlearningin non-lineardynamicmodels.J.Forecast.,9,
119-136.
Pole, A., West,M. and Harrison,P. J.(1988) Non-normaland non-lineardynamicBayesianmodelling.
In BayesianAnalysisof TimeSeriesand DynamicModels (ed. J. C. Spall). New York: Dekker.
models:a generalapproachto non-lineartimeseriesanalysis.J.
Priestley,M. B. (1980) State-dependent
TimeSer. Anal., 1, 47-71.
Queen, C. M. (1992) Using the multiregression
dynamic model to forecastthe brand sales in a
competitiveproductmarket.Research Report 92-10. NottinghamStatisticsGroup, Universityof
Nottingham,Nottingham.
Quintana,J. M. and West,M. (1987) Multivariatetimeseriesanalysis:newtechniquesapplied to internationalexchangeratedata. Statistician,36, 275-281.
probabilisticinference.In Uncertainty
and ArtificialIntelligence(eds
Shachter,R. D. (1986) Intelligent
L. N. Kanal and J. Lemmer),pp. 371-382. Amsterdam:North-Holland.
Smith,J. Q. (1989) Influencediagramsforstatisticalmodelling.Ann. Statist.,17, 654-672.
(1990) Statisticalprinciplesof graphs.In InfluenceDiagrams,BeliefNets and Decision Analysis
(eds R. M. Oliverand J. Q. Smith),pp. 89-120. New York: Wiley.
Wermuth,N. and Lauritzen,S. L. (1990) On substantive
researchhypotheses,
conditionalindependence
graphsand graphicalchain models. J. R. Statist.Soc. B, 52, 21-50.
West,M. and Harrison,P. J. (1989) BayesianForecastingand DynamicModels. New York: Springer.
West, M., Harrison,P. J. and Migon, H. J. (1985) Dynamicgeneralisedlinearmodels and Bayesian
forecasting(withdiscussion).J. Am. Statist.Ass., 80, 73-97.
Zeeman, E. C. (1977) CatastropheTheory:SelectedPapers (1972-1977). Reading:Addison-Wesley.
Zellner,A. (1984) Basic Issues in Econometrics.Chicago: University
of Chicago Press.
This content downloaded from 86.30.149.44 on Tue, 22 Oct 2013 11:53:08 AM
All use subject to JSTOR Terms and Conditions