Applied Time Series Notes
White noise: et mean 0, variance 52 uncorrelated
( 14 )
Moving Average
Order 1: (Yt  .) œ et  )1 et-1
all t
Order q: (Yt  .) œ et  )1 et-1  â  )q et-q
all t
Infinite order: (Yt  .) œ et  )1 et-1  )2 et-2  )3 et-3  â all t
Have to be careful here - may not "converge" i.e. may not exist.
Example: Yt  . œ et +et-1 +et-2 +et-3 +â has infinite variance
Pr {Y>C} = Pr{ Z > (Y-.)/_ } = Pr{ Z > 0 } =1/2 for any C (makes no sense!)
Example: Yt  . œ et +3et-1 +32 et-2 +33 et-3 +â
Variance{Y} = 52 (1+32 +34 +â ) = 52 /(1  32 ) for |3|<1.
Yt  . œ et +3et-1 +32 et-2 +33 et-3 +â
Yt-1  . œ
et-1 +3et-2 +32 et-3 +33 et-4 +â
(Yt  .)-3(Yt-1  .)=et +0
Autoregressive - AR(1)
(Yt  .) œ 3(Yt1  .)  et
all t
E(Yt )  . œ 3[E(Yt1 )  .]
Stationarity:
E(Yt ) constant all t (call it .)
Cov(Yt , Ytj ) œ # (j) œ function of j only
E(Yt ) œ .
(if ± 3 ±  1)
Assuming ± 3 ±  1
(yt  .) œ 3(Yt1  .)  et
œ 3[3(Yt2  .)  et1 ]  et
œ 3[3[3(Yt3  .)  et2 ]  et1 ]+et
etc.
(Yt  .) œ et  3et1  32 et2  † † †
Again, E(Yt ) œ .
Var (Yt ) œ 52 (1  32  34  36  † † † ) œ 52 ‚(1  32 )
Cov (Yt , Ytj ) œ 3 j 52 ‚(1  32 )
Applied Time Series Notes
( 15 )
Example:
Plot of # (j) versus j is Autocovariance Function
j=
# (j)
0
64
1
32
2
16
3
8
4
4
5
2
6
1
7
1/2
(1) Find 3, variance of Yt variance of et and .:
3 = 0.5 (geometric decay rate)
Variance of Yt is 64.
Variance of e: [Using # (0) = 64 = 52 /(1-32 ) = 52 /(1-.25)]
52 = 64(.75) = 48.
Covariances have no information about ..
Forecast:
Yn1 œ .  3(Yn  .)  et
. œ 90 known (or estimated)
Data Y1 , Y2 , † † † , Yn with Yn œ 106. We see that 3 =0.5
sn1 œ 90  .5(106  90) œ 98 error Yn1  Y
sn1 œ et
Y
Yn2 œ .  32 (Yn  .)  en2  3en1
sn2 œ .  32 (Yn  .) œ 94, error œ en2  3en1
Y
snj œ .  3 j (Yn  .) error œ enj +3enj1  † † †  3 j1 en1
Y
98, 94, 92, 91, 90.5, † † †
forecasts.
Forecast intervals (large n).
., 3 known (or estimated and assumed known)
Forecast errors en1 ,
en2  3en1 , en3  3en2  32 en1 , † † †
(1) Can't know future e's
(2) Can estimate variance 52 (1  32  † † †  32j2 ).
(3) Estimate 52 .
Use rt œ (Yt  .)  3(Yt1  .) then
5
s 2 œ Dr2t ‚n or

Get S2y œ D(Yt  Y )2 ‚n
then 5
s 2 œ S2y (1  32 )
snj „ 1.96É5
Y
s 2 (1  32  † † †  32j2 )
Applied Time Series Notes
( 16 )
Estimating an AR(1)
Yt = .(1-3) + 3Yt-1 + et = - + 3Yt-1 + et
Looks
Regress Yt on 1, Yt-1 or
_ like a regression:
_
Yt -Y on Yt-1 -Y (noint)
_
n
1. n-1 !(Yt-1 - Y)2 converges to E{ (Yt -.)2 } = # (0) =
t=2
52
1-32
_
2. Èn [ !(Yt-1 -Y) et /n ] is Èn times a mean of (Yt-1 -.) et terms
uncorrelated (but not independent)
_
Nevertheless Èn [ !(Yt-1 -Y) et /n ] converges to N(0, ? ) where
variance is E{ (Yt-1 -.)2 et2 } = E{ (Yt-1 -.)2 }E{ et2 }= # (0) 52
_
_
n
3. Èn (3^ -3 ) = Èn [ !(Yt-1 -Y) et /n ] / [ n-1 !(Yt-1 - Y)2 ]
t=2
in the limit this is N(0, # (0) 52 / # (0)2 ) = N(0, 1-32 ).
EXAMPLE:
Winning Percentage (x 10) for baseball's National League pennant winner.
Regression:
Year
1921
1922
ã
1993
Yt
614
604
ã
642
1
1
ã
1
Yt-1
.
614
ã
605
PROC REG: Y^t =
341.24 + .44116 Yt-1 + et ,
(66.06)
(.1082)
^
Yt - 610.62 = .44116 (Yt-1 -610.62) + et
Year
1994:
1995:
or
1995:
Forecast
341.24 + .44 (642) = 624.46
341.24 + .44 (624.46) = 616.73
610.62 + .442 (642-610.62)
s2 = 863.7
Forecast Standard Error
È863.7 = 29.4
È863.7(1+.442 ) = 32.10
È863.7/(1-.442 ) = È#^ (0)
^
2054: 610.62 + .4460 (41.38) = 610.62 = .
so long term forecast is just mean.
Theory assigns std. error È(1-32 )/n to 3^ . We have È(1-.442 )/73 = .105
Applied Time Series Notes
( 17 )
Identification - Part 1
Auto correlation 3(j) œ # (j)‚# (0)
(For AR(1) , 3(j) œ 3 j )
ACF
Partial Autocorrelation
Regress Yt on Yt1 , Yt2 , † † † , Ytj
Last coefficient Cj is called jth partial autocorrelation PACF.
More formally, #^ (j) = n1 D(Yt  .)(Ytj  .) estimates # (j)
1
n
1
n
Xw X regression matrix looks like
Ô

D(Yt1  Y )2


D(Yt1  Y )(Yt2  Y )
ã
Õ


D(Yt1  Y )(Yt2  Y )

D(Yt2  Y )2
† † †×
† † †
Ø
so formally, Xw Xb œ Xw Y is analogous to the population equation (also "best predictor" idea)
Ô # (0)
Ö # (1)
Ö
# (1)
# (0)
Õ # (j-1) # (j-2)
† † † †
† † † †
# (j  1) ×
# (j  2) Ù
Ù
# (0) Ø
Ô
Ö
Ö
b1
×
b2
Ù
ٜ
ã
Õ bj ( œ Cj ) Ø
This defines Cj œ jth partial autocorrelation
For AR(1), partials are
jœ1
Cj œ 3
2
0
3
0
† † †
† † †
2
0
3
0
Moving Average MA(1)
Yt œ .  et  ) et1
E(Yt ) œ .
Var (Yt ) œ 52 (1  )2 )
Autocovariances
j
0
1
2
2
# (j ) 5 (1  ) )  52 )
4
0
† † †
† † †
Ô # (1) ×
Ö # (2) Ù
Ö
Ù
ã
Õ # (j) Ø
Applied Time Series Notes
Example:
jœ 0
# (j) œ 10
1
4
2
0
3
0
( 18 )
† † †
† † †
4
0
52 (1  )2 ) œ 10
 )52 œ  4
take ratio 4(1  )2 ) œ 10 )
)2  2.5)  1 œ 0
()  .5)()  2) œ 0
)œ
1
2
or ) œ 2
Forecast MA(1)
Yn1 œ . + en1  )en
en has already occurred but what is it? I want
sn1 œ .  ) en
Y
so I need en . Use backward substitution:
en œ Yn  .  ) en1
œ (Yn  .)  )(Yn1  .)  )2 (Yn2  .)  )3 (Yn3  .) † † †
If |)|  1, truncation (i.e. not knowing Y0 , Y1 , etc.) won't hurt too much.
If |)| 1, major problems
Moral: In our example, choose ) œ 12 so we can “invert" the process, i.e. write it as long AR.
Yt  90 œ et  .5et1
Data œ 98
94
92
85
st1 œ 90  .5e
Y
st
89
93
92
(how to start?)
One way: recursion with se0 œ 0
Y.
s.
Y
(0)
8
0
8
4
4
0
2
0
2
5
1
6
s8 œ 90  .5(1) œ 90.5
Y
1
3
2
3
1
2
2
1
1
error e8
s9 œ Y
s10 œ Y
s11 œ † † † œ 90 œ ..
Y
error en + .5 en-1
AR(p)
(Yt  .) œ !1 (Yt1  .)  !2 (Yt2  .)  † † †  !p (Ytp  .)  et
Applied Time Series Notes
MA(q)
Yt œ .  et  )1 et1  † † †  )q etq
Covariance, MA(q)
Yt  . œ et  )1 et1  † †
Ytj  . œ
 )j etj  )j1 etj1  † † †  )q etq
etj  )1 etj1  † † †
Covariance œ [  )j  )1 )j1  † † †  )qj )q ]52
Example
j=
0
# (j) = 285
1
182
2
40
3
0
4
0
(0 if j  q)
5
0
MA(2)
52 [1  )12  )22 ] œ 285
52 œ 100
52 [  )1  )1 )2 ] œ 182
)1 œ  1.3
52 [  )2 ] œ 40
)2 œ  .4
Yt œ .  et  1.3et1  .4et2
Can we write et œ (Yt  .)  C1 (yt1  .)  C2 (Yt2  .)  † † † ?
Will Cj die off exponentially? i.e. is this invertible?
Backshift: Yt œ .  (1  1.3B  .4B2 )et
where
B(et ) œ et1 , B2 (et ) œ B(et1 ) œ et2 , etc.
et œ
1
(11.3B.4B2 )
(Yt  .)
Formally
1
(1.5B)(1.8B)
and


5
3
8
3
1
1X
œ
 53
1.5B

8
3
1.8B
œ 1  X  X2  X3  † † †
if |X|  1
(1  .5B  .25B2  .125B3  † † †
(1  .8B  .64B2  .512B3  † † †
œ 1  1.3B  1.29B2 
† † †
so
,
,
œ 1  C1 B  † † †
Obviously Cj 's die off exponentially.
( 19 )
Applied Time Series Notes
( 20 )
AR(2)
(Yt  .)  .9(Yt1  .)  .2(Yt2  .) œ et
(1  .5B)(1  .4B)(Yt  .) œ et
(1  .5B)(1  .4B)(Yt  .) œ et
Right away, we see that (1  .5X)(1  .4X) œ 0 has all roots exceeding 1 in magnitude so as
we did with MA(2), we can write
(Yt  .) œ et  C1 et1  C2 et2  † † †
with Cj dying off exponentially. Past “shocks" etj are not so important in determining Yt .
AR(p)
(1  !1 B  !2 B2  † † †  !p Bp )(Yt  .) œ et
If all roots of (1  !1 m  !2 m2  † † †  !p mp ) œ 0 have |m|  1, series is stationary
(shocks temporary)
MA(2)
Yt  . œ (1  )1 B  )2 B2 )et .
If all the roots of (1  )1 m  )2 m2 ) œ 0 have |m|  1, series is invertible (can extract et from
Y's).
Alternative version of characteristic equation (I prefer this)
mp  !1 mp-1  !2 mp-2  † † †  !p = 0 stationary <=> |roots|<1.
Mixed Models
ARMA(p, q)
Example: (Yt  .)  .5(Yt1  .) œ et  .8et1
Yt  . œ [(1  .8B)/(1  .5B)]et
œ (1  .8B)(1  .5B  .25B2  .125B3  † † † )et
œ et  1.3et1  .65et2  .325et3  † † †
Yule-Walker equations
(Ytj  .)[(Yt  .)  .5(Yt1  .)] œ (Ytj  .)(et  .8et1 )
Take expected value
jœ0
# (0)  .5# (1) œ 52 (1  1.04)
jœ1
j1
# (1)  .5# (0) œ 52 (.8)
# (j)  .5# (j  1) œ 0
# (0)  !# (1) = 52 Š1  )(!  ))‹
# (1)  !# (0) = -)52
# (0)
Π# (1)
j
3( j)
0
1
Applied Time Series Notes
1
1
 .5
2.04
œ
52 œ Œ
 ”  .5
1 • Œ 0.80 
1
2
3
4
.746 .373 .186 .093
( 21 )
3.2533
2.4266
5
2
etc.
Define # (  j) œ # (j), 3(  j) œ 3(j).
In general Yule-Walker relates covariances to parameters. Two uses:
(1) Given model, get # (j) and 3(j)
(2) Given estimates of # (j) get rough estimates of parameters.
Identification - Part II
Inverse Autocorrelation IACF
For the model
(Yt  .)  !1 (yt1  .)  † † †  !p (Ytp  .) œ et  )1 et1  † † †  )q etq
define IACF as ACF of the dual model:
(Yt  .)  )1 (Yt1  .)  † † †  )q (Ytq  .) œ et  !1 et1  † † †  !p etp
IACF of AR(p) is ACF of a MA(p)
IACF of MA(q) is ACF of an AR(q)
How do we estimate ACF, IACF, PACF from data?
Autocovariances s
# (j) œ
ACF
nj


D (Yt  Y )(Ytj  Y )În
tœ1
3(j) œ s
# (j)Îs
# (0)
s
sj .
PACF plug s
# (j) into formal defining formula and solve for C
IACF: Approximate by fitting long autoregression
(Yt  .) œ !
s1 (Yt1  .)  † † †  !
sk (Ytk  .)  et
Compute ACF of dual model
^ 1 et1  † † †  !
^ k etk .
Yt  . œ et  !
To fit the long autoregressive plug s
# (j) into Yule-Walker equations for AR(k), or just
Applied Time Series Notes
( 22 )
regress Yj on Yt1 , Yt2 , â,Ytk .
All 3 functions IACF, PACF, ACF computed in PROC ARIMA. How do we interpret
them? Compare to catalog of theoretical IACF, PACF, ACF for AR, MA, and ARMA models.
See SAS System for Forecasting Time Series book for several examples - section 3.3.2.
Variance for IACF, PACF approximately
1
n
For ACF, SAS uses Bartlett's formula. For s
3(j) this is
n1
j1
2
D
3(i)‹
Šs
i=  j  1
(Fuller gives Bartlett's formula as 6.2.11 after first deriving a more accurate estimate of the
variance of s
3(i). The sum there is infinite so in SAS the hypothesis being tested is H0 :3(j)=0
assuming 3(i)=0 for i>j. Assuming a MA of order no more than j, is the j>2 autocorrelation 0?)
Syntax
PROC ARIMA;
IDENTIFY
ESTIMATE
FORECAST
VAR=Y
(NOPRINT NLAG=10
CENTER);
P=2
Q=1
(NOCONSTANT
NOPRINT
ML
PLOT);
LEAD=7
OUT=OUT1 ID=DATE INTERVAL=MONTH;
(1) I, E, F will work.
(2) Must have I preceding E, E preceding F

(3) CENTER subtracts Y
(4) NOCONSTANT is like NOINT in PROC REG
(5) ML (maximum likelihood) takes more time but has slightly better accuracy than
the default least squares.
(6) PLOT gives ACF, PACF, IACF of residuals.
Diagnostics: Box-Ljung chi-square on data Yt or residuals set .
(1) Compute estimated ACF s
3(j)
2
k
(2) Test is Q œ n(n  2) D Šs
3(j)‹ ‚(n  j)
jœ1
(3) Compare to ;2 distribution with k  p  q d.f. ŠARMA(p, q)‹
Applied Time Series Notes
( 23 )
ñ SAS (PROC ARIMA) will give Q test on original data and on residuals from fitted models.
ñ Q statistics given in sets of 6, i.e. for j=1 to 6, for j=1 to 12, for j=1 to 18, etc.
Note that these are cumulative
ñ For original series H0 : Series is white noise to start with.
ñ For residuals H0 : Residual series is white noise.
Suppose residuals autocorrelated - what does it mean?
Can predict future residuals from past - then why not do it?
Model predicts using correlation.
Autocorrelated residuals => model has not captured all the predictability in the data.
So.... H0 : Model is sufficient vs. H1 : Needs more work <=> "lack of fit" test
Let's try some examples. All have this kind of header, all have 1500 obs.
ARIMA Procedure
Name of variable = Y1.
Mean of working series = -0.03206
Standard deviation
= 1.726685
Number of observations =
1500
Applied Time Series Notes
Y1
( 24 )
Autocorrelations
Lag Covar
Corr
-1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
0 2.98144 1.00000
|
|********************|
1 2.39661 0.80384
|
2 1.89578 0.63586
|
. |*************
|
3 1.49191 0.50040
|
. |**********
|
4 1.20474 0.40408
|
. |********
|
5 1.00738 0.33788
|
. |*******
|
6
0.8373 0.28084
|
. |******
|
7 0.67985 0.22803
|
. |*****
|
8 0.58866 0.19744
|
. |****
|
.|****************
|
"." marks two standard errors
Inverse Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
-0.50067
|
**********|.
|
2
-0.00386
|
.|.
|
3
0.02033
|
.|.
|
4
0.01656
|
.|.
|
5
-0.01834
|
.|.
|
6
-0.02593
|
*|.
|
7
0.04455
|
.|*
|
8
-0.02228
|
.|.
|
Partial Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
0.80384
|
.|****************
|
2
-0.02912
|
*|.
|
3
-0.00625
|
.|.
|
4
0.02961
|
.|*
|
5
0.03108
|
.|*
|
6
-0.00511
|
.|.
|
7
-0.01304
|
.|.
|
8
0.03765
|
.|*
|
Autocorrelation Check for White Noise
To
Lag
Chi
Autocorrelations
Square DF
6 2493.02
6
Prob
0.000
0.804
0.636
0.500
0.404
0.338
0.281
Applied Time Series Notes
Y2
( 25 )
Autocorrelations
Lag Covar
Corr
-1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
0 2.84007 1.00000
|
1 -2.2489 -.79184
|
2 1.80586 0.63585
|
3 -1.4603 -.51416
|
4 1.14644 0.40367
|
5 -0.9219 -.32460
|
6 0.76776 0.27033
|
7 -0.6261 -.22044
|
8 0.48619 0.17119
|
|********************|
****************|.
. |*************
**********| .
. |********
******| .
. |*****
****| .
. |***
|
|
|
|
|
|
|
|
"." marks two standard errors
Inverse Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
0.47434
|
.|*********
|
2
0.00302
|
.|.
|
3
0.04342
|
.|*
|
4
0.03132
|
.|*
|
5
-0.01538
|
.|.
|
6
-0.01612
|
.|.
|
7
0.01805
|
.|.
|
8
0.01532
|
.|.
|
Partial Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
-0.79184
|
****************|.
|
2
0.02372
|
.|.
|
3
-0.01028
|
.|.
|
4
-0.03180
|
*|.
|
5
-0.01921
|
.|.
|
6
0.02570
|
.|*
|
7
0.00905
|
.|.
|
8
-0.02456
|
.|.
|
Autocorrelation Check for White Noise
To
Lag
Chi
Autocorrelations
Square DF
6 2462.73
6
Prob
0.000 -0.792
0.636 -0.514
0.404 -0.325
0.270
Applied Time Series Notes
Y3
( 26 )
Autocorrelations
Lag Covar
Corr
-1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
0 1.68768 1.00000
|
|********************|
1 0.87193 0.51664
|
.|**********
|
2 0.92573 0.54852
|
.|***********
|
3 0.60333 0.35749
|
. |*******
|
4 0.54891 0.32524
|
. |*******
|
5 0.43268 0.25637
|
. |*****
|
6 0.38316 0.22704
|
. |*****
|
7 0.28252 0.16740
|
. |***
|
8 0.26912 0.15946
|
. |***
|
"." marks two standard errors
Inverse Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
-0.17153
|
***|.
|
2
-0.32421
|
******|.
|
3
0.04133
|
.|*
|
4
0.02702
|
.|*
|
5
-0.03447
|
*|.
|
6
-0.02051
|
.|.
|
7
0.03163
|
.|*
|
8
-0.01685
|
.|.
|
Partial Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
0.51664
|
.|**********
|
2
0.38414
|
.|********
|
3
-0.02480
|
.|.
|
4
0.00940
|
.|.
|
5
0.03415
|
.|*
|
6
0.02322
|
.|.
|
7
-0.02602
|
*|.
|
8
0.02131
|
.|.
|
Autocorrelation Check for White Noise
To
Lag
Chi
Autocorrelations
Square DF
6 1382.13
6
Prob
0.000
0.517
0.549
0.357
0.325
0.256
0.227
Applied Time Series Notes
Y4
( 27 )
Autocorrelations
Lag Covar
Corr
-1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
0 1.87853 1.00000
|
|********************|
1 0.90481 0.48166
|
2 -0.3135 -.16687
|
***|.
|
3 -0.7114 -.37872
|
********|.
|
4 -0.3603 -.19181
|
****|.
|
5 0.10377 0.05524
|
.|*
|
6 0.24624 0.13108
|
.|***
|
7 0.13762 0.07326
|
.|*
|
8 0.05574 0.02967
|
.|*
|
.|**********
|
"." marks two standard errors
Inverse Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
-0.60608
|
************|.
|
2
0.27383
|
.|*****
|
3
0.00795
|
.|.
|
4
0.00599
|
.|.
|
5
-0.00347
|
.|.
|
6
-0.02802
|
*|.
|
7
0.03922
|
.|*
|
8
-0.03526
|
*|.
|
Partial Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
0.48166
|
.|**********
|
2
-0.51936
|
**********|.
|
3
-0.01149
|
.|.
|
4
0.00598
|
.|.
|
5
-0.00605
|
.|.
|
6
-0.01601
|
.|.
|
7
0.02135
|
.|.
|
8
0.06300
|
.|*
|
Autocorrelation Check for White Noise
To
Lag
6
Chi
Autocorrelations
Square DF
692.35
6
Prob
0.000
0.482 -0.167 -0.379 -0.192
0.055
0.131
Applied Time Series Notes
Y5
( 28 )
Autocorrelations
Lag Covar
Corr
-1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
0 1.77591 1.00000
|
|********************|
1 0.88862 0.50037
|
.|**********
|
2 -0.0056 -.00314
|
.|.
|
3
-0.074 -.04169
|
*|.
|
4 -0.0503 -.02831
|
*|.
|
5 0.03023 0.01702
|
.|.
|
6
0.0327 0.01841
|
.|.
|
7 0.00366 0.00206
|
.|.
|
8 0.06513 0.03667
|
.|*
|
"." marks two standard errors
Inverse Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
-0.77909
|
****************|.
|
2
0.58550
|
3
-0.42620
|
4
0.30980
|
5
-0.20537
|
6
0.10971
|
.|**
|
7
-0.04141
|
*|.
|
8
0.00364
|
.|.
|
.|************
*********|.
|
|
.|******
|
****|.
|
Partial Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
0.50037
|
.|**********
2
-0.33819
|
3
0.19526
|
4
-0.15162
|
5
0.14907
|
6
-0.11616
|
7
0.09129
|
.|**
|
8
-0.00938
|
.|.
|
*******|.
|
|
.|****
|
***|.
|
.|***
|
**|.
|
Autocorrelation Check for White Noise
To
Lag
6
Chi
Autocorrelations
Square DF
381.10
6
Prob
0.000
0.500 -0.003 -0.042 -0.028
0.017
0.018
Applied Time Series Notes
Y6
Lag Covar
0
( 29 )
Autocorrelations
Corr
-1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1.3101 1.00000
|
1 -0.1529 -.11669
|
**|.
|
2 -0.4136 -.31571
|
******|.
|
3 -0.0558 -.04262
|
*|.
|
4 -0.0349 -.02664
|
*|.
|
5
0.0462 0.03526
|
.|*
|
6 0.02675 0.02042
|
.|.
|
7 -0.0657 -.05012
|
*|.
|
8
|
.|*
|
0.0442 0.03374
|********************|
"." marks two standard errors
Inverse Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
0.46020
|
.|*********
|
2
0.52259
|
.|**********
|
3
0.33470
|
.|*******
|
4
0.28528
|
.|******
|
5
0.17483
|
.|***
|
6
0.11407
|
.|**
|
7
0.08309
|
.|**
|
8
0.01913
|
.|.
|
Partial Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
-0.11669
|
**|.
|
2
-0.33387
|
*******|.
|
3
-0.14914
|
***|.
|
4
-0.19317
|
****|.
|
5
-0.08641
|
**|.
|
6
-0.08499
|
**|.
|
7
-0.11042
|
**|.
|
8
-0.02902
|
*|.
|
Autocorrelation Check for White Noise
To
Lag
6
Chi
Autocorrelations
Square DF
176.67
6
Prob
0.000 -0.117 -0.316 -0.043 -0.027
0.035
0.020
Applied Time Series Notes
Y7
( 30 )
Autocorrelations
Lag Covar
Corr
-1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
0 1.05471 1.00000
|
|********************|
1 0.02858 0.02710
|
.|*
|
2 -0.0025 -.00234
|
.|.
|
3 -0.0332 -.03150
|
*|.
|
4 -0.0234 -.02215
|
.|.
|
5 0.01823 0.01729
|
.|.
|
6 0.02353 0.02231
|
.|.
|
7 -0.0265 -.02510
|
*|.
|
8 0.03498 0.03316
|
.|*
|
"." marks two standard errors
Inverse Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
-0.02879
|
*|.
|
2
-0.00014
|
.|.
|
3
0.03119
|
.|*
|
4
0.02154
|
.|.
|
5
-0.01921
|
.|.
|
6
-0.02185
|
.|.
|
7
0.02939
|
.|*
|
8
-0.03521
|
*|.
|
Partial Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
0.02710
|
.|*
|
2
-0.00307
|
.|.
|
3
-0.03137
|
*|.
|
4
-0.02049
|
.|.
|
5
0.01832
|
.|.
|
6
0.02035
|
.|.
|
7
-0.02759
|
*|.
|
8
0.03539
|
.|*
|
Autocorrelation Check for White Noise
To
Lag
6
Chi
Autocorrelations
Square DF
Prob
4.55
0.603
6
0.027 -0.002 -0.031 -0.022
0.017
0.022
Applied Time Series Notes
Y8
( 31 )
Autocorrelations
Lag Covar
Corr
-1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
0 2.75902 1.00000
|
|********************|
1 2.08823 0.75687
|
2 1.21753 0.44129
|
. |*********
|
3 0.67832 0.24585
|
. |*****
|
4 0.40167 0.14558
|
. |***
|
5 0.28744 0.10418
|
. |**
|
6 0.21441 0.07771
|
. |**
|
7 0.15825 0.05736
|
. |*.
|
8 0.15586 0.05649
|
. |*.
|
.|***************
|
"." marks two standard errors
Inverse Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
-0.67877
|
**************|.
|
2
0.26582
|
3
-0.09853
|
**|.
|
4
0.05250
|
.|*
|
5
-0.02564
|
*|.
|
6
-0.01058
|
.|.
|
7
0.02696
|
.|*
|
8
-0.01597
|
.|.
|
.|*****
|
Partial Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
0.75687
|
.|***************
2
-0.30802
|
3
0.10891
|
.|**
|
4
-0.01306
|
.|.
|
5
0.03785
|
.|*
|
6
-0.01800
|
.|.
|
7
0.01313
|
.|.
|
8
0.03612
|
.|*
|
******|.
|
|
Autocorrelation Check for White Noise
To
Lag
Chi
Autocorrelations
Square DF
6 1302.24
6
Prob
0.000
0.757
0.441
0.246
0.146
0.104
0.078
Applied Time Series Notes
( 32 )
Back to National League example:
Winning Percentage for National League Pennant Winner
Name of variable = WINPCT.
Mean of working series = 610.3699
Standard deviation
= 32.01905
Number of observations =
73
Autocorrelations
Lag Covariance Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
0
1025.219
1.00000
|
|********************|
1
446.181
0.43521
|
.
|*********
|
2
454.877
0.44369
|
.
|*********
|
3
212.299
0.20708
|
.
|**** .
|
4
145.290
0.14172
|
.
|***
.
|
5
154.803
0.15099
|
.
|***
.
|
6
92.698646
0.09042
|
.
|**
.
|
7
125.231
0.12215
|
.
|**
.
|
8
144.686
0.14113
|
.
|***
.
|
"." marks two standard errors
Inverse Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
-0.19575
|
.****|
.
|
2
-0.34856
|
*******|
.
|
3
0.11418
|
.
|**
.
|
4
0.08214
|
.
|**
.
|
5
-0.11465
|
.
.
|
6
0.04885
|
.
|*
.
|
7
0.03710
|
.
|*
.
|
8
-0.06110
|
.
.
|
**|
*|
Partial Autocorrelations
Lag Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
1
0.43521
|
.
|*********
|
2
0.31370
|
.
|******
|
3
-0.08479
|
.
**|
.
|
4
-0.05397
|
.
*|
.
|
5
0.11732
|
.
|**
.
|
6
0.00206
|
.
|
.
|
7
0.02711
|
.
|*
.
|
8
0.09302
|
.
|**
.
|
Applied Time Series Notes
( 33 )
Autocorrelation Check for White Noise
To
Chi
Lag
Square DF
6
Autocorrelations
37.03
Prob
6
0.000
0.435
0.444
0.207
0.142
0.151
0.090
Looks like AR(2) or MA(2) may fit well.
How to fit an MA ?
Data: 10 12 13 11 9 10 8 9 8 ( Mean = 10 )
Sum of squares for ) = -.5
Yt
yt = Yt -10
y^t = .5 et-1
et = yt -y^t
10
0
0
0
12
2
0
2
13
3
1
2
Yt = et - )et-1
11
1
1
0
9
-1
0
-1
10
0
-0.5
0.5
8
-2
0.25
-2.25
9
-1
-1.125
0.125
8
-2
0.0625
-2.0625
sum of squared errors = 02 +22 + ... + 2.06252 = 18.582.
)
SS(err)
.1
26.7
0
24
- .1
21.9
- .2
20.3
- .3
19.2
- .4
18.64
- .5
18.58
- .6
19.2
-.7
20.6
so )^ ¸ -.5
A better way:
Make
`
` ) SSq(
^
)) = 2!et ()^)
^
`
` ) et () )
=0
How? If et ()^) is a residual from a regression on
^
`
` ) et ())
then
derivative is 0 by orthogonality of residuals to regressors.
Taylor's Series:
et ( ) ) = et ( )^ ) +
^
`
` ) et () )
( ) - )^ ) + remainder
Ignore remainder and evaluate at et (true )0 ) = white noise
et ( )^ ) ¸ -
^
`
` ) et () )
( )0 - )^ ) + et ( )0 )
Can calculate et ( )^ ) and - ``) et ()^), error term is white noise!
Estimate ( )0 - )^ ) by regression and iterate to convergence.
Applied Time Series Notes
( 34 )
Also: Can show regression standard errors justified in large samples.
1. et ( )^ ) = Yt + )^ et-1 ( )^ ) for initial )^
2.
^
`
` ) et () )
= et-1 ( )^ ) + ()^)
^
`
` ) et-1 () )
3. Regress sequence (1) on sequence (2).
Data MA;
* begin Hartley modification ;
theta = -.2 -.447966+.3168 - .6*.244376;
call symput('tht', put(theta,8.5));
title "Using theta = &tht " ;
if _n_ = 1 then do; e1=0; w1=0; end;
input y @@;
e = y + theta*e1;
w = -e1 + theta*w1;
output; retain; e1=e; w1=w;
cards;
0 2 3 1 -1
0
-2 -1 -2
;
proc print noobs; var y e e1 w w1;
proc reg; model e = w / noint;
run;
-----------------------------------------------------------------------------------Using theta = -0.47779
Y
E
E1
W
W1
0
0.00000
0.00000
0.00000
0.00000
2
2.00000
0.00000
0.00000
0.00000
3
2.04442
2.00000
-2.00000
0.00000
1
0.02319
2.04442
-1.08883
-2.00000
-1
-1.01108
0.02319
0.49704
-1.08883
0
0.48309
-1.01108
0.77360
0.49704
-2
-2.23081
0.48309
-0.85271
0.77360
-1
0.06586
-2.23081
2.63823
-0.85271
-2
-2.03147
0.06586
-1.32639
2.63823
Parameter Estimates
Variable
W
Parameter
Standard
T for H0:
DF
Estimate
Error
Parameter=0
Prob > |T|
1
0.034087
0.38680072
0.088
0.9319
Applied Time Series Notes
( 35 )
Another way to estimate ARMA models is EXACT MAXIMUM LIKELIHOOD
Gonzalez-Farias' dissertation uses this methodology for nonstationary series.
AR(1): ( Yt - . ) = 3( Yt-1 - . ) + et
Y1 - . ~ N( 0 , 52 /(1-32 ) )
( Yt - . ) - 3( Yt-1 - . ) ~ N( 0 , 52 ), t=2,3,, ...,n
Likelihood:
È1-32
” 5È21
e
_ =
- "# (Y1
2
2
- . ) (1-3 )/5
2
•”Š
1
5È21
‹
- "# ![(Yt - . ) - 3 (Yt-1 - . ) ]2 /52
n
n-1
e
t=2
•
Positive in (-1,1) and 0 at +1, -1 => easy to maximize.
Logarithms:
ln(_) = "# ln (1-32 ) - n2 ln [ 21 s2 (3) ] - "# n
where s2 (3) = SSq / n and SSq = (Y1 - . )2 (1-32 ) + ![(Yt - . ) - 3 (Yt-1 - . ) ]2
n
(Y1 +Yn )+(1- 3) !Yt
t=2
n-1
.=.(3) =
t=2
2+(n-2)(1- 3)
If |3|<1 then choosing 3 to maximize ln(_) does not differ in the limit from choosing 3
to minimize ![(Yt - . ) - 3 (Yt - . ) ]2 .
n
t=2
(least squares and maximum likelihood are about the same for large samples OLS ¸ MLE).
Gonzalez-Farias shows MLE and OLS differ in a nontrivial way, even in the limit, when 3=1.
Applied Time Series Notes
Example of MLE for Iron and Steel Exports data:
( 36 )
DATA STEEL STEEL2;
ARRAY Y(44); n=44; pi = 4*atan(1);
do t=1 to n; input EXPORT @@;OUTPUT STEEL2; Y(t)=EXPORT; end;
Do RHO = .44 to .51 by .01;
MU = (Y(1) + Y(n) + (1-rho)*sum(of Y2-Y43) )/(2+(1-rho)*42);
SSq = (1-rho**2)*(Y(1)-mu)**2;
Do t=2 to n;
SSq = SSq + (Y(t)-mu
- rho*(Y(t-1)-mu) )**2;
end;
lnL = .5*log(1-rho*rho) - (n/2)*log(2*pi*SSq/n) - n/2;
output Steel; end; drop y1-y44;
CARDS;
3.89 2.41 2.80 8.72 7.12 7.24 7.15 6.05 5.21 5.03 6.88
4.70 5.06 3.16 3.62 4.55 2.43 3.16 4.55 5.17 6.95 3.46
2.13 3.47 2.79 2.52 2.80 4.04 3.08 2.28 2.17 2.78 5.94
8.14 3.55 3.61 5.06 7.13 4.15 3.86 3.22 3.50 3.76 5.11
;
proc arima data=steel2; I var=export noprint; e p=1 ml;
proc plot data=steel; plot lnL*rho/vpos=20 hpos=40;
title "Log likelihood for Iron Exports data";
proc print data=steel;
run;
Log likelihood for Iron Exports data
ARIMA Procedure
Maximum Likelihood Estimation
Approx.
Parameter
Estimate
Std Error
T Ratio
Lag
MU
4.42129
0.43002
10.28
0
AR1,1
0.46415
0.13579
3.42
1
Applied Time Series Notes
Plot of lnL*RHO.
( 37 )
Legend: A = 1 obs, B = 2 obs, etc.
lnL ‚
‚
-81.18 ˆ
‚
A
A
‚
A
‚
A
-81.20 ˆ
‚ A
A
‚
‚
-81.22 ˆ
‚
A
‚
‚
-81.24 ˆ
‚
A
‚
‚
-81.26 ˆ
‚
Šƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒ
0.44 0.45 0.46 0.47 0.48 0.49 0.50 0.51
RHO
IRON AND STEEL EXPORTS EXCLUDING SCRAPS
WEIGHT IN MILLION TONS
1937-1980
OBS
N
PI
T
EXPORT
RHO
MU
SSQ
LNL
1
44
3.14159
45
5.11
0.44
4.42100
102.765
-81.2026
2
44
3.14159
45
5.11
0.45
4.42112
102.687
-81.1914
3
44
3.14159
45
5.11
0.46
4.42123
102.636
-81.1861
4
44
3.14159
45
5.11
0.47
4.42135
102.610
-81.1866
5
44
3.14159
45
5.11
0.48
4.42148
102.611
-81.1929
6
44
3.14159
45
5.11
0.49
4.42161
102.638
-81.2051
7
44
3.14159
45
5.11
0.50
4.42174
102.692
-81.2231
8
44
3.14159
45
5.11
0.51
4.42188
102.772
-81.2469