Homework 3
1
Data analysis problems
For the homework, be sure to give full explanations where required and to
turn in any relevant plots.
1. Explore the behavior of the ACF and PACF. Simulate a single path
of 10,000 observations for each of the following models.
(a) xt = 0.6xt−1 − 0.8xt−2 + wt
(b) xt = wt + 0.8wt−1 + 1.1wt−2
(c) xt = 0.8xt−1 + wt + 0.8wt−1
Plot the ACF and PACF for each path. Discuss what you see in the
plots and if it is what you would expect.
[Sol] The plots and discussion would be as follows:
(a) Since this is an AR(2), we expect an ACF that falls off
exponentially and a PACF which falls off quickly after
the second lag. This is seen in the plots.
(b) Since this is an MA(2), we expect an ACF that falls off
after a lag of two and a PACF which falls off exponentially. This is what is observed.
(c) This is an ARMA(1,1), we expect the PACF and the
ACF to fall off exponentially.
1
Figure 1: 1-(a) ACF and PACF of AR(2)
-0.5
ACF
0.0
0.5
1.0
Series: ar
10
20
LAG
30
40
0
10
20
LAG
30
40
-0.5
PACF
0.0
0.5
1.0
0
Figure 2: 1-(b) ACF and PACF of MA(2)
-0.4
0.0
ACF
0.4
0.8
Series: ma
10
20
LAG
30
40
0
10
20
LAG
30
40
-0.4
0.0
PACF
0.4
0.8
0
2
Figure 3: 1-(b) ACF and PACF of ARMA(1,1)
-0.5
0.0
ACF
0.5
1.0
Series: arma
10
20
LAG
30
40
0
10
20
LAG
30
40
-0.5
0.0
PACF
0.5
1.0
0
3
2. Load the Berkeley and Santa Barbara temperature data using the
following commands
berk=scan("berkeley.dat", what=list(double(0),double(0),double(0)))
time=berk[[1]]
berkeley=berk[[2]]
stbarb=berk[[3]]
See if berkeley and stbarb are stationary. If not, fix the nonstationarity with the first order differencing. Do you have an opinion on
possible models based only on these plots?
[Sol] The plots are seen below. For berkeley and stbarb, in
both plots (left), there is a trend in the series and the mean is
not constant. It means that the data is not very stationary in
appearance. So we need to take the first order differencing.
From the plots below (right), it appears that both differenced
series have constant means.
0.5
0.0
dberk
-1.0
13.5
12.5
berkeley
14.5
1.0
1.5
Figure 4: 2 - time series plots for berkeley and stbarb (left), and their
differenced series (right)
0
20
40
60
80
100
0
20
40
60
80
100
80
100
Time
1.0
0.0
dstbarb
16
-1.0
15
14
stbarb
17
18
Time
0
20
40
60
80
100
0
Time
20
40
60
Time
4
Figure 5: 2 - ACF and PACF of berkeley
-0.2
0.2
ACF
0.6
1.0
Series: berkeley
10
5
10
LAG
15
20
15
20
-0.2
0.2
PACF
0.6
1.0
5
LAG
Figure 6: 2 - ACF and PACF of stbarb
-0.2
0.2
ACF
0.6
1.0
Series: stbarb
10
5
10
LAG
15
20
15
20
-0.2
0.2
PACF
0.6
1.0
5
5
LAG
For the differenced data dberk=diff(berkeley) and dstbarb=diff(stbarb),
see Figures below. Since ACF cuts off at lag 1 and PACF
tails off, so a possible model for dberkeley is MA(1); similarly, a possible model for dstbarb is MA(1) (Other models
can also be correct and your proposed model need to be well
defensed.)
Figure 7: 2 - ACF and PACF of differenced berkeley
-0.5
0.0
ACF
0.5
1.0
Series: dberk
10
5
10
LAG
15
20
15
20
-0.5
0.0
PACF
0.5
1.0
5
LAG
Figure 8: 2 - ACF and PACF of differenced berkeley
-0.5
0.0
ACF
0.5
1.0
Series: dstbarb
10
5
10
LAG
15
20
15
20
-0.5
0.0
PACF
0.5
1.0
5
LAG
6
3. Load the global average temperature using the command
temperature=scan("globtemp.dat")
See if temperature is stationary. If not, fix the nonstationarity with
the first order differencing. Do you have an opinion on possible models
based on these plots?
[Sol] Similar to the previous exercise, from the plots it appears to be non-stationary for the original series. Differencing in needed here. For the differenced data,ACF cuts off at
lag 2 and PACF tails off. So a possible candidate model is
MA(2).
(You need to defense your model in order to get a full credit.)
0.1
-0.3
-0.1
dtemp
0.2
0.0
-0.4 -0.2
temperature
0.4
0.3
Figure 9: 3 - time series plots for temperature (left), and its differenced
series (right)
0 20
60
100
140
0 20
Time
60
Time
7
100
140
Figure 10: 3 - ACF and PACF of temperature
-0.2
0.2
ACF
0.6
1.0
Series: temperature
10
5
10
LAG
15
20
15
20
-0.2
0.2
PACF
0.6
1.0
5
LAG
Figure 11: 3 - ACF and PACF of differenced temperature
-0.2
ACF
0.2
0.6
1.0
Series: dtemp
10
5
10
LAG
15
20
15
20
-0.2
PACF
0.2
0.6
1.0
5
LAG
8
4. Verify Example 3.5 (pages 91-92) from the textbook. Simulate the two
MA models of length 10,000 using your newly created function masim
and the two sets of parameters in this example. Now, fit those two
MA models using the command arima(x,c(0,0,1)) where x is the
name of your time series. (We’ll talk more about this command later)
Does this confirm what was shown in theory?
[Sol] For this problem, they should obtain something like
> x1=masim(c(5),1,10000)
>
> x2=masim(c(1/5),25,10000)
>
> arima(x1, c(0,0,1))
Call:
arima(x = x1, order = c(0, 0, 1))
Coefficients:
ma1 intercept
0.2095
0.0378
s.e. 0.0101
0.0600
sigma^2 estimated as 24.60:
>
> arima(x2, c(0,0,1))
log likelihood = -30202.29,
aic = 60410.59
Call:
arima(x = x2, order = c(0, 0, 1))
Coefficients:
ma1 intercept
0.2122
0.0117
s.e. 0.0099
0.0608
sigma^2 estimated as 25.2:
>
log likelihood = -30323.32,
aic = 60652.65
The important thing to notice was that while the parameters
used to simulate are quite different R fits the same model. R
finds the invertible parameters.
9
2
Theoretical problems
1. Check if there are any parameter redundancies. It there are, then
remove it. Determine whether the removed models are causal and/or
invertible.(Refer to the example 3.7 in the textbook)
(a) xt = 0.6xt−1 − 0.08xt−2 + wt − 0.7wt−1 + 0.1wt−2
[Sol] The ar polynomial is φ(z) = (1 − 0.2z)(1 − 0.4z) and
the ma polynomial is θ(z) = (1 − 0.2z)(1 − 0.5z). Since the
ar and ma polynomials for this model share a common
root 5, there is a parameter redundancy. After removing
it, the root of ar polynomial is greater than 1 and the
root of the ma polynomial is also greater than 1. So the
removed model is both causal and invertible.
(b) xt = −0.2xt−1 + 0.15xt−2 + wt + 2.5wt−1 + wt−2
[Sol] The ar polynomial is φ(z) = (1 − 0.3z)(1 + 0.5z) and
the ma polynomial is θ(z) = (1 + 2z)(1 + 0.5z). Since the ar
and ma polynomials for this model share a common root
-2, there is a parameter redundancy. After removing it,
the root of ar polynomial is greater than 1 and the root
of the ma polynomial is smaller than 1. So the removed
model is causal but not invertible.
(c) xt = xt−1 + wt − 0.5wt−1 − 0.5wt−2
[Sol] The AR polynomial is φ(z) = 1 − z, which has root 1.
the MA polynomial is θ(z) = 1−z/2−z 2 /2, which has roots
-2 and 1. since these polynomials share a common root,
they have the common factor 1 − z. Factoring these out,
the reduced representation has AR polynomial φ(z) =
1(which has no roots) and MA polynomial θ(z) = 1 + z/2
(which has root -2). thus, this is an ARMA(0,1) process
(or MA(1) process) which is causal and invertible.
(d) xt = 3xt−1 + wt + 2wt−1 − 8wt−2
[Sol] The AR polynomial is φ(z) = 1 − 3z, which has root
1/3. the MA polynomial is θ(z) = 1 + 2z − 8z 2 , which has
roots -1/4 and 1/2. Thus, this is an ARMA(1,2) process
which is neither causal nor invertible.
2 ),
2. Verify that for an MA(1) process, xt = wt +θ1 wt−1 with wt ∼ wn(0, σw
max
−∞<θ1 <∞
ρ(1) = 0.5,
10
min
−∞<θ1 <∞
ρ(1) = 0.5
(Hint : calculate the autocorrelation at lag 1, ρ(1), and take the first
derivative of ρ(1) with respect to θ1 to see the maximum and minimum).
−θ2 +1
∂ρ(1)
θ1
[Sol] Recall that ρ(1) = 1+θ
= (1+θ1 2 )2 which is seen
2 so that ∂θ
1
1
1
to be positive on the range −1 < θ1 < 1 and zero at both -1 and
+1. For | θ1 |> 1, the derivative is negative. Taken together,
these facts imply the desired results. Or you can show that
the second derivative evaluated at θ1 = −1 is positive and the
second derivative evaluated at θ1 = 1 is negative.
3. For the ARMA(1,2) process, xt = 0.8xt−1 +wt +0.7wt−1 +0.6wt−2 with
2 ), show that ρ(k) = 0.8ρ(k − 1) for k > 2 under the aswt ∼ wn(0, σw
sumption that the mean of the series is zero (Hint : it is enough to show
γ(k) = 0.8γ(k − 1) for k > 2 and use E(xt−1 xt−k ) = E(xt xt−(k−1) ))
[Sol]
cov(xt , xt−k ) = E[(0.8xt−1 + wt + 0.7wt−1 + 0.6wt−2 )xt−k ]
= 0.8E(xt−1 xt−k ) + E(wt xt−k ) + 0.7E(wt−1 xt−k ) + 0.6E(wt−2 xt−k )
= 0.8E(xt−1 xt−k ) because k > 2
= 0.8E(xt xt−(k−1) ) = 0.8cov(xt , xt−(k−1) ).
Thus, γ(k) = 0.8γ(k − 1) and ρ(k) = 0.8ρ(k − 1).
2 ).
4. Consider the AR(1) process xt = φ1 xt−1 + wt with wt ∼ wn(0, σw
Show that if | φ1 |= 1, the process cannot be stationary.
[Sol] Assume that {xt } is stationary. Then var(xt ) = φ21 var(xt−1 )+
2
2 or σ 2 = σw . If | φ |= 1, this is
var(wt ) and σy2 = φ21 σy2 + σw
1
y
1−φ21
impossible and we have a proof by contradiction.
5. Consider a process that satisfies the zero-mean, “stationary” AR(1)
2 ) and | φ |< 1. Let c
equation xt = φ1 xt−1 + wt with wt ∼ wn(0, σw
1
be any nonzero constant, and define yt = xt + cφt1 .
1) Show that E(yt ) = cφt1 .
[Sol] E(yt ) = E(xt + cφt1 ) = E(xt ) + cφt1 = cφt1 .
2) Is {yt } stationary?
11
[Sol] Since E(yt ) = cφt1 is not constant in time, the solution is
not stationary.
3) Show that {yt } satisfies the “stationary” AR(1) equation yt =
φ1 yt−1 + wt .
[Sol] yt = xt + cφt1 = φ1 xt−1 + wt + cφt1 = φ1 (xt−1 + cφt−1
1 ) + wt =
φ1 yt−1 + wt .
12