Lecture 3: ARIMA(p,d,q) models
Florian Pelgrin
University of Lausanne, École des HEC
Department of mathematics (IMEA-Nice)
Sept. 2011 - Jan. 2012
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
1 / 17
Introduction
Motivation
Characterize the main properties of ARIMA(p) models.
Discuss asymptotic equivalence with ARMA(p,q) models
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
2 / 17
Introduction
Road map
1
Introduction
2
ARIMA(p,d,q) model
Definition
Fundamental representations
Equivalence with ARMA(p,q) models
Autoregressive approximation
Moving average approximation
3
Application
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
3 / 17
ARIMA(p,d,q) model
Definition
1. Definition
To some extent, ARIMA(p,d,q) models are a generalization of
ARMA(p,q) models : the d-differenced process ∆d Xt is
(asymptotically) an ARMA(p,q) process :
On the other hand, the statistical properties of the two models are
different, especially in terms of forecasting.
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
4 / 17
ARIMA(p,d,q) model
Definition
Definition
A stochastic process (Xt )t≥−p−d is said to be an ARIMA(p, d, q)—an
integrated mixture autoregressive moving average model—if it satisfies the
following equation :
Φ(L)(1 − L)d Xt = µ + Θ(L)t ∀t ≥ 0
where t is a (weak) white noise process with variance σ2 , the lag
polynomials are given by :
Φ(L) = 1 − φ1 L − · · · − φp Lp with φp 6= 0
Θ(L) = 1 + θ1 L + · · · + θq Lq with θq 6= 0,
and the initial conditions :
Z−1 = {X−1 , · · · , X−p−d , −1 , · · · , −q }
are such that :
Florian Pelgrin (HEC)
Cov ( , ZUnivariate
) =time
0 series
∀t ≥ 0.
Sept. 2011 - Jan. 2012
5 / 17
ARIMA(p,d,q) model
Definition
Remarks :
1. The stochastic process Xt also writes :
Φ(L)∆d Xt = µ + Θ(L)t or Φ(L)Yt = µ + Θ(L)t
where Yt = ∆d Xt = (1 − L)d Xt .
2. Initial conditions are fundamental.
Example : Consider the two stochastic processes :
Xt
= ρXt−1 + t
Yt
≡ Xt − Xt−1 = ηt
where ρ = 0.9, t and ηt are (weak) white noises.
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
6 / 17
ARIMA(p,d,q) model
Definition
AR(1) model
Random walk (without drift)
25
12
20
8
15
4
10
5
0
0
-4
-5
-10
-8
10 20 30 40 50 60 70 80 90 100
10 20 30 40 50 60 70 80 90 100
Note: The black, blue, and red solid lines correspond respectively to X_0 = 0, X_0 = 2, and X_0 = -2.
Figure: Initial conditions and stationarity
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
7 / 17
ARIMA(p,d,q) model
Fundamental representations
1.2. Fundamental representations
Definition (Fundamental representation)
Let (Xt )t≥−p−d denote the following ARIMA(p,d,q) stochastic process :
Φ(L)∆d Xt
= µ + Θ(L)t ∀t ≥ 0
where Φ(L) = 1 − φ1 L − · · · − φp Lp and Θ(L) = 1 + θ1 L + · · · + θq Lq ,
θq 6= 0,φp 6= 0, µ is a constant term, (t ) is a weak white noise, and the
initial conditions are uncorrelated with t (t ≥ 0). This representation is
said to be causal or fundamental if and only if :
(i) All of the roots of the (inverse) characteristic equation associated to
Φ are of modulus less (larger) than one.
(ii) All of the roots of the (inverse) characteristic equation associated to
Θ are of modulus less (larger) than one.
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
8 / 17
ARIMA(p,d,q) model
Fundamental representations
Definition (Fundamental minimal representation)
Let (Xt )t≥−p−d denote the following ARIMA(p,d,q) stochastic process :
Φ(L)∆d Xt
= µ + Θ(L)t ∀t ≥ 0
where Φ(L) = 1 − φ1 L − · · · − φp Lp and Θ(L) = 1 + θ1 L + · · · + θq Lq ,
θq 6= 0,φp 6= 0, µ is a constant term, (t ) is a weak white noise, and the
initial conditions are uncorrelated with t (t ≥ 0). This representation is
said to be causal or fundamental if and only if :
(i) All of the roots of the (inverse) characteristic equation associated to
Φ are of modulus less (larger) than one.
(ii) All of the roots of the (inverse) characteristic equation associated to
Θ are of modulus less (larger) than one.
(iii) The (inverse) characteristic polynomials have no common roots.
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
9 / 17
ARIMA(p,d,q) model
Equivalence with ARMA(p,q) models
1.3. Equivalence with ARMA(p,q) models
Proposition
Let (Xt )t≥−p−d denote a minimal and causal ARIMA(p, d, q) stochastic
process :
Φ(L)(1 − L)d Xt = µ + Θ(L)t .
The stochastic process defined by :
Yt = ∆d Xt = (1 − L)d Xt
is asymptotically equivalent to an ARMA(p, q) process.
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
10 / 17
ARIMA(p,d,q) model
Autoregressive approximation
1.4. Autoregressive approximation
Definition
The autoregressive approximation (and not the AR(∞) representation) of
a causal and minimal ARIMA(p,d,q) stochastic process is given by :
At (L)Xt = µ0 + t + h(t)0 Z−1 .
where At (L) =
t
P
aj Lj , a0 = 1, and the aj terms are the coefficients of the
j=0
division (by increasing powers) of Φ(u) by Θ(u), µ0 is a constant term,
and h(t) ∈ Rp+d+q (with lim h(t) = 0).
t→+∞
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
11 / 17
ARIMA(p,d,q) model
Moving average approximation
1.5. Moving average approximation
Definition
The moving average approximation (and not the MA(∞) representation)
of a causal and minimal ARIMA(p,d,q) stochastic process is given by :
Xt = µ1 + Bt (L)t + h̃(t)0 Z−1 .
where Bt (L) =
t
P
bj Lj , b0 = 1, and the bj terms are the coefficients of the
j=0
division (by increasing powers) of Θ(u) by Φ(u), µ1 is a constant term,
and h̃(t) ∈ Rp+d+q (with lim h̃(t) = 0).
t→+∞
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
12 / 17
ARIMA(p,d,q) model
Moving average approximation
Example
Starting from :
(1 − φL)(1 − L)Xt = t − θt−1 ∀t ≥ 0
with the initial conditions −1 , X−1 and X−2 ;
One gets :
∆Xt
=
=
Florian Pelgrin (HEC)
1 − θL
˜t − φt θ−1 + φt+1 (X−1 − X−2 )
1 − φL
1 − θL
˜t + ht0 Z−1 .
1 − φL
Univariate time series
Sept. 2011 - Jan. 2012
13 / 17
Application
2. Application : modelling of the US stock market
dividends
Visual inspection of the autocorrelation function may indicate that
the (log) US dividend is not stationary. The autocorrelogram of the
first-differenced variable decreases quickly to zero—the first-difference
variable may be stationary.
Unit root tests tends to favor the assumption of non-stationarity (see
later on).
The chosen specification is then an ARIMA(p,1,q) model :
Φ(L)(1 − L)dt = Θ(L)t
where t is a white noise process, and :
Φ(L) = 1 − φ1 L − · · · − φp Lp
Θ(L) = 1 + θ1 L + · · · + θq Lq .
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
14 / 17
Application
Figure 34: (Partial) autocorrelogram of (log) US stock market dividents
Level
Florian Pelgrin (HEC)
First-difference
Univariate time series
Sept. 2011 - Jan. 2012
15 / 17
Application
Estimation of an ARIMA(2,1,1) model
Parameter Estimate Std. Error t-stat. p-value
c
0.0007
0.0003
1.8777 0.0611
φ1
0.7342
0.1096
6.6982 0.0000
φ2
0.1206
0.0619
1.9477 0.0521
-0.6442
0.1038
-6.2008 0.0000
θ1
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
16 / 17
Application
Figure 35: Adjusted error terms
.04
.02
.00
-.02
-.04
-.06
-.08
-.10
-.12
1975 1980 1985 1990 1995 2000 2005
Florian Pelgrin (HEC)
Univariate time series
Sept. 2011 - Jan. 2012
17 / 17