Statistical Models
“The primary objective of time series analysis is to develop mathematical models that provide plausible descriptions for sample data. . . ”
• We model a time series as a collection of random variables:
x1, x2, x3, . . . , or more generally {xt, t ∈ T }.
• Often the phenomenon being observed evolves in continuous
time, but our observations are always discrete samples.
1
• If the sampling times t1, t2, . . . are equally spaced, their separation δt = tn − tn−1 is the sampling interval and 1/δt is the
sampling rate (samples per unit time).
• Choice of sampling rate affects all aspects of data collection,
analysis, and interpretation.
2
Example: White Noise
• Uncorrelated random variables wt with mean 0 and variance
2 , written w ∼ wn(0, σ 2 ).
σw
t
w
– Why white noise?
– By analogy with white light: in the frequency domain, all
frequencies are present with the same strength.
• If in addition the w’s are independent and identically dis2 ).
tributed, we write wt ∼ iid(0, σw
3
Iid White Noise
−10
−5
w
0
5
# t-distributed with 3 degrees of freedom:
w = ts(rt(500, df = 3));
plot(w);
0
100
200
300
400
500
Time
4
• If in addition the w’s are normally distributed, we write
2 ).
wt ∼ iid N(0, σw
5
Iid Normal White Noise
0
−1
−2
w
1
2
w = ts(rnorm(500));
plot(w);
0
100
200
300
400
500
Time
6
Example: Moving Average
• Many observed series are smoother than white noise.
• Possible model:
1
wt−1 + wt + wt+1
vt =
3
7
Moving Average
−2
−1
v
0
1
w = ts(rnorm(500));
v = filter(w, sides = 2, rep(1, 3) / 3);
plot(v);
0
100
200
300
400
500
Time
8
• Averaging attenuates the faster oscillations, leaving the slower
oscillations more apparent.
• More generally, a weighted average of 2, 3, or more noise
terms.
9
Example: Autoregression
• Recursive model:
xt = xt−1 − 0.9xt−2 + wt,
t = 1, 2, . . . , 500
• Like a regression equation, but the RHS contains past (“lagged”)
LHS variables, hence autoregression.
• Shows many different types of behavior for different choices
of coefficients.
10
Autoregression
−5
0
v
5
w = ts(rnorm(500));
v = filter(w, filter = c(1, -0.9), method = "recursive");
plot(v);
0
100
200
300
400
500
Time
11
Example: Random Walk
• One model for trend; recursive definition:
xt = δ + xt−1 + wt
• Explicitly:
xt = δt +
t
X
wj
j=1
• δ is the drift (per unit time).
12
Random Walk
0
20
40
x
60
80
# drift delta = 0.2 per sample:
x = ts(cumsum(rnorm(500) + 0.2));
plot(x);
0
100
200
300
400
500
Time
13
• The white noise we build it from could be non-normal.
14
Non-Normal Random Walk
−250
−200
−150
x
−100
−50
0
# t-distributed increments, 1 degree of freedom, no drift:
x = ts(cumsum(rt(500, df = 1)));
plot(x);
0
100
200
300
400
500
Time
15
Example: Signal in Noise
• Sine-wave signal:
xt = 2 cos(2πt/50 + 0.6π) + wt,
t = 1, 2, . . . , 500
• More generally, the wave term could be
A cos(2πωt + φ),
where:
– A is amplitude;
– ω is frequency (in cycles per unit time);
– φ is phase (in this case, in radians).
16
Cosine wave signal plus noise
0
−2
−4
x
2
4
w = ts(rnorm(500));
x = 2 * cos(2 * pi * time(w) / 50 + 0.6 * pi) + w;
plot(x);
0
100
200
300
400
500
Time
17