Spectral Density
Spectral Density (Chapter 12)
Periodogram
Outline
1
Spectral Density
2
Periodogram
Arthur Berg
Spectral Density
Periodogram
Spectral Approximation
xt =
Spectral Density
xt = U cos(2πω0 t) + V sin(2πω0 t)
k =1
where Uk and Vk are independent zero-mean random variables with
variances σk2 at distinct frequencies ωk .
The autocovariance function of xt is (HW 4c Exercise)
q
X
where U, V are independent zero-mean random variables with equal
variances σ 2 . We shall assume ω0 > 0, and from the Nyquist rate, we
can further assume ω0 < 1/2.
The “period” of xt is 1/ω0 , i.e. the process makes ω0 cycles going from xt
to xt+1 .
From the formula
e−iα + e−iα
cos(α) =
2
we have
σk2 cos(2πωk h)
k =1
In particular
γ(0) =
|{z}
var(xt )
Periodogram
Let’s consider the stationary time series
Uk cos(2πωk t) + Vk sin(2πωk t)
γ(h) =
q
X
σk2
k =1
| {z }
sum of variances
Note: γ(h) is not absolutely summable,
i.e.
∞
X
|γ(h)| = ∞
Arthur Berg
h=−∞
2/ 19
Motivating Example
Any stationary time series has the approximation
q
X
Spectral Density (Chapter 12)
Spectral Density (Chapter 12)
γ(h) = σ 2 cos(2πω0 h) =
4/ 19
Arthur Berg
σ 2 −2πω0 h
e
+ 22πiω0 h
2
Spectral Density (Chapter 12)
5/ 19
Spectral Density
Periodogram
Riemann-Stieltjes Integration
Spectral Density
Periodogram
Integration w.r.t. a Step Function
Riemann-Stieltjes integration enables one to integrate with respect to
a general nondecreasing function.
Usual
R Integration
g(x) dx
General
Situation
R
g(x) dF (x)
Given F (x) =
Case of interest: F (x) is a step function, i.e. a function with a finite
number of jump discontinuities. Such an F (x) has the representation
F (x) =
n
X
P
αi 1(xi ≤ x), then
Z
X
g(x) dF (x) =
αi g(xi )
αi 1(xi ≤ x)
i=1
So F (x) has jumps at xi with heights αi .
(Simplest step function is the Heaviside step function given by
H(x) = 1(x > 0).)
Arthur Berg
Spectral Density (Chapter 12)
Spectral Density
6/ 19
Periodogram
Back to our Motivating Example
Arthur Berg
Spectral Density (Chapter 12)
Spectral Density
7/ 19
Periodogram
General Theorem
We can now write the autocovariance function of
xt = U cos(2πω0 t) + V sin(2πω0 t)
The autocovariance function for any stationary time series has the
representation
Z 1/2
γ(h) =
e2πiωh dF (ω)
as
σ 2 −2πω0 h
e
+ 22πiω0 h
2
Z 1/2
=
e2πiωh dF (ω)
γ(h) =
−1/2
for a unique monotonically increasing function F (ω) such that
−1/2
where
F (−∞) = F (−1/2) = 0
and
F (∞) = F (1/2) = σ 2 .
The function F (ω) behaves like a CDF for a discrete random variable,
except that F (∞) = σ 2 = γ(0). More on this later.


0,
ω < −ω0
2
σ
F (ω) = 2 , −ω0 ≤ ω < ω0

 2
σ , ω ≥ ω0
F (ω) jumps by σ 2 /2 at −ω0 and ω0 .
Arthur Berg
Spectral Density (Chapter 12)
8/ 19
Arthur Berg
Spectral Density (Chapter 12)
9/ 19
Spectral Density
Periodogram
Approximating Step Functions
Spectral Density
Periodogram
Spectral Density
Suppose the autocovariance function of a stationary process satisfies
Any step function can easily be approximated by an absolutely
continuous function.
∞
X
|γ(h)| < ∞
(?)
h=−∞
then autocovariance function has the representation
Z 1/2
γ(h) =
eiωh f (ω),
h = 0, ±1, ±2, . . .
−1/2
where f (ω) is the spectral density. Similarly,
f (ω) =
∞
X
γ(h)e−iωh ,
−π ≤ ω ≤ π
h=−∞
For an absolutely continuous spectral distribution function, a spectral
density exists.
Arthur Berg
Spectral Density (Chapter 12)
Spectral Density
Note: (?) is sufficient, but not necessary, to guarantee a spectral density.
The functions γ(h) and f (ω) form a Fourier transform pair.
10/ 19
Periodogram
Spectral Density of an ARMA(p, q) Process
Arthur Berg
Spectral Density (Chapter 12)
Spectral Density
11/ 19
Periodogram
Example — Spectral Density of an MA(2) and an
AR(1)
Spectral density of xt = (wt+1 + wt + wt−1 )/3.
Theorem
Let xt by an ARMA(p, q) process (not necessarily causal or invertible)
satisfying φ(B)xt = θ(B)wt where wt ∼ WN(0, σ 2 ), θ and φ have no
common zeros, and φ has no zeros on the unit circle. Then xt has the
spectral density
f (ω) = σ 2
|θ(e−iω )|2
,
|φ(e−iω )|2
−π ≤ ω ≤ π
Spectral density of xt = xt−1 − .9xt−2 .
Many people define the spectral density on −π to π, but this is just a
matter of scaling.
The spectral density of white noise is a constant (equal to σ 2 ).
Arthur Berg
Spectral Density (Chapter 12)
12/ 19
Arthur Berg
Spectral Density (Chapter 12)
13/ 19
Spectral Density
Periodogram
Spectral Density
Discrete Fourier Transform
The Periodogram
Definition
Given data x1 , . . . , xn , the discrete Fourier transform is defined to be
1
d(ωj ) = √
n
n
X
Periodogram
xt e−2πiωj t
Definition
The periodogram is defined at Fourier frequencies to be I(ωj ) where
I(ωj ) = |d(ωj )|2
t=1
for j = 0, 1, . . . , n − 1 where wj = j/n are the “Fourier frequencies”.
Amazingly, via some mathematical calculations, we have the identity
The discrete Fourier transform is a one-to-one transform of the data. The
original data can be produced from
n−1
X
I(ωj ) =
γ
b(h)e−2πiωj h
h=−(n−1)
n−1
1 X
xt = √
d(ωj )e2πiωj t
n j=0
Arthur Berg
Spectral Density (Chapter 12)
Arthur Berg
15/ 19
Spectral Density
Periodogram
Spectral Density
Periodogram as an Analysis of Variance
>
>
>
>
>
>
>
>
n
1 X
xt sin(2πωj t)
ds (ωj ) = √
n t=1
Then the total variation can be written as (for n odd)
(n−1)/2 h
(n−1)/2
n
i
X
X
X
2
2
2
(xt − x̄) = 2
dc (ωj ) + ds (ωj ) = 2
I(ωj )
j=1
Source
ω1
ω2
..
.
Arthur Berg
ω(n−1)/2
Total
df
2
2
..
.
2
2I(ω
)
P (n−1)/2 2
n − 1 Spectralnt=1
(x
−
x̄)
t
Density (Chapter 12)
Periodogram
x = c(1,2,3,2,1)
c1 = cos(2*pi*1:5*1/5)
s1 = sin(2*pi*1:5*1/5)
c2 = cos(2*pi*1:5*2/5)
s2 = sin(2*pi*1:5*2/5)
omega1 = cbind(c1, s1)
omega2 = cbind(c2, s2)
anova(lm(x~omega1+omega2))
Analysis of Variance
Response: x
Df Sum Sq
omega1
2 2.74164
omega2
2 0.05836
Residuals 0 0.00000
j=1
SS
2I(ω1 )
2I(ω2 )
..
.
16/ 19
Calculation in R
The cosine and sine transforms are defined as
n
1 X
dc (ωj ) = √
xt cos(2πωj t)
n t=1
t=1
Spectral Density (Chapter 12)
MS
I(ω1 )
I(ω2 )
..
.
Table
Mean Sq F value Pr(>F)
1.37082
0.02918
> abs(fft(x))^2/5
[1] 16.20000000
# ANOVA Table
# the periodogram (as a check)
1.37082039
0.02917961
0.02917961
1.37082039
I(ω(n−1)/2 )
17/ 19
Arthur Berg
Spectral Density (Chapter 12)
18/ 19
Spectral Density
Periodogram
Limitations of the Periodogram
From its structure:
n−1
X
I(ωj ) =
γ
b(h)e−2πiωj h
h=−(n−1)
it looks like a potential estimate of the spectral density given by
f (ω) =
∞
X
γ(h)e−2πiωh
h=−∞
But there’s a problem:
2I(ωj ) d
−→ χ22
f (ωj )
What we would like is a limit to a constant in the right hand side of the
above equation. Pros, Cons and fixes to the periodogram will be
discussed next time.
Arthur Berg
Spectral Density (Chapter 12)
19/ 19