Spectral Estimation Overview
Introduction
• Periodogram
jω
Rx (e ) • Bias, variance, and distribution
∞
rx ()e−jω
l=−∞
• Blackman-Tukey Method
• Most stationary random processes have continuous spectra
• Welch-Bartlett Method
• If we have time, will discuss line spectra at end of term
• Others
• Recall the definition of PSD above
• The estimation problem is to find a R̂x (ejω ) given only a finite
−1
data record {x(n)}N
0
• If the autocorrelation can be estimated with great accuracy at
long lags, can treat as a deterministic problem
• With short records (or equivalently, local stationarity), is more
difficult
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
1
J. McNames
Example 1: Data Windowed Autocorrelation
1
N
∞
ECE 538/638
Spectral Estimation
Ver. 1.14
2
Periodogram
2
N −1
2
1 1 jω
−jωn R̂x (e ) v(n)e
V (ejω )
=
N n=0
N
Solve for the expected value of the biased estimate of autocorrelation
applied to a windowed data segment. How should w(n) be scaled such
that the estimate is unbiased at = 0.
r̂x () =
Portland State University
where v(n) x(n)w(n).
x(n + ||)w(n + ||)x∗ (n)w∗ (n)
• If w(n) = c (a constant), is called the Periodogram
n=N −||−1
• If w(n) is not constant, is called the Modified Periodogram and
w(n) is called the data window
• Can be estimated quickly using the FFT
• Is related to the biased autocorrelation estimate we discussed last
time
N
−1
R̂x (ejω ) =
r̂v ()e−jω
=−(N −1)
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
3
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
4
Example 2: Periodogram and Biased Autocorrelation
Example 2: Workspace
Let x(n) be a finite duration sequence, 0 to N − 1. Show that the
biased estimate of autocorrelation is given by
r̂x () =
1
x() ∗ x(−)
N
and that the DTFT of r̂x () is equal to the Periodogram of x(n) when
w(n) = 1.
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
5
J. McNames
Periodogram
2
N −1
2
1 1 jω
−jωn R̂x (e ) v(n)e
V (ejω ) = F {r̂v ()}
=
N n=0
N
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
6
Example 3: Periodogram as a Filter Bank
2
N −1
1 jω
−jωn R̂x (e ) v(n)e
N
n=0
• Remember that we must use the biased estimate of
autocorrelation
– Otherwise the estimated PSD may be negative at some
frequencies
2
– The equality 1 V (ejω ) = F {r̂v ()} no longer holds
• How is the periodogram a filter bank?
• What is the transfer function of the filter?
• How good are the filters? How close are they to ideal filters?
N
• This relationship to r̂() means we can use the FFT to calculate
r̂() very efficiently
– Take FFT of signal, magnitude square, and inverse FFT
– Requires O(N log N ) operations instead of O(N 2 )!
– Must take care to zero pad sufficiently
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
7
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
8
Example 3: Work Space
Example 4: Peridogram
Generate the periodogram for a signal from a known LTI system with
two sets of complex conjugate poles close to one another and near the
unit circle. Repeat for various signal lengths.
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
9
J. McNames
Example 4: MATLAB Code
M
N
NA
cb
NZ
=
=
=
=
=
500;
[25,50,100,250,500,1000];
500;
95;
1024;
%
%
%
%
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
10
Ver. 1.14
12
Example 4: Pole Zero Map of ARMA Process
Padding to eliminate transient
Length of signal segment
No. averages
Confidence Bands
1
0.5
b = poly([-0.8,0.97*exp(j *pi/4),0.97*exp(-j *pi/4),...
0.97*exp(j *pi/6),0.97*exp(-j *pi/6)]);
% Numerator
a = poly([ 0.8,0.95*exp(j*3*pi/4),0.95*exp(-j*3*pi/4),...
0.95*exp(j*2.5*pi/4),0.95*exp(-j*2.5*pi/4)]); % Denominator
b = b*sum(a)/sum(b);
% Scale DC gain to 1
0
−0.5
−1
−1
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
11
J. McNames
−0.5
Portland State University
0
ECE 538/638
0.5
1
Spectral Estimation
Example 4: MATLAB Code
Example 4: Signal
figure
h = circle;
z = roots(b);
p = roots(a);
hold on;
h2 = plot(real(z),imag(z),’bo’,real(p),imag(p),’rx’);
hold off;
axis square;
xlim([-1.1 1.1]);
ylim([-1.1 1.1]);
axislines;
box off;
Length:100
300
200
Signal
100
0
−100
−200
0
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
13
J. McNames
10
20
ECE 538/638
70
80
90
Spectral Estimation
100
Ver. 1.14
14
Example 4: True PSD
n = 100;
w = randn(M+n,1);
x = filter(b,a,w); % System with known PSD
nx = length(x);
x = x(nx-n+1:nx); % Eliminate start-up transient (make stationary)
5
x 10
True PSD
3
2
PSD
figure;
k = 0:n-1;
h = stem(k,x);
set(h,’Marker’,’none’);
set(h,’LineWidth’,0.5);
box off;
ylabel(’Signal’);
xlabel(’Sample Index’);
title(sprintf(’Length:%d’,n));
xlim([0 n]);
ylim([min(x) max(x)]);
Portland State University
40
50
60
Sample Index
Portland State University
Example 4: MATLAB Code
J. McNames
30
1
PSD
0
ECE 538/638
Spectral Estimation
Ver. 1.14
15
0
0.05
0.1
0.15
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
0.4
0.45
0.5
0
10
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
16
Example 4: MATLAB Code
Example 4: Periodogram Estimate
[R,w] = freqz(b,a,NZ);
R2 = abs(R).^2;
f = w/(2*pi);
subplot(2,1,1);
h = plot(f,R2,’r’);
set(h,’LineWidth’,1.2);
box off;
ylabel(’PSD’);
title(’True PSD’);
xlim([0 0.5]);
ylim([0 max(R2)*1.2]);
subplot(2,1,2);
h = semilogy(f,R2,’r’);
set(h,’LineWidth’,1.2);
box off;
ylabel(’PSD’);
xlim([0 0.5]);
ylim([0.2*min(R2) max(R2)*5]);
xlabel(’Frequency (cycles/sample)’);
J. McNames
Portland State University
5
x 10
N:25 NA:1000 Confidence Bands:95%
4
Single Estimate
True PSD
Confidence Bands
Average Estimate
PSD
3
2
1
PSD
0
ECE 538/638
Spectral Estimation
Ver. 1.14
17
0
0.05
0.1
0.15
0
0.05
0.1
0.15
J. McNames
0.45
0.5
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
0.4
0.45
0.5
ECE 538/638
Spectral Estimation
Ver. 1.14
18
Example 4: Periodogram Estimate
x 10
2
N:100 NA:1000 Confidence Bands:95%
4
Single Estimate
True PSD
Confidence Bands
Average Estimate
Single Estimate
True PSD
Confidence Bands
Average Estimate
3
PSD
PSD
0.4
5
3
2
1
1
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0
0.5
PSD
PSD
0.35
Portland State University
N:50 NA:1000 Confidence Bands:95%
4
0
0
J. McNames
0.3
0
5
x 10
10
0.25
10
Example 4: Periodogram Estimate
0
0.2
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
19
0.05
0.1
0.15
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
0.4
0.45
0.5
0
10
0.5
Ver. 1.14
0
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
20
Example 4: Periodogram Estimate
Example 4: Periodogram Estimate
5
5
x 10
4
2
1
10
1
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0
0.5
PSD
PSD
0
0
0
J. McNames
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
21
Single Estimate
True PSD
Confidence Bands
Average Estimate
PSD
3
2
1
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.15
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
0.4
0.45
0.5
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
22
for c1 = 1:length(N),
nx = M+N(c1);
Rh = zeros(NA,NZ/2+1);
kh = 1:NZ/2+1;
fh = (kh-1)/NZ;
for c2 = 1:NA,
w = randn(M+N(c1),1);
x = filter(b,a,w); % System with known PSD
x = x(nx-N(c1)+1:nx); % Eliminate start-up transient (make stationary)
rh = (1/N(c1))*abs(fft(x,NZ)).^2;
Rh(c2,:) = rh(kh).’;
end;
N:1000 NA:1000 Confidence Bands:95%
4
0.1
Example 4: MATLAB Code
5
x 10
0.05
0
0.5
Ver. 1.14
0
10
Example 4: Periodogram Estimate
0
Single Estimate
True PSD
Confidence Bands
Average Estimate
3
PSD
2
N:500 NA:1000 Confidence Bands:95%
4
Single Estimate
True PSD
Confidence Bands
Average Estimate
3
PSD
x 10
N:250 NA:1000 Confidence Bands:95%
0.4
0.45
0.5
PSD
Rha = mean(Rh);
% Average
Rhu = prctile(Rh,100-(100-cb)/2); % Upper confidence band
Rhl = prctile(Rh,
(100-cb)/2); % Lower confidence band
10
0
0
J. McNames
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
23
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
24
Example 4: MATLAB Code
Periodogram Properties
figure;
subplot(2,1,1);
h = plot(fh,Rh(1,:),’g’,fh,Rhl,’b’,fh,Rhu,’b’,f,R2,’r’,fh,Rha,’k’);
set(h(1) ,’LineWidth’,0.3);
set(h(2:3),’LineWidth’,0.5);
set(h(4) ,’LineWidth’,1.5);
set(h(5) ,’LineWidth’,0.5);
ylabel(’PSD’);
title(sprintf(’N:%d NA:%d Confidence Bands:%d%%’,N(c1),NA,cb));
xlim([0 0.5]);
ylim([0 max(R2)*1.5]);
legend(h([1 2 4 5]),’Single Estimate’,’Confidence Bands’,’Average’,’True’,2);
subplot(2,1,2);
h = semilogy(fh,Rh(1,:),’g’,fh,Rhl,’b’,fh,Rhu,’b’,f,R2,’r’,fh,Rha,’k’);
set(h(1) ,’LineWidth’,0.3);
set(h(2:3),’LineWidth’,0.5);
set(h(4) ,’LineWidth’,1.5);
set(h(5) ,’LineWidth’,0.5);
ylabel(’PSD’);
xlabel(’Frequency (cycles/sample)’);
xlim([0 0.5]);
ylim([0.2*min(R2) max(R2)*5]);
end;
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
25
• The main disadvantage of the Periodogram appears to be
excessive variance
• As with any estimator, we would like to know the bias, variance,
covariance, confidence intervals, and distribution
• We will consider these each in turn
J. McNames
Portland State University
Periodogram Mean
E R̂x (ejω )
N
−1
=
E R̂x (ejω )
E [r̂v ()] e−jω
E [r̂v ()] = E
=
1
N
1
N
∞
x(n)w(n)x∗ (n − )w∗ (n − )
rx ()w(n)w∗ (n − )
n=−∞
= rx ()
1
N
n=−∞
ECE 538/638
Spectral Estimation
E [r̂x ()] e−jω
=
1
N
=
1
2π
N
−1
rx ()rw ()e−jω
=−(N −1)
π
−π
Rx (eju )
1
Rw ej(ω−u) du
N
• Here Rw (ejω ) = |W (ejω )|2 is the ESD of the window
1
rx ()rw ()
N
The expected value of r̂v () is the true autocorrelation rx () windowed
with rw () = N1 w(n) ∗ w(−n)∗ .
Portland State University
26
• Since E R̂x (ejω ) = Rx (ejω ), the Periodogram is a biased
estimator
w(n)w∗ (n − )
=
J. McNames
Ver. 1.14
=−(N −1)
∞
N
−1
=
n=−∞
∞
Spectral Estimation
Periodogram Mean Continued
=−(N −1)
ECE 538/638
Ver. 1.14
• Ideally, we would like Rw (ejω ) = 2πN δ(ω)
• The smearing limits our ability to distinguish sharp features (e.g.
distinct peaks) in the PSD
27
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
28
Periodogram Asymptotic Bias
E R̂x (e )
jω
=
1
N
N
−1
Periodogram Asymptotic Bias Continued
lim E R̂x (ejω ) = Rx (ejω )
N →∞
rx ()rw ()e−jω
• The estimator is unbiased as long as
π
N
−1
1
|w(n)|2 =
Rw (ejω ) dω = N
2π
−π
n=0
=−(N −1)
• Most windows are relatively constant over a range of samples
proportional to the window length:
N || N
rw () ≈
0 || > N
and the mainlobe window width decreases as
1
N
• Not of much relevance (asymptotic result)
• Most stationary signals have an autocorrelation that approaches
zero as → ∞
• Practically, the bias is introduced by the sidelobes and smearing of
the spectrum by the main lobe
• Thus, the periodogram can be considered to be an asymptotically
unbiased estimator
lim E R̂x (ejω ) = Rx (ejω )
• Can reduce with better windows, but can only trade main lobe
width for decreased sidebands
• If signal is white noise (flat spectrum), the smearing of the
convolution does not cause bias
N →∞
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
29
J. McNames
Portland State University
Periodogram Covariance Theory
1
R̂w (e ) =
N
jω
2
N −1
jωn w(n)e n=0
1
=
N
ECE 538/638
30
Recall that when zero padding is not used, the exponentials are
orthogonal
2π 2π ∗
2π
ejk N n ej N n =
ej(k−) N n
n=0
• The text lists the covariance of the Periodogram, but does not
derive it
n=<N >
n=<N >
N
=
0
• Details of the derivation can be found in [1]
• An outline of the derivation is provided here
k = 0, ±N, ±2N, . . .
otherwise
Now consider W (ejω ) evaluated at ω = k 2π
N .
• Suppose for now that no zero padding is used so we calculate the
DFT only at the N points that we have samples of the signal
• Let w(n) be a WGN process with variance
Ver. 1.14
Orthogonal Components
2
N −1
jk 2π
n
w(n)e N 2
σw
Spectral Estimation
W (ejω ) =
and zero mean
N
−1
w(n)ejωn
n=0
If w(n) is a WGN process, then W (ejω ) is also a Gaussian random
variable, since it is a linear combination of Gaussian RVs.
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
31
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
32
WGN PSD Mean and Variance
WGN DFT Mean and Variance Continued
If we don’t use zero padding,
−1
N
E W (ejω ) =
E[w(n)]ejωn = 0
2π
N
where m1 and m2 are integers. Then
ω 1 = m1
n=0
cov W (ejω1 ), W (ejω2 ) = E W (ejω1 )W (ejω2 )∗
N −1
∗ N −1
=E
w(k)ejω1 k
w()ejω2 k=0
=
−1
N
−1 N
=0
k=0
2
= σw
k=0 =0
=
=
2
σw
δ(k − )ej(ω1 k−ω2 )
k=0 =0
2
= σw
N
−1
Portland State University
ej(ω1 −ω2 )k
ECE 538/638
Spectral Estimation
Ver. 1.14
33
J. McNames
χ2 Random Variables
0.5
0.3
0.6
0.4
0.1
0.2
0
0
2
4
6
8
10
0
0
2
4
6
8
10
• Let xi be a set of IID RVs with a Normal distribution,
xi ∼ N (0, 1)
• Then the RV χ2 where
χ2 =
v
x2i
i=1
has a χ distribution with v degrees of freedom
2
• The PDF and CDF is available in MATLAB and equivalent
J. McNames
Portland State University
ECE 538/638
k=0
2
σw N δ(m1
− m2 )
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
34
Spectral Estimation
Ver. 1.14
36
x = linspace(0,10,100)’;
df = 1:5; % Degrees of freedom
nd = length(df);
nx = length(x);
P = zeros(nx,nd);
C = zeros(nx,nd);
for c1=1:5,
P(:,c1) = chi2pdf(x,df(c1));
C(:,c1) = chi2cdf(x,df(c1));
ls{c1} = sprintf(’v=%d’,df(c1));
end;
figure
h = plot(x,P);
set(h,’LineWidth’,1.5);
box off;
xlim([0 x(end)]);
ylim([0 .5]);
legend(ls);
figure
h = plot(x,C);
set(h,’LineWidth’,1.5);
box off;
xlim([0 x(end)]);
ylim([0 1.05]);
legend(ls);
v=1
v=2
v=3
v=4
v=5
0.8
0.2
2π
ej(m1 −m2 ) N k
MATLAB Code
1
v=1
v=2
v=3
v=4
v=5
0.4
N
−1
Thus, for WGN and no zero padding, the random variables are
uncorrelated and therefore independent! The variance of the RVs is
2
. Alternative, the number of uncorrelated estimates in the frequency
σw
domain scales with N . More data → more uncorrelated samples.
k=0
J. McNames
2π
N
N
−1
2
ej(ω1 −ω2 )k
cov W (ejω1 ), W (ejω2 ) = σw
E[w(k)w()∗ ]ejω1 k e−jω2 −1
N
−1 N
ω 2 = m2
Spectral Estimation
Ver. 1.14
35
J. McNames
Portland State University
ECE 538/638
WGN PSD Mean and Variance Continued
WGN PSD Distribution
Thus the distribution of |W (ejω )|2 is given by
π
χ2 (2)/2 ω =
|W (ejω )|2
∼
2
2N
σw
ω = π
χ (1)
−1
N
E W (ejω ) =
E[w(n)]ejωn = 0
n=0
2
cov W (ejω1 ), W (ejω2 ) = σw
N δ(ω1 − ω2 )
1
R̂w (ejω ) = |W (ejω )|2
N
where χ2 (1) is a chi-squared distribution with one degree of freedom.
Thus
σw χ2 (2)/2 ω = π
jω
R̂w (e ) ∼
ω = π
σw χ2 (1)
• If W (ejω ) is a complex Gaussian RV with zero mean and variance
2
, then what is the distribution of |W (ejω )|2 ?
σw
2
E |W (ejω )|2 ] = E Re{W (ejω )}2 + Im{W (ejω )}2 = σw
Thus, we know what the distribution of R̂w (ejω ) is in the WGN case,
but what about non-white processes?
where the real and imaginary components are both independent
2
/2
Gaussian RVs each with variance σw
• Note that Im{W (ejω )} = 0 if ω = 0 or ω = π
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
37
J. McNames
Generalized Linear Processes
x(n) = h(n) ∗ w(n) =
∞
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
38
Periodogram Distribution
It’s beyond the scope of this class to prove it (see [1, pg. 424]), but
h(k)w(n − k)
R̂x (ejω ) = Rx (ejω )
k=−∞
2
Rx (ejω ) = |H(ejω )|2 σw
1
R̂w (ejω ) + Rε (ω)
2
σw
where
• This is a standard and widely used model for wide-sense stationary
processes
E [Rε (ω)] = O(1/N 2 )
• Note that h(n) is two sided (non-causal)
• Thus Rε (ω) → 0 as N → ∞
• We require that h(n) has finite energy (a sufficient condition for
stability)
• If we assume N is “large” we can disregard it
– N must be large enough to make the bias due to windowing
negligible
• But how is R̂x (ejω ) related to the known distribution of R̂w (ejω )?
• This means that the periodogram estimate is χ2 distributed when
the process is Gaussian
• We can then immediately write the distribution of R̂x (ejω )
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
39
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
40
Periodogram Distribution
Rx (ejω )χ2 (2)/2 ω = π
jω
R̂x (e ) ∼
ω = π
Rx (ejω )χ2 (1)
Periodogram Covariance
• Depends on the true, unknown PSD so is of limited value
• Can make some qualitative observations
• Estimates at two frequencies ω1 = (2π/N )k1 and ω2 = (2π/N )k2
where k1 and k2 are integers are approximately uncorrelated
Since
var χ2 (2)/2 = 1
var χ2 (1) = 2
cov{R̂x (ejω1 ), R̂x (ejω2 )} ≈ 0 for k1 = k2
The variance of the periodogram is given by
Rx2 (ejω ) 0 < ω < π
jω
var{R̂x (e )} ≈
2Rx2 (ejω ) ω = 0, π
• The number of uncorrelated frequency pairs scales with N
• Thus, the periodogram at adjacent frequencies, say ω and ω + δ,
becomes more variable as N increases
which agrees with the text.
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
41
J. McNames
var[PSD]
0<ω<π
ω = 0, π
var[PSD]
• Thus, the Periodogram is not a consistent estimator
Ver. 1.14
43
Estimated Variance
Theoretical Variance
0
0.05
0.1
0.15
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
0.4
0.45
0.5
0
10
• The estimate becomes more variable as a function of frequency,
but the variance at a fixed frequency does not decrease
Spectral Estimation
42
5
0
• Note that the variance does not decrease as N → ∞
ECE 538/638
Ver. 1.14
N:25 NA:1000 Confidence Bands:95%
10
• Perhaps not surprising since the variance of r̂() depended on the
true autocorrelation
Portland State University
Spectral Estimation
10
x 10
• The variance depends on the true spectrum
J. McNames
ECE 538/638
Example 4: Periodogram Variance
Periodogram Variance (Again)
2 sin(ωN )
jω
2 jω
var{R̂x (e )} ≈ Rx (e ) 1 +
N sin(ω)
For large N this can be approximated as
Rx2 (ejω )
jω
var R̂x (e ) ≈
2Rx2 (ejω )
Portland State University
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
44
Example 4: Periodogram Variance
Example 4: Periodogram Variance
10
10
x 10
5
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0
0
10
0
J. McNames
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
45
0
0.05
0.1
0.15
0
0.05
0.1
0.15
J. McNames
var[PSD]
var[PSD]
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0
0
J. McNames
0.5
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
0.4
0.45
0.5
ECE 538/638
Spectral Estimation
Ver. 1.14
46
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
47
0
0.05
0.1
0.15
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
0.4
0.45
0.5
0
10
0.5
Ver. 1.14
Estimated Variance
Theoretical Variance
5
0.5
var[PSD]
var[PSD]
0.1
10
0.05
0.45
N:500 NA:1000 Confidence Bands:95%
10
5
0
0.4
10
x 10
N:250 NA:1000 Confidence Bands:95%
0.05
0.3
Example 4: Periodogram Variance
Estimated Variance
Theoretical Variance
0
0.35
Portland State University
10
x 10
0
0.25
0
Example 4: Periodogram Variance
10
0.2
10
0.5
Ver. 1.14
Estimated Variance
Theoretical Variance
5
0.5
var[PSD]
0
N:100 NA:1000 Confidence Bands:95%
10
Estimated Variance
Theoretical Variance
var[PSD]
var[PSD]
10
var[PSD]
x 10
N:50 NA:1000 Confidence Bands:95%
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
48
Example 4: Periodogram Variance
Periodogram Variance
2
N −1
2
1 1 jω
−jωn R̂x (e ) v(n)e
V (ejω ) = F {r̂()}
=
N n=0
N
10
x 10
N:1000 NA:1000 Confidence Bands:95%
var[PSD]
10
Estimated Variance
Theoretical Variance
• Since the ensemble variance of the Periodogram does not decrease
as N increases, it is considered a poor estimator
5
• We cannot fix this by choosing a better window
0
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
• This is perhaps surprising, because it is the “natural” estimator
0.5
var[PSD]
• To obtain a better estimator, we need to reduce the ensemble
variance
0
10
0
J. McNames
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
49
J. McNames
Fixing Problems with the Periodogram
• Thus we are essentially able to estimate the true PSD at more
frequencies
• We have better frequency resolution as N increases
50
• There are several ways to achieve this
– Average contiguous values of the periodogram
– Average periodograms obtained from multiple segments
• Key concept: If the PSD is smooth and our estimate consists of
uncorrelated high-resolution estimates, we can obtain a better
estimate by averaging estimates at adjacent frequencies
Spectral Estimation
Ver. 1.14
• Thus, we can tradeoff some of the excessive variance of the
Periodogram for increases bias by smoothing
• Key assumption: the true PSD is smooth
– Clearly true for all ARMA processes with poles and zeros not
too close to the unit circle
ECE 538/638
Spectral Estimation
• Smoothing the Periodogram can be thought of in several ways
– It reduces the frequency resolution
– It blurs the spectrum
– It increases the bias of the estimator
– It reduces the variance of the estimator
– It could either increase or decrease the MSE depending on the
degree of smoothing and the variability of the true PSD
• As N increases, the covariation at neighboring frequencies, ω and
ω + δ, decreases
Portland State University
ECE 538/638
Periodogram Smoothing
• As N increases, the ensemble variation stays constant
– The estimator is not consistent (statistically)
J. McNames
Portland State University
Ver. 1.14
• Both approaches produce about the same quality of estimators
51
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
52
Smoothing the Periodogram
R̂x(PS) (ejωk ) Example 5: Smoothed Periodogram Estimate
M
1
R̂x (ejωk−j )
2M + 1
5
x 10
PSD
2
1
• In the definition above (borrowed from the book), only applies if
R̂x (ejω ) is known at discrete frequencies ωk (2π/N )k
0
• This is usually the case because we estimate using the FFT
PSD
• Since the examples at these discrete frequencies are approximately
uncorrelated
1
var{R̂x (ejωk )}
var{R̂x(PS) (ejωk )} ≈
2M + 1
ECE 538/638
Spectral Estimation
Ver. 1.14
53
0
0.05
0.1
0.15
0
0.05
0.1
0.15
J. McNames
x 10
0.45
0.5
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
0.4
0.45
0.5
ECE 538/638
Spectral Estimation
Ver. 1.14
54
2
M:51 N:500 NA:1000 Confidence Bands:95%
4
Single Estimate
True PSD
Confidence Bands
Average Estimate
Single Estimate
True PSD
Confidence Bands
Average Estimate
3
PSD
PSD
0.4
5
3
2
1
1
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0
0.5
PSD
PSD
0.3
Example 5: Smoothed Periodogram Estimate
M:21 N:500 NA:1000 Confidence Bands:95%
4
0
0
J. McNames
0.35
Portland State University
5
x 10
10
0.25
0
Example 5: Smoothed Periodogram Estimate
0
0.2
10
• However, we increase the bias, especially near sharp features in the
frequency domain
Portland State University
Single Estimate
True PSD
Confidence Bands
Average Estimate
3
• One approach to smoothing is to apply a moving average filter in
the frequency domain
J. McNames
M:11 N:500 NA:1000 Confidence Bands:95%
4
j=−M
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
55
0.05
0.1
0.15
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
0.4
0.45
0.5
0
10
0.5
Ver. 1.14
0
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
56
Example 5: Smoothed Periodogram Estimate
Example 5: MATLAB Code
Q
N
NA
P
cb
NZ
5
x 10
M:101 N:500 NA:1000 Confidence Bands:95%
4
Single Estimate
True PSD
Confidence Bands
Average Estimate
PSD
3
2
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
[11,21,51,101];
500;
1000;
500;
95;
2048;
%
%
%
%
%
MA filter order
Length of signal segment
No. averages
Padding to eliminate transient
Confidence Bands
b = poly([-0.8,0.97*exp(j *pi/4),0.97*exp(-j *pi/4),...
0.97*exp(j *pi/6),0.97*exp(-j *pi/6)]);
% Numerator
a = poly([ 0.8,0.95*exp(j*3*pi/4),0.95*exp(-j*3*pi/4),...
0.95*exp(j*2.5*pi/4),0.95*exp(-j*2.5*pi/4)]); % Denominator
b = b*sum(a)/sum(b);
% Set DC gain to 1
1
0
=
=
=
=
=
=
0.5
PSD
[R,w] = freqz(b,a,NZ);
R2
= abs(R).^2;
f
= w/(2*pi);
10
0
0
J. McNames
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
57
J. McNames
Example 5: MATLAB Code Continued
for c1 = 1:length(Q),
nx = P+N;
Rh = zeros(NA,NZ/2+1);
kh = 1:NZ/2+1;
fh = (kh-1)/NZ;
for c2 = 1:NA,
w = randn(P+N,1);
x = filter(b,a,w);
x = x(nx-N+1:nx);
rh = (1/N)*abs(fft(x,NZ)).^2;
% System with known PSD
% Eliminate transient (make stationary)
Rh(c2,:) = rhs(kh).’;
end;
Rha = mean(Rh);
% Average
Rhu = prctile(Rh,100-(100-cb)/2); % Upper confidence band
Rhl = prctile(Rh,
(100-cb)/2); % Lower confidence band
Portland State University
ECE 538/638
Spectral Estimation
ECE 538/638
Spectral Estimation
Ver. 1.14
58
Example 5: MATLAB Code Continued
rh = [rh;rh;rh];
% Eliminate edge effects
rhs = filter(ones(Q(c1),1)/Q(c1),1,rh);
rhs = rhs((Q(c1)-1)/2+(NZ+1:2*NZ)); % Account for filter delay
J. McNames
Portland State University
Ver. 1.14
59
figure;
FigureSet(1,’LTX’);
subplot(2,1,1);
h = plot(fh,Rh(1,:),’g’,f,R2,’r’,fh,Rhl,’b’,fh,Rhu,’b’,fh,Rha,’k’);
set(h(1) ,’LineWidth’,0.3);
set(h(2) ,’LineWidth’,1.3);
set(h(3:4),’LineWidth’,0.5);
set(h(5) ,’LineWidth’,0.5);
ylabel(’PSD’);
title(sprintf(’Q:%d N:%d NA:%d Confidence Bands:%d%%’,Q(c1),N,NA,cb));
xlim([0 0.5]);
ylim([0 max(R2)*1.5]);
legend(h([1 2 4 5]),’Single Estimate’,’Confidence Bands’,’Average’,’True’,2);
subplot(2,1,2);
h = semilogy(fh,Rh(1,:),’g’,f,R2,’r’,fh,Rhl,’b’,fh,Rhu,’b’,fh,Rha,’k’);
set(h(1) ,’LineWidth’,0.3);
set(h(2) ,’LineWidth’,1.3);
set(h(3:4),’LineWidth’,0.5);
set(h(5) ,’LineWidth’,0.5);
ylabel(’PSD’);
xlabel(’Frequency (cycles/sample)’);
xlim([0 0.5]);
ylim([0.2*min(R2) max(R2)*5]);
end;
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
60
Periodogram Smoothing Comments
R̂x(PS) (ejωk )
Blackman-Tukey Spectral Estimation Idea
E R̂x (ejω )
M
1
R̂x (ejωk−j )
2M + 1
var Rx(PS) (ejω ) ≈
=
j=−M
1
var Rx (ejω )
2M + 1
1
2π
π
−π
Rx (eju )
1
Rw (ej(ω−u) ) du
N
• Recall that windowing the signal is equivalent to convolving the
signal’s PSD with the window’s PSD, in expectation
• Despite the blurring and smoothing of this effect, the estimate is
still has too high of variance
• Smoothing the periodogram is a good idea
• Trades reduced frequency resolution and increased bias for
reduced variance
• We can increase the smoothing and reduce the variance by
multiplying the autocorrelation by an even shorter window
• The moving average is just a lowpass filter
• There are many other lowpass filters that might perform better
• Examples: smoothing splines, local polynomial models, weighted
averaging (kernel smoothing)
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
61
J. McNames
Blackman-Tukey Spectral Estimation
R̂x(BT) (ejω ) L−1
r̂x ()wa ()e−jω
R̂x(BT) (ejω )
=−(L−1)
Ver. 1.14
62
L−1
r̂x ()wa ()e−jω
• Conceptually, this is using an even more biased estimate of the
autocorrelation: r̂x ()wa ()
• The additional bias is towards zero
• In order for the estimate to be nonnegative, W (ejω ) ≥ 0 for all ω
(BT)
• Thus, it reduces the variance of both r̂x () and R̂x
• Not all of the windows we described have this property
• The window can be any positive definite window with even
symmetry (why?)
• Assuming the window’s spectrum is bumped shape, this
convolution smooths the spectrum
• The length of the window L has a larger impact than the shape
• Note that the FFT can be used to calculate both r̂x () and
(BT)
R̂x (ejω ) efficiently
Spectral Estimation
Ver. 1.14
(ejω )
• The user controls this tradeoff by picking the correlation window
wa ()
• In the frequency domain, the product r̂x ()w() is equivalent to
convolving with W (ejω )
ECE 538/638
Spectral Estimation
=−(L−1)
• Called the correlation or lag window
Portland State University
ECE 538/638
Blackman-Tukey Bias & Variance
• Here the window is user specified and has a finite duration of
2L − 1 samples
J. McNames
Portland State University
63
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
64
Example 6: Blackman-Tukey Estimate
Example 7: Blackman-Tukey Coverage
5
x 10
N:500 L:25 NA:100 Confidence Bands:95%
4
Single Estimate
Average Estimated CIs
True PSD
Confidence Bands
Average Estimate
2
1
PSD
0
10
0
0.05
0.1
0.15
0.2
0.8
0.25
0.3
0.35
0.4
0.45
Coverage
PSD
3
0.5
0.6
0.4
0.2
0
0
0
J. McNames
N:500 L:25 NA:100 Confidence Bands:95%
1
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
65
0
J. McNames
0.05
N:500 L:51 NA:100 Confidence Bands:95%
1
PSD
0
10
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
ECE 538/638
0.35
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
66
0.6
0.4
0.2
0
0
0
J. McNames
0.3
0.8
Coverage
PSD
2
0.25
N:500 L:51 NA:100 Confidence Bands:95%
1
Single Estimate
Average Estimated CIs
True PSD
Confidence Bands
Average Estimate
3
0.2
Example 8: Blackman-Tukey Coverage
5
4
0.15
Portland State University
Example 7: Blackman-Tukey Estimate
x 10
0.1
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
67
J. McNames
0
0.05
0.1
0.15
Portland State University
0.2
0.25
0.3
ECE 538/638
0.35
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
68
Example 8: Blackman-Tukey Estimate
Example 9: Blackman-Tukey Coverage
5
x 10
N:500 L:101 NA:100 Confidence Bands:95%
4
Single Estimate
Average Estimated CIs
True PSD
Confidence Bands
Average Estimate
2
1
PSD
0
10
0
0.05
0.1
0.15
0.2
0.8
0.25
0.3
0.35
0.4
0.45
Coverage
PSD
3
0.5
0.6
0.4
0.2
0
0
0
J. McNames
N:500 L:101 NA:100 Confidence Bands:95%
1
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
69
0
J. McNames
0.05
N:500 L:251 NA:100 Confidence Bands:95%
1
PSD
0
10
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
ECE 538/638
0.35
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
70
0.6
0.4
0.2
0
0
0
J. McNames
0.3
0.8
Coverage
PSD
2
0.25
N:500 L:251 NA:100 Confidence Bands:95%
1
Single Estimate
Average Estimated CIs
True PSD
Confidence Bands
Average Estimate
3
0.2
Example 10: Blackman-Tukey Coverage
5
4
0.15
Portland State University
Example 9: Blackman-Tukey Estimate
x 10
0.1
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
71
J. McNames
0
0.05
0.1
0.15
Portland State University
0.2
0.25
0.3
ECE 538/638
0.35
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
72
Example 10: Blackman-Tukey Estimate
Example 11: Blackman-Tukey Coverage
5
x 10
N:500 L:451 NA:100 Confidence Bands:95%
4
Single Estimate
Average Estimated CIs
True PSD
Confidence Bands
Average Estimate
2
1
PSD
0
10
0
0.05
0.1
0.15
0.2
0.8
0.25
0.3
0.35
0.4
0.45
Coverage
PSD
3
0.5
0.6
0.4
0.2
0
0
0
J. McNames
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
73
J. McNames
Example 11: MATLAB Code
L
N
NA
P
cb
NZ
np
=
=
=
=
=
=
=
N:500 L:451 NA:100 Confidence Bands:95%
1
[25,51,101,251,451];
500;
1000;
500;
95;
1024;
2^nextpow2(2*N-1);
%
%
%
%
%
0
0.05
0.1
0.15
Portland State University
0.2
0.25
0.3
ECE 538/638
0.35
0.4
0.45
Spectral Estimation
0.5
Ver. 1.14
74
Example 11: MATLAB Code Continued
MA filter order
Length of signal segment
No. averages
Padding to eliminate transient
Confidence Bands
for c1 = 1:length(L),
nx = P+N;
Rh = zeros(NA,NZ/2+1);
kh = 1:NZ/2+1;
fh = (kh-1)/NZ;
for c2 = 1:NA,
w = randn(P+N,1);
x = filter(b,a,w);
x = x(nx-N+1:nx);
xf = fft(x,np);
r
= ifft(xf.*conj(xf));
r
= real(r(1:nx))/N;
no = (L(c1)+1)/2;
r
= [r(no:-1:2);r(1:no)];
wn = blackman(length(r));
wn = wn/max(wn);
rh = abs(fft(r.*wn,NZ));
Rh(c2,:) = rh(kh).’;
end;
% Figure out how much to pad the signal
b = poly([-0.8,0.97*exp(j *pi/4),0.97*exp(-j *pi/4),...
0.97*exp(j *pi/6),0.97*exp(-j *pi/6)]);
% Numerator
a = poly([ 0.8,0.95*exp(j*3*pi/4),0.95*exp(-j*3*pi/4),...
0.95*exp(j*2.5*pi/4),0.95*exp(-j*2.5*pi/4)]); % Denominator
b = b*sum(a)/sum(b);
% Set DC gain to 1
[R,w] = freqz(b,a,NZ);
R2
= abs(R).^2;
f
= w/(2*pi);
%
%
%
%
%
%
%
%
%
%
System with known PSD
Eliminate transient (make stationary)
Calculate FFT of x
Calculate biased autocorrelation
Eliminate superfluous zeros
Number of one-sided points to included
Make two-sided
Create window
Make maximum = 1
Window autocorrelation
Rha = mean(Rh);
% Average
Rhu = prctile(Rh,100-(100-cb)/2); % Upper confidence band
Rhl = prctile(Rh,
(100-cb)/2); % Lower confidence band
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
75
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
76
Example 11: MATLAB Code Continued
Blackman-Tukey Mean
figure;
subplot(2,1,1);
h = plot(fh,Rh(1,:),’g’,f,R2,’r’,fh,Rhl,’b’,fh,Rhu,’b’,fh,Rha,’k’);
set(h(1) ,’LineWidth’,0.3);
set(h(2) ,’LineWidth’,1.3);
set(h(3:4),’LineWidth’,0.5);
set(h(5) ,’LineWidth’,0.5);
ylabel(’PSD’);
title(sprintf(’N:%d L:%d NA:%d Confidence Bands:%d%%’,N,L(c1),NA,cb));
xlim([0 0.5]);
ylim([0 max(R2)*1.5]);
AxisSet(8);
legend(h([1 2 4 5]),’Single Estimate’,’Confidence Bands’,’Average’,’True’,2);
subplot(2,1,2);
h = semilogy(fh,Rh(1,:),’g’,f,R2,’r’,fh,Rhl,’b’,fh,Rhu,’b’,fh,Rha,’k’);
set(h(1) ,’LineWidth’,0.3);
set(h(2) ,’LineWidth’,1.3);
set(h(3:4),’LineWidth’,0.5);
set(h(5) ,’LineWidth’,0.5);
ylabel(’PSD’);
xlabel(’Frequency (cycles/sample)’);
xlim([0 0.5]);
ylim([0.2*min(R2) max(R2)*5]);
end;
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
77
R̂x(BT) (ejω )
L−1
r̂x ()wa ()e−jω
=−(L−1)
E R̂x(BT) (ejω )
L−1
=
=−(L−1)
L−1
=
=−(L−1)
=
E [r̂x ()] wa ()e−jω
1
rx () rw () wa ()e−jω
N
Rx (ejω ) ∗ W (ejω ) ∗ Wa (ejω )
• Recall that the periodogram had an expected value given by the
convolution of the Bartlett window’s spectrum and the true PSD
• Multiplication by another window causes another convolution
• Text assumes the data window w() is rectangular
J. McNames
Blackman-Tukey Mean
E R̂x(BT) (ejω ) = Rx (ejω ) ∗ W (ejω ) ∗ Wa (ejω )
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
78
Blackman-Tukey Sampling Distribution
R̂x(BT) (ejω )
• Note that the convolution is circular since each term is a periodic
function of ω
L−1
r̂x ()wa ()e−jω
=−(L−1)
• The estimate is asymptotically unbiased if wa (0) = 1
• Sampling properties of the spectral estimates are extremely
complicated
• As L and N approach ∞, both windows become impulse trains
• Exact results can only be obtained in special circumstances
• Most expressions for mean, variance, covariance, and distributions
are asymptotic and only apply when N is large
• The asymptotic expressions assume that L increases with N , but
more slowly than N : L → ∞, N → ∞, L/N → 0
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
79
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
80
Blackman-Tukey Covariance
Blackman-Tukey Variance
π
1
var{R̂x(BT) (ejω1 )} ≈
R2 (eju )Wa2 (ej(ω−u) ) du
2πN −π x
cov{R̂x(BT) (ejω1 ), R̂x(BT) (ejω2 )}
π
1
≈
R2 (eju )Wa (ej(ω1 −u) )Wa (ej(ω2 −u) ) du
2πN −π x
If Rx2 (ejω ) is smooth over the width of the window and can be
approximated as a constant, then
• Assumptions
– N is large so that WB (ejω ) behaves like an impulse train
– L is sufficiently large that Wa (ejω ) is narrow and the product
Wa (ejω1 )Wa (ejω2 ) is negligible
var{R̂x(BT) (ejω )}
• Thus, covariance increases as
– The window’s spectrum, Wa (ej(ω1 −u) ), grows wider
– The overlap between windows centered at different frequencies,
ω1 and ω2 , increases
– For estimates to be uncorrelated, they must be separated in
frequency by an amount greater than the “bandwidth” of the
window
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
Ew
=−(L−1)
• Ew is the energy of the correlation window
•
81
Ew
N
J. McNames
Blackman-Tukey Confidence Intervals
is called the variance reduction factor or variance ratio
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
82
Example 12: Blackman-Tukey Bias-Variance Tradeoff
Plot the bias and variance at the frequencies of the poles and zeros of
the LTI system for a range of window lengths ranging from 25–501
samples.
χ2ν (1 − α/2)
< 10 log R̂x(BT) (ejω ),
ν
ν
10 log R̂x(BT) (ejω ) < 10 log R̂x(BT) (ejω ) + 10 log 2
χν (1 − α/2)
10 log R̂x(BT) (ejω ) − 10 log
ν = L
π
1
≈
W 2 (ej(ω−u) ) du
2πN −π a
Ew
= Rx2 (ejω )
0<ω<π
N
L−1
=
wa2 ()
Rx2 (ejω )
2N
=−(L−1)
wa2 ()
• These are only approximate confidence intervals
• Only works when
– Bias is small N → ∞
– L N such that L/N → 0 (this is why we don’t obtain the
Periodogram estimate when L = N )
– This estimate is a linear combination of the periodogram
estimate, so it is also approximately has a χ2 distribution, but
with more degrees of freedom
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
83
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
84
Example 12: Blackman-Tukey Bias-Variance Frequencies
Example 12: Blackman-Tukey Bias-Variance Tradeoff
5
ω = 0.785 rad/sample
2500
3
2
PSD
N:500 NA:2500
3000
True PSD
1
ω = 0.524 rad/sample
x 10
2500
2000
1500
1000
500
2000
1500
1000
500
0
0
100
0
0.5
1
1.5
2
2.5
3
x 10
200
300
ω = 2.356 rad/sample
PSD
10
500
x 10
Variance
2
Bias
2
Variance + Bias
MSE
3
2
1
0
J. McNames
0.5
1
1.5
2
Frequency (cycles/sample)
Portland State University
ECE 538/638
2.5
200
300
400
500
9
8
6
4
2
0
100
200
300
400
500
Autocorrelation Window Length, L (samples)
0
100
10
4
0
400
ω = 1.963 rad/sample
0
100
200
300
400
500
Autocorrelation Window Length, L (samples)
3
Spectral Estimation
Ver. 1.14
85
J. McNames
Example 12: Blackman-Tukey Bias-Variance Tradeoff
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
86
Example 12: MATLAB Code
L = round(linspace(12,250,25))*2+1; % Autocorrelation window length
ef = [pi/4 pi/6 3*pi/4 2.5*pi/4]; % Evaluation frequencies
50
N
NA
P
NZ
np
100
150
200
=
=
=
=
=
500;
2000;
500;
2^9;
2^nextpow2(2*N-1);
%
%
%
%
%
Length of signal segment
No. averages
Padding to eliminate transient
No. zeros
Figure out how much to pad the signal
b = poly([-0.8,0.97*exp(j *pi/4),0.97*exp(-j *pi/4),...
0.97*exp(j *pi/6),0.97*exp(-j *pi/6)]);
% Numerator coefficients
a = poly([ 0.8,0.95*exp(j*3*pi/4),0.95*exp(-j*3*pi/4),...
0.95*exp(j*2.5*pi/4),0.95*exp(-j*2.5*pi/4)]); % Denominator coefficients
b = b*sum(a)/sum(b);
% Scale DC gain to 1
[R,w] = freqz(b,a,NZ);
R
= abs(R).^2;
R
= R’;
f
= w;
nL
= length(L);
250
300
350
400
450
500
0
J. McNames
0.5
1
Portland State University
1.5
2
ECE 538/638
2.5
Bias = zeros(nL,NZ);
Var = zeros(nL,NZ);
kh = 1:NZ;
nx = P+N;
3
Spectral Estimation
Ver. 1.14
87
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
88
Example 12: MATLAB Code Continued
for c1
Rh
no
wn
wn
wn
=
=
=
=
=
=
1:nL,
zeros(NA,NZ);
(L(c1)+1)/2;
blackman(no*2-1+2);
wn(2:end-1);
wn/max(wn);
%
%
%
%
Example 12: MATLAB Code Continued
Number of one-sided points to included in estimate
Create window
Trim MATLAB’s zeros
Make maximum = 1
for c2 = 1:NA,
w = randn(P+N,1);
x = filter(b,a,w); % System with known PSD
x = x(nx-N+1:nx); % Eliminate start-up transient (make stationary)
xf = fft(x,np);
% Calculate FFT of x
r
= ifft(xf.*conj(xf));
% Calculate biased autocorrelation
r
= real(r(1:nx))/N;
% Eliminate superfluous zeros and nearly zero imaginar
r
= [r(no:-1:2);r(1:no)]; % Make two-sided
rh = abs(fft(r.*wn,2*NZ)); % Window autocorrelation and calculate the estimate
Rh (c2,:) = rh(kh).’;
end;
Rha = mean(Rh);
% Average
Bias(c1,:) = R-Rha;
Var (c1,:) = mean((Rh - ones(NA,1)*Rha).^2);
MSE (c1,:) = mean((Rh - ones(NA,1)*R ).^2);
drawnow;
end;
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
89
figure;
for c1=1:length(ef),
[jnk,id] = min(abs(ef(c1)-f));
subplot(2,2,c1);
h = plot(L,Var(:,id),L,Bias(:,id).^2,L,Var(:,id)+Bias(:,id).^2,L,MSE(:,id));
set(h(3),’LineWidth’,3);
set(h(4),’LineWidth’,1);
xlim([min(L) max(L)]);
ylim([0 1.05*max(MSE(:,id))]);
box off;
ylabel(sprintf(’\\omega = %5.3f rad/sample’,ef(c1)));
if c1==4,
xlabel(’Autocorrelation Window Length, L (samples)’);
elseif c1==1,
title(sprintf(’N:%d NA:%d’,N,NA));
elseif c1==3,
xlabel(’Autocorrelation Window Length, L (samples)’);
legend(’Variance’,’Bias^2’,’Variance + Bias^2’,’MSE’);
end;
end;
figure;
sMSE = zeros(size(MSE));
for c1=1:length(f),
sMSE(:,c1) = MSE(:,c1)/max(MSE(:,c1));
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
90
Periodogram Averaging
end;
h = imagesc(f,L,sMSE,[0 1]);
[jnk,id] = min(sMSE);
hold on;
plot(f,L(id),’w.’);
hold off;
R̂x(PA) (ejω )
K−1
1 R̂x (ejω )
K i=0
• Consider K IID random variables
• The variance of the mean is 1/K times the variance of the
random variables
• If we had different independent realizations of the process we
could average them and obtain a similar reduction in the variance
of the PSD
• In most situations, only a single realization is available
• Can subdivide the record into K possibly overlapping segments of
length L
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
91
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
92
Welch-Bartlett Spectral Estimation
2
K−1
L−1
K−1
2
1 1 1 (WB) jωk
−jωn R̂x
(e ) vi (n)e
Vi (ejω )
=
K
L
KL
n=0
i=0
Example 13: Welch-Bartlett Spectral Estimation
Generate the Welch-Bartlett estimated PSD for a signal from a known
LTI system with two sets of complex conjugate poles close to one
another and near the unit circle. Repeat for various segment lengths.
i=0
where v(n) x(n)w(n).
• Where w(n) is called the data window
• Window does not need to be positive definite
• Window trades sidelobe leakage (ripples) for mainlobe width
(blurring)
• If no overlap between windows, called the Bartlett estimate
• If overlap, called Welch’s method
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
93
J. McNames
Portland State University
Example 13: Welch-Bartlett Estimate
4
4
2
1
10
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0
0.5
0
0
J. McNames
N:500 L:50 OL:50% K:19 NA:1000 Confidence Bands:95%
1
PSD
PSD
0
94
Single Estimate
True PSD
Confidence Bands
Average Estimate
3
PSD
PSD
2
Ver. 1.14
5
x 10
N:500 L:10 OL:50% K:99 NA:1000 Confidence Bands:95%
Single Estimate
True PSD
Confidence Bands
Average Estimate
3
Spectral Estimation
Example 13: Welch-Bartlett Estimate
5
x 10
ECE 538/638
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
95
0.05
0.1
0.15
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
0.4
0.45
0.5
0
10
0.5
Ver. 1.14
0
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
96
Example 13: Welch-Bartlett Estimate
Example 13: Welch-Bartlett Estimate
5
x 10
4
4
Single Estimate
True PSD
Confidence Bands
Average Estimate
2
2
1
10
1
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0
0.5
PSD
PSD
0
0
0
J. McNames
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
Portland State University
ECE 538/638
0.4
0.45
Spectral Estimation
97
=
=
=
=
=
=
=
[10,50,100,200];
0.50;
500;
500;
1000;
95;
1024;
%
%
%
%
%
%
0.05
0.1
0.15
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.2
0.25
0.3
0.35
Frequency (cycles/sample)
0.4
0.45
0.5
0
0.5
Ver. 1.14
0
10
J. McNames
Example 13: MATLAB Code
L
O
M
N
NA
cb
NZ
N:500 L:200 OL:50% K:4 NA:1000 Confidence Bands:95%
Single Estimate
True PSD
Confidence Bands
Average Estimate
3
PSD
3
PSD
5
x 10
N:500 L:100 OL:50% K:9 NA:1000 Confidence Bands:95%
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
98
Example 13: MATLAB Code Continued
Segment lengths
Overlap
Padding to eliminate transient
Length of signal segment
No. averages
Confidence Bands
b = poly([-0.8,0.97*exp(j *pi/4),0.97*exp(-j *pi/4),...
0.97*exp(j *pi/6),0.97*exp(-j *pi/6)]);
% Numerator coefficients
a = poly([ 0.8,0.95*exp(j*3*pi/4),0.95*exp(-j*3*pi/4),...
0.95*exp(j*2.5*pi/4),0.95*exp(-j*2.5*pi/4)]); % Denominator coefficients
b = b*sum(a)/sum(b);
% Scale DC gain to 1
for c1 = 1:length(L),
K = floor((N-O*L(c1))./(L(c1)-O*L(c1))); % No. segments
ss = floor(L(c1)-O*L(c1));
% Step size
nx = M+N;
Rh = zeros(NA,NZ/2+1);
kh = 1:NZ/2+1;
fh = (kh-1)/NZ;
for c2 = 1:NA,
w = randn(M+N,1);
x = filter(b,a,w);
% System with known PSD
x = x(nx-N+1:nx); % Eliminate start-up transient (make stationary)
rh = zeros(NZ,1);
for c3 = 1:K,
id = (1:L(c1)) + (c3-1)*ss;
rh = rh + abs(fft(x(id),NZ)).^2;
end;
rh = rh/(K*L(c1));
Rh(c2,:) = rh(kh).’;
end;
[R,w] = freqz(b,a,NZ);
R2
= abs(R).^2;
f
= w/(2*pi);
Rha = mean(Rh);
% Average
Rhu = prctile(Rh,100-(100-cb)/2); % Upper confidence band
Rhl = prctile(Rh,
(100-cb)/2); % Lower confidence band
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
99
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
100
Example 13: MATLAB Code Continued
Welch-Bartlett Mean
figure;
subplot(2,1,1);
h = plot(fh,Rh(1,:),’g’,f,R2,’r’,fh,Rhl,’b’,fh,Rhu,’b’,fh,Rha,’k’);
set(h(1) ,’LineWidth’,0.5);
set(h(2) ,’LineWidth’,1.1);
set(h(3:4),’LineWidth’,0.5);
set(h(5) ,’LineWidth’,0.8);
ylabel(’PSD’);
title(sprintf(’N:%d L:%d OL:%d%% K:%d NA:%d Confidence Bands:%d%%’,N,L(c1),O*10
xlim([0 0.5]);
ylim([0 max(R2)*1.5]);
legend(h([1 2 4 5]),’Single Estimate’,’True PSD’,’Confidence Bands’,’Average Estimat
subplot(2,1,2);
h = semilogy(fh,Rh(1,:),’g’,f,R2,’r’,fh,Rhl,’b’,fh,Rhu,’b’,fh,Rha,’k’);
set(h(1) ,’LineWidth’,0.5);
set(h(2) ,’LineWidth’,1.1);
set(h(3:4),’LineWidth’,0.5);
set(h(5) ,’LineWidth’,0.8);
ylabel(’PSD’);
xlabel(’Frequency (cycles/sample)’);
xlim([0 0.5]);
ylim([0.2*min(R2) max(R2)*5]);
end;
E R̂x(WB) (ejωk )
K−1 2
1 1 Vi (ejω )
E
K i=0
L
1 jω 2
= E
Vi (e )
L
1
= Rx (ejω ) ∗ Rw (ejω )
L
π
1
Rx (ejω )Rw (ej(ω−θ) ) dθ
=
2π L −π
=
As with the periodogram, if the window is normalized such that
π
L−1
1
2
w (n) =
Rw (ejω ) dω = L
2π
−π
n=0
then the estimate is asymptotically unbiased
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
101
Welch-Bartlett Variance
var{R̂x(WB) (ejωk )} ≈
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
102
Welch-Bartlett Concepts
2
K−1
L−1
K−1
2
1 1 1 (WB) jωk
−jωn R̂x
(e ) vi (n)e
Vi (ejω )
=
K i=0 L n=0
KL i=0
1
1
var{R̂x (ejωk )} ≈
var{R̂x(WB) (ejωk )}
K
K
• Assumes segments are independent
• Thus, the user can once again trade variance for bias
• Not true, so estimate is optimistic
• If N = KL is fixed, then can reduce variance at the expense of
increased variance by increasing K and decreasing L
• Book fails to mention this
• The variance is insensitive to the shape of the window
• Increasing overlap up to 50% reduces variance without
substantially impacting the bias
• Further overlap increases the computational requirements, but
does not help bias or variance
• Confidence intervals listed in the text
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
103
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
104
Welch-Bartlett Comments
R̂x(WB) (ejωk ) =
1
KL
K−1
Summary
Vi (ejω )2
• We discussed several different nonparametric methods of spectral
estimation
i=0
• The “natural” estimate was the periodogram
• Consider the case when the overlap is L − 1 samples
• This would reduce variance the most
• This estimate has unacceptable variance for most applications
• In this case it becomes equivalent to periodogram smoothing [1,
pp. 583–585]!
• There are two fundamental approaches to reduce variance
– Periodogram smoothing
– Periodogram averaging
• Bartlett originally proposed averaging periodograms of separate
segments, but showed that this was essentially equivalent to the
Blackman-Tukey method with a “Bartlett” autocorrelation
window [1, pp. 439-440]
• They have comparable performance and ultimately are
approximately equivalent
• Each enables the user to control the bias-variance tradeoff
• When the overlap is less than L − 1
– It is only approximately equivalent to periodogram smoothing
– It requires L FFTs
– Variance is increased
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
• Typically the smoothing parameter is chosen to minimize bias
• Only approximate confidence intervals are available for each
105
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
106
References
MATLAB Resources
• The MATLAB implementation of these functions is constantly
changing (improving?)
[1] M. B. Priestley. Spectral Analysis and Time Series. Academic
Press, 1981.
• The documentation is pretty good
– Open up the help window (helpwin)
– Signal Processing Toolbox → Statistical Signal Processing
• Most of the techniques we have discussed are implemented
• Interface is not user-friendly
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
107
J. McNames
Portland State University
ECE 538/638
Spectral Estimation
Ver. 1.14
108