Introduction and preliminaries
Main results
Sketch of proofs
On the cross-correlogram estimation of impulse
response functions in linear systems
Blazhievska Irina
National Technical University of Ukraine "KPI",
Department of Mathematical Analysis and Probability Theory
January, 2011
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Introduction
Problem definition
Example of the spectral densities for X∆
Form of the cross-correlogram estimator
Form of the error term
Let H = (H (t ), t ∈ R) be a real-valued response function of a
time-invariant continuous linear system with inner noises V1 .
The response of the system to an input process X (t ), t ∈ R, has
the form
Z ∞
U (t ) =
H (t − s )X (s )ds + Z (t ), t ∈ R,
−∞
where Z (t ), t ∈ R, describes inner noises of the system.
In practice V1 is often supposed to be a causal linear system,
that is H (s ) = 0, s < 0. We will consider a general framework.
Let us focus on the problem of estimation of the unknown
function H by observations of responses of the system to certain
inputs. If the system is stable (H ∈ L1 (R), then the method of
periodograms may be used for solving this problem [H. Akaike
(1965); R. Bentkus (1972)]. For an unstable system, it is
reasonable to use other methods, in particular, a method of
correlograms.
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Introduction
Problem definition
Example of the spectral densities for X∆
Form of the cross-correlogram estimator
Form of the error term
The sample cross-correlograms for different classes of stochastic
processes and fields, in particular Gaussian, were studied in
works [D. Brillinger (1980)], [J. Bendat, А. Piersol (1983)], [A.
Dykhovichny (1983)], [N. Leonenko and A. Ivanov (1986)], [V.
Buldygin and Fu Li (1997)], [V. Buldygin, F. Utzet and V.
Zaiats (2004)] and [V. Buldygin and Yu. Kozachenko (2000)].
In this talk we use the method of correlograms for the
estimation of the response function H ∈ L2 (R). Such an
assumption makes it possible to consider unstable systems with
resonant singularities. The inputs are supposed to be zero-mean
stationary Gaussian processes that are "close"to a white noise.
The integral-type sample cross-correlograms between the inputs
and outputs are taken as estimators for H . We continue the
search of [V. Buldygin and Fu Li (1997)], and focuses on the
asymptotic normality of finite-dimensional distributions of the
normalized error term and it asymptotic normality in space of
continuous functions.
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Introduction
Problem definition
Example of the spectral densities for X∆
Form of the cross-correlogram estimator
Form of the error term
Let X∆ = (X∆ (t ), t ∈ R), ∆ > 0, be a family of real-valued
stationary zero-mean Gaussian processes that disturb the
system V1 . The spectral densities f∆ = (f∆ (λ), λ ∈ R), ∆ > 0, of
processes X∆ satisfy the following conditions:
f∆ (λ) = f∆ (−λ), λ ∈ R;
(1a)
sup kf∆ k∞ < ∞;
(1b)
∆>0
f∆ ∈ L1 (R);
c ∃c ∈ (0, ∞) ∀a ∈ (0, ∞) : lim sup f∆ (λ) − = 0;
∆→∞ −a ≤λ≤a
2π
K∆ ∈ L1 (R),
(1c)
(1d)
(1e)
R∞
where K∆ (t ) = −∞ e i λt f∆ (t ) dt , t ∈ R, is a correlation function of
X∆ .
The conditions (1c)-(1e) show in what sense the processes X∆
tend to a Gaussian white noise as ∆ → ∞.
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Introduction
Problem definition
Example of the spectral densities for X∆
Form of the cross-correlogram estimator
Form of the error term
Example 1
The conditions (1a) - (1e) are satisfied by family of functions
!
c
λ2
f∆ =
exp −
, λ ∈ R , ∆ > 0.
2π
∆
We assume that the inner noise (Z (t ), t ∈ R) is a separable
real-valued stationary zero-mean Gaussian process, which is
orthogonal to X∆ ; that is, EX∆ (s )Z (t ) = 0, s , t ∈ R.
Let g (λ), λ ∈ R, be a spectral density of the process Z . Suppose,
this function satisfies the conditions:
g (λ) = g (−λ);
(2a)
g ∈ L1 (R).
(2b)
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Introduction
Problem definition
Example of the spectral densities for X∆
Form of the cross-correlogram estimator
Form of the error term
The reaction of the system V1 on X∆ is represented by
Z ∞
H (t − s )X∆ (s ) ds + Z (t ), t ∈ R.
U∆ (t ) =
−∞
The so-called cross-correlogram
Z T
1
b
HT ,∆ (τ) =
U∆ (t + τ)X∆ (t ) dt , τ ∈ R,
cT
0
will be used as an estimator for H . Here T is the length of
averaging interval [0, T ].
Remark that these integrals are interpreted as the mean-square
Riemann integrals.
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Introduction
Problem definition
Example of the spectral densities for X∆
Form of the cross-correlogram estimator
Form of the error term
Generally speaking, for all T > 0, ∆ > 0, and τ ∈ R
Z ∞
bT ,∆ (τ) = 1
H (τ) , EH
K∆ (τ − s )H (s ) ds ,
c
−∞
bT ,∆ is biased.
that is, the estimator H
Consider the normalized error term
√
b T ,∆ (τ) = T [H
bT ,∆ (τ) − H (τ)], τ ∈ R,
W
and focus on the problem of asymptotic normality of the process
b T ,∆ = (W
b T ,∆ (τ), τ ∈ R) as parameters T , ∆ tend to infinity.
W
The complexity of this problem is that the function H ∈ L2 (R)
bT ,∆ is biased.
and the estimator H
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
b T ,∆
Asymptotic normality of the distributions of W
Example functions that satisfy the balance conditions
b
Asymptotic normality of the distributions WT ,∆ in C[a , b
Consider the real-valued zero-mean Gaussian process
A = (A (τ), τ ∈ R) with the correlation function
EA (τ1 )A (τ2 ) =
1
=
2π
Z
∞
e
i (τ1 −τ2 )λ
2π
i (τ1 +τ2 )λ
∗
2
∗
2
(H (λ)) d λ,
|H (λ)| + g (λ) + e
c
−∞
for all τ1 , τ2 ∈ R.
b T ,∆ , and A are separable processes
Further we suppose that W
defined on the same probability space {Ω, F , P}.
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
b T ,∆
Asymptotic normality of the distributions of W
Example functions that satisfy the balance conditions
b
Asymptotic normality of the distributions WT ,∆ in C[a , b
Theorem 1
Assume that g ∈ L1 (R); for some α ∈ (0, 1] H ∈ Lipα (R) ∩ L2 (R)
and T → ∞, ∆ → ∞ in such a way that the balance conditions:
√ 2πf∆ (0) T 1−
→ 0;
(3a)
c
√ Z ∞
∀δ > 0 : T
K∆ (t ) dt → 0;
(3b)
Z δ∞
2
K∆
(t ) dt → 0;
(3c)
∀δ > 0 : T
δ
Z
δ
√
∃δ > 0 : T
|K∆ (t )||t |α dt → 0,
(3d)
−δ
are satisfied. Thenthe relation 


m
m
Y

Y

b T ,∆ (τj ) → E 
E 
W
A (τj )



j =1
j =1
holds for any m ∈ N and any τ1 , ..., τm ∈ R.
b T ,∆ =⇒ A by given character of tending T and
In particular, W
∆ to infinity.
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
b T ,∆
Asymptotic normality of the distributions of W
Example functions that satisfy the balance conditions
b
Asymptotic normality of the distributions WT ,∆ in C[a , b
Example 2
Let g ∈ L1 (R) and for some α ∈ (0, 1] H ∈ Lipα (R) ∩ L2 (R).
Spectral densities
!
c
λ2
f∆ (λ) =
exp −
, λ ∈ R,
2π
∆
and correlation functions r
∆t 2 c ∆
K∆ (t ) =
exp −
, t ∈ R,
2
π
4
of the processes X∆ from Example 1 satisfy the balance
conditions (3a) - (3d), if T → ∞, ∆ → ∞ in such a way that
T ∆−α → 0.
Thus, all conditions of Theorem 1 are fulfilled, and it follows
b T ,∆ =⇒ A .
W
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
b T ,∆
Asymptotic normality of the distributions of W
Example functions that satisfy the balance conditions
b
Asymptotic normality of the distributions WT ,∆ in C[a , b
In addition to Theorem 1 it is natural to study the asymptotic
b T ,∆ in the space of continuous functions.
normality of W
Consider the function [V. Buldygin and I. Blazhievska (2009)]
"Z ∞
# 12
2 τλ
∗
2
sin
σH ,g (τ) =
|H (λ)| + g (λ) d λ , τ ≥ 0.
−∞
2
Since H ∈ L2 (R) and g ∈ L1 (R), then this function is well-defined
and generates two pseudometrics:
q
√
σ(τ1 , τ2 ) = σH ,g (|τ1 −τ2 |), σ(τ1 , τ2 ) = σH ,g (|τ1 − τ2 |), τ1 , τ2 ∈ R.
Let
Hσ ([a , b ], ε), H √σ ([a , b ], ε), ε > 0,
be the metric entropies of [a , b ] with respect to σ and
respectively.
Blazhievska I.P.
√
σ
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
b T ,∆
Asymptotic normality of the distributions of W
Example functions that satisfy the balance conditions
b
Asymptotic normality of the distributions WT ,∆ in C[a , b
Theorem 2
Assume that g ∈ L1 (R); for some α ∈ (0, 1] H ∈ Lipα (R) ∩ L2 (R)
and ∀[a , b ] ⊂ R the inequality
Z u
H √σ ([a , b ], ε) d ε < ∞,
∀u > 0
(4)
0
is satisfied. Then the following statements hold true:
I) A ∈ C [a , b ] almost surely;
b T ,∆ ∈ C [a , b ] almost surely, T > 0, ∆ > 0.
II) W
Moreover, if T → ∞, ∆ → ∞ in such a way that the balance
conditions (3a) - (3d) are satisfied, then
C [a ,b ]
b T ,∆ =⇒ A
III) W
and, in



 particular, for all x > 0








√


bT ,∆ (τ) − H (τ) > x  → P  sup A (τ) > x 
T sup H
.
P








τ∈[a ,b ]
τ∈[a ,b ]
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
b T ,∆
Asymptotic normality of the distributions of W
Example functions that satisfy the balance conditions
b
Asymptotic normality of the distributions WT ,∆ in C[a , b
Remark 1
By Dudley’s Theorem, it follows that the statement I) of
Theorem 2 holds true under a weaker condition than (4),
Z u 1
namely:
Hσ2 ([a , b ], ε) d ε < ∞.
∀u > 0
0
In particular, the last relation always holds if there exists β > 0
such that Z
∞
|H ∗ |2 + g (λ) log1+β (1 + λ) d λ < ∞.
0
Remark 2
The inequality
Z ∞(4)
holds if there
exists β > 0 such that
∗2
|H | + g (λ) log4+β (1 + λ) d λ < ∞.
0
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Sketch proof of Theorem 1
√
b T ,∆ = ( T [H
bT ,∆ (τ) − H (τ)], τ ∈ R)
Let us represent the process W
as the sum
b T ,∆ = A
bT ,∆ + B
bT ,∆ ,
W
√
where
b
b
b
AT ,∆ (τ) = T HT ,∆ (τ) − EHT ,∆ (τ) ;
√ bT ,∆ (τ) = T EH
bT ,∆ (τ) − H (τ) .
B
b T ,∆ are
Thus, the asymptotic properties of the process W
characterized by properties of the stochastic zero-mean process
bT ,∆ and the function B
bT ,∆ as T , ∆ tend to infinity.
A
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Sketch proof of Theorem 1
bT ,∆ in the
In order to eliminate the dependence of B
b T ,∆ = A
bT ,∆ + B
bT ,∆ , we apply the following
representation: W
result:
Lemma 1 (V. Buldygin and Fu Li (1997))
Let α ∈ (0, 1]; H ∈ Lipα (R) ∩ L2 (R) and T → ∞, ∆ → ∞ in such a
way that the balance conditions (3a) - (3d) hold true, then:
bT ,∆ (τ) → 0;
I) ∀τ ∈ R
B
bT ,∆ (τ)| → 0.
II) ∀[a , b ] ⊂ R
sup |B
τ∈[a ,b ]
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Sketch proof of Theorem 1
bT ,∆ we
To prove the asymptotical normality of the process A
apply the Brillinger’s method of cumulants [D. Brillinger (1980)]
reinforced by the results concerning the integrals involving
cyclic products of kernels [V. Buldygin, F. Utzet, and V. Zaiats
(2002, 2004)].
Thus, we establish the following main statement:
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Sketch proof of Theorem 1
Lemma 2
Let g ∈ L1 (R) and H ∈ L2 (R), then for any m ∈ N and
τ1 , τ2 , ..., τm ∈ R

0,
m = 1;



b
C (τ , τ ), m = 2;
lim cum(AT ,∆ (τj ), j = 1, ..., m) = 

 ∞ 1 2
T ,∆→∞
0,
m ≥ 3,
bT ,∆ (τj ), j = 1, ..., m) is a joint simple cumulant of
where cum(A
bT ,∆ (τj ), j = 1, ..., m, and
ramdom variables A
C∞ (τ1 , τ2 ) =
=
1
2π
Z
∞
e i (τ1 −τ2 )λ |H ∗ (λ)|2 +
−∞
2π
g (λ) + e i (τ1 +τ2 )λ (H ∗ (λ))2 d λ,
c
is the correlation function of the Gaussian zero-mean process A .
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Sketch proof of Theorem 1
Lemma 2 and the formula of the connection between moments
and cumulants of the distribution, imply that for any m ∈ N and
τ1 , τ2 , ..., τm ∈ R
m
m
Y
Y
b
lim E
AT ,∆ (τk ) = E
A (τk )
(5)
T ,∆→∞
k =1
k =1
bT ,∆ =⇒ A as T , ∆ → ∞. The statement of
In particular, A
Theorem 1 follows from the representation




1
m
m
Y

Y

X
k
1
−
k
j
j



b
b
b



E 
WT ,∆ (τj ) = E
AT ,∆ (τj )BT ,∆ (τj ) =



k1 =0,k2 =0,...,km =0 j =1
j =1
=
1
X
k1 =0,k2 =0,...,km =0


m
m
 Y
 Y
kj
E

b1−kj (τj ),
b
A
(τ
)
B

T ,∆ j 
T ,∆

j =1
j =1
Lemma 1 and the relation (5).
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Sketch proof of Theorem 1
H. Akaike, On statistical estimation of the frequence
response function of a system having multiple input, Ann.
Inst. Statist. Math. 17 (1965), 185–210.
R. Bentkus, On asymptotic normality of the estimator of
the spectral function, Lithuanian Math. J. 12 (1972), no. 3,
3–17.
V.V. Buldygin and I.P. Blazhievska, On asymptotic
behavior of cross-correlogram estimators of response
functions in linear Volterra systems, Theory of Stochastic
Processes. Vol. 15 (31), no. 2 (2009), 84–98.
V. V. Buldygin and Fu Li, On asymptotical normality of an
estimation of unit impulse responses of linear systems I, II,
Theor. Probab. and Math. Statist. 54 (1997), 17–24; 55
(1997), 29–36.
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Sketch proof of Theorem 1
V. V. Buldygin and V. G. Kurotschka, On cross-correlogram
estimators of the response function in continuous linear
systems from discrete observations, Random Oper. and
Stoch. Eq. 7 (1999), no. 1, 71–90.
V. Buldygin, F. Utzet, and V. Zaiats, Asymptotic normality
of cross-correlongram estimates of the response function,
Statistical Interference for Stochastic Processes 7 (2004),
1–34.
A.A. Dykhovichny, On estimate of the correlation function
of a homorgeneous and isotropic Gaussian field Theor.
Probab. and Math. Statist. 29 (1983), 37–40.
R.M. Dudley, Sample functions of the Gaussian process,
Ann. of Probab. 1 (1973), 66–103.
Blazhievska I.P.
Some properties of cross-correlogram estimators
Introduction and preliminaries
Main results
Sketch of proofs
Sketch proof of Theorem 1
J. S. Bendat and A. G. Pirsol, Engineering Applications of
Correlation and Spectral Analysis, "Mir Moskov, 1983 (in
Russian).
P. Billingsley, Convergence of probability measures,
"Nayka Moskov, 1977 (in Russian).
D. R. Brillinger, Time series. Data analysis and theory,
"Mir Moskov, 1980 (in Russian).
V. V. Buldygin and Yu.V. Kozachenko, Metric
charactirization of random variables and ramdom processes,
American Mathematical Society, Providence, RI, 2000.
R. E. Edwards, Functional analysis: theory and
applications, New York, Holt, Rinehart and Winston, 1965.
N.N. Leonenko and A.V. Ivanov, Statistical analysis of
random fields, "Vuscha shkola Kiev, 1986 (in Russian).
Blazhievska I.P.
Some properties of cross-correlogram estimators